DOCUMENT

ED 029 679

52

RESUMELI 001 535

By-Cunningham. Jay L.; And Others
A Study of the Organization and Search of Bibliographic Holdings Records in On-Line Computer Systems:
Phase L Final Report.
California Univ.. Berkeley. Inst of Library Research.
Spons Agency-Office of Education (DHEW). Washington. D.C. Bureau of Research.
Bureau No- BR-7-1083

Pub Date Mar 69
Grant- OEG-1-7-071083-5068
Note- 307p.
EDRS Price MF-$1.25 HC-$15.45
Descriptors-`,Qtomation. Bibliographic Citations. Catalogs. Computer Programs. Computer Storage Devices.
Costs. *Information Processing. Information Retrieval. Information Storage. Information Systems.
Libraries. Library Technical Processes. Search Strategies. Systems Development

This report presents the results of the initial phase of the File Organization
Project. a study which focuses upon the on-line maintenance and search of thelibrary's catalog holdings record. The focus of the project is to develop a facility for
research and experimentation with the many issues of on-line file organizations andsearch. The first year has been primarily devoted to &fining issues to be studied.
developing the facility for experiment, and carrying out initial research on the issues.
Achievements involved: (1) obtaining equipment: (2) programming and testing an initialsoftware system. and then expanding it to supply access to the central processor
from two different mechanical terminals at two remote locations: (3) planning for
acquisition and incorporation of an existing machine file as well as bibliographic
records which require original conversions: (4) developing software for data basepreparation and for file handling and access: and (5) initiating analyses on issues
such as optimum length of search keys. Appended are six reports which cover
specific aspects of the project and an article entitled *The Organization. Maintenanceand Search of Machine Files." reprinted from *The Annual Review of Information
Science and Technology:2 volume 3. (JB)

FINAL REPORT
Project No. 7-1083
Grant No. OEG-1-7-071083-5068

A STUDY OF THE ORGANIZATION AND SEARCH OF BIBLIOGRAPHIC HOLDINGSRECORDS IN ON-LINE COMPUTER SYSTEMS: PHASE I

By
Jay L. Cunningham
William D. Schieber
andRalph M. Shoffner
Institute of Library Research
University of California
Berkeley, California 94720
U.S. DEPARTMENT OF HEALTH, EDUCATION it WELFARE
OFFICE OF EDUCATIONTHIS DOCUMENT HAS BEEN REPRODUCED EXACTLY AS RECEIVED FROM THE

PERSON OR ORGANIZATION ORIGINATING IT.

POINTS OF VIEW OR OPINIONSSTATED DO NOT NECESSARILY REPRESENT OFFICIAL OFFICE OF EDUCATIONPOSITION OR POLICY.

March 1969

The research reported herein was performed pursuant to a grantwith the Office of Education, U.S. Department of Health, EducaContractors undertaking such projects undertion, and Welfare.
Government sponsorship are encouraged to express freely theirprofessional judgment in the conduct of the project. Points ofview or opinions stated do not, therefore, necessarily represent
official Office of Education position or policy.

U.S. DEPARTMENT OFHEALTH, EDUGATION, AND WELFARE
Office of Education
Bureau of ResearchTABLE OF CONTENTS

List of Figures

Acknowledgments
I.

II.

III.

IV.

INTRODUCTION AND SUMMARYA.

The Research Problem

1

B.

Research Method

3C.

Significant Findings and Achievements.

D.

Future Directions.

FACILITY ESTABLISHMENT

7

A.

General7

B.

Fquipment

12C.

Computer Programs

14

D.

Data Base Development

20

E.

File Structure

21

THE BIBLIOGRAPHIC RECORD

35

A.

General

35

B.

Record Content

37

C.

Record Form

38

D.

The Representation of Typographical Characters48

E.

Logical Similarity of Bibliographic Records.

60

75

DATA BASE DEVELOPMENTA.

General75

B.

Strategies of Conversion

77

C.

Translation of Existing Machine Files,

D.

Data Base Production Procedure

94

E.

Issues of Cost and Quality

107

References

.

85

115

TABLE 07 CONTENTS,(Cont.)Laza
APPENDIX I:APPENDIX II:APPENDIX III:

APPENDIX IV:

APPENDIX V:

APPENDIX VI:

APPENDIX VII:

AN ALGORITHM FOR NOISY MATCHES IN CATALOGSEARCHING, By James L. Dolby

117

USER'S GUIDE TO THE TERMINAL MONITOR SYSTEM(WS), By William D. Schieber

137

A DESCRIPTION OF LYRIC, A LANGUAGE FOR
REMOTE INSTRUCTION BY COMPUTER, By
Steven S. Silver

145

ILR PROCESSING RECORD SPECIFICATION, By
Jay L. Cunningham

163

SUMMARY OF RECORD FORMATS FOR DATA BASESTO BE CONVERTED TO ILR PROCESSING RECORDFORMAT
1.

Santa Cruz Record Format

193

2.

ILR Input Record Format

207

3.

Experimental On-line Mathematics
Citation Data Base

263

SAMPLE SIZE DETERMINATION FOR DATACONVERSION QUALITY CONTROL, By
Jorge Rodriguez

271

THE ORGANIZATION, MAINTENANCE AND SEARCH
OF MACHINE FILES, By Ralph M. Shoffner(Published in the Annual Review ofInformation Science and Technology, v. 3,
edited by Carlos A. Cuadra. Chicago,
Encyclopaedia Britannica, Inc., 1968.
pp. 137-167)

277-

LIST OF FIGURES

Title

FigureSECTION II:LEIFACILITY ESTABLISHMENT

1.

Schematic Diagram of Project Facility

2.

File Generation Process173.

Blocking Strategy

24

4

Block a Non-Keyed File25

5.

Uniqueness of Author Identification

30

6.

Distribution of Number of Fields of Length N

31

7.

Schematic of Multi-level File Structure Linkage

SECTION III:
8.

8

.

.

THE BIBLIOGRAPHIC RECORD

Functional System Components Related to Different
Record Formats

9.

36

MARC II Elements Deferred in File Organization
Project Data Base

10.

33

39

File Organization Project Data Elements Not
Defined in MARC II

40

11.

Coding Sheet - Monographs

46

12.

Keying Blocks of Text

52

13.

The Tentative Harvard List of Diacritics

56

14.

Alphabetical Index of Diacritic Codes

15.

Proposed Single Keying Codes Compatible withTransliteration Schemes for Modern Cyrillic
16.

57-58

59

A Spelling Equivalent Abbreviation Algorithm for
Personal Names (Dolby Version 1 - Variable Length).

65

17.

Equivalence Class Computation (Manual)

66

18.

Equivalence Class Computation (Computer)

68

19.

Abbreviation Algorithm for Personal Names(Version 2 - Fixed Length)69

LIST OF FIGURES (Cont.)

FigureTitle

DATA BASE DEVELOPMENTSECTION IV:
20.

Page

Distribution of DupliLate Titles as a Functionof Publication Date:
A.

Titles in English Language

B.

Titles in Languages Other Than English That86

Use a Roman Alphabet
C.

87

Titles in Languages That Use a Non-Roman
Alphabeb

21.

87

Conventional Conversion Compared to Automatic
Format Translation and Computer-Assisted Editing.

.

89

22.

Plow Chart of Personal Author Field Algorithm

.

91

23.

Flow Chart of Title Field Algorithm

24.

Summary Chart of Data Base Production

25.

On-line Search for Duplicates

97

26.

Verification of Match

98

27.

Data Preparation and Transcription

100

28.

Computer Edit, Correction Cycle, and File Update.

101

29.

Diagnostic Printout, Part 1 - Logical Field

.

.

92-93
95

.

Listing

105

30.

Diagnostic Printout, Part 2 - Card Image Listing.

106

31.

Schematic of Quality Control Subsystem

108

32.

Relation of Initial Keying Cost to Accuracy

33.

Acceptability in Terms of Accuracy and Cost for
Three Price Quotations for Keying

APPENDIX IV:

.

.

113

114

ILR PROCESSING RECORD SPECIFICATION

1.

Indicator for Main Entry - Personal Name

166

2.

Storage Record Organization

166

- iv-

(Cont.)LIST OF FIGURES

Page

Title

168

3.

Schematic of ILR Storage Record, INFOCAL Version 1

4.

ILR Processing Record - Segment 1, Leader

169

5.

ILR Processing Record - Segment 2, Record Directory

172

6.

ILR Processing Record - Segment 3, Fixed Length.

173

Data Elements
7.

Variable Field Tags and Data Elements

177

8.

Values for Indicator 1 in Applicable Fields

180

9.

Sub-Field Delimiter Codes

184

Proposed Variable Field Header

188

10.

APPENDIX V-1:
1.

SANTA CRUZ RECORD FORMAT

Sample Catalog Record in Original Santa Cruz Format.

APPENDIX 11-2:

195

ILR INPUT RECORD FORMAT

Storage Record Components & Organization

208

1.

Structural Patterns in MARC Record Data Definition

211

2.

3.

Example of Input Format Mapping Into Processing
213

Format4.

Example of Tab Card Decklet - ILR Input Fonuat

5.

ILR Input Record Format - I-Fields:.

.

Data Elements
219& Codes6.

ILR Input Record Format - A-Fields:

Data Elements
224& Codes7.

ILR Input Record Format - B-Fields:

Data Elements
226& Codes8.

Input Code Values Table for Type of Main Entry

9.

Input Code Values Table for Type of Added Entries

.

232

233

(Series Traced Same)
10.

217

Input Code Values Table for Type of Added Entries
234(Subject Added Entries)

=i

IMO

LIST OF FIGURES (Cont.)

FigureTitle

11.

Input Code Values Table for Type of Added Entries
236(Other Added Entries)
12.

Page

Input Code Values Table for Type of Added Entries(Series Traced Differently)

238

13.

Presence of Fields in an Input Record

239

14.

INFOCAL Default Initializations

241

15.

Default Settings for Indicator 1

243

16.

List of Tag Numbers Which are Currently Repeatable.

244

17.

Revised Field Coding:

18.

Table of Valid Symbols

APPENDIX V-3:

A-Fi3lds & B-Fields

246

250

EXPERIMENTAL ON-LINE MATHEMATICS CITATIONDATA BASE

1.

Cards Punched for One Paper Published in Vol. 66of the Communications in Pure and Applied Mathematics

268

ACKNOWLEDGMENTS

This report comprises the results of the first year ofeffort under a grant, 0EG-1-7-071083-5068, from the Bureau of
Research, Office of Education, U.S, Department of Health, EducaThe content and conclusions presented in thetion and Welfare.
The
report pertain to the period July 1, 1967 - June 30, 1968.
University of California also provided contributory support.
M.E. Maron, Associate Director of the Institute, acted as Principal Investigator and Ralph M. Shoffner as Project Director.
For constructive criticism concerning goals and methods, andfor otherwise inaccessible information, we are especially gratefulThe
to the members of the project's Consultant Advisory Panel.
Mrs. Henriette D. Avram, Library ofmembers of this Panel were:
Congress; Richard Dougherty, University of Colorado Libraries;
Anthony Hall, UCLA University Research Library; Foster Palmer,
Harvard University Libraries; Charles Payne, University of Chicago
Libraries; Charles Stevens, Project INTREX, Massachusetts Instituteof Technology; and Allen Veaner, Stanford University Libraries.
Valuable consultant support in the development of an algorithm
for searching through "noise" was also received from James L. Dolby,of The R & D Consultants Company, Los Altos, California.
Our campus Computer Center has been most helpful in planningand obtaining the system needed to support the project. Our
initial monographic data base was supplied through the cooperationof Donald Clark, the University Librarian of the Santa Cruz campus,
without whose help we could not have had access to such a large
file.
The data base of the citation index in mathematics was
supplied by Mary Tompkins of the Institute's Los Angeles staff.
We would also like to give credit to the individual contributions made by staff members and research assistants of theInstitute, in particular Harriet Zais, who performed statisticalstudies of the data base and who supervised the pilot testing of
the input coding system; and Thomas Hargrove, who designed thescheme for conversion of foreign language materials and much of
the procedure for the data base production.
Technical studies
were produced by Jorge Rodriguez and Naresh Kripalani. Programming
support was rendered by Arjun Aiyer, Regina Frey, John Reinke andSteve Silver. Input editors who got the original conversion task
started were Mrs. Ann Giglioli, Diane Kristell, Lucy Liang,
Elizabeth Poole, Janet Redd, Roberta Roberts, and Douglas Romney.
Finally, we would like to thank the Institute office staff
who were instrumental in the physical preparation of this report:
Patricia Barkeley, Linda Child, Kitty Colburn, Betty Geer,
Marion Gordon, Judith Sutliff, and Connie Torii.
J.L.C.
W.D.S.
R.M.S.

.64.

I.

A.

INTRODUCTION AND SUMKARY

THE RESEARCH PROBLEM

Prologue. This is a period of major intellectual andtechnological change in libraries. Among the many aspects ofthis change, the applicatior of computers to library operations
is of recognized importance. Much effort is currently being
directed toward such use of computers. For example, there area growing aunber of files of bibliographic records which arebeing provided in camputer processable form by both public andprivate organizations. Most forecasts of the library of the
future include on-line computer use as a key feature of thelibrary's operation. Attractive as such forecasts may appear,
there are numerous unresolved and often ill-defined problems
which must be solved before on-line systems can be effectively
applied to the acquisition, maintenance, and retrieval ofrecorded information. This report presents the results of theinitial phase of a research study, the File Organization Project,which focuses upon the on-line maintenance and search of thelibrary's central apparatus of bibliographic control, the catalog hdldings record.
1.

Machine files are being sought and created as a feasible
solution to the problem of growth and complexity in the catalog,
as a response to the need for new and expanded services, fortheir speed and convenience in access, and to replace human laborin generating products such as book-form catalogs and bibliographies. What are the dimensions of the task of acquiring,
maintaining and using the very large data base? Why is it
desirable or even necessary that library files be accessible
on-line? What techniques can be recommended as efficient, economical and acceptable for organizing and searching large files
in operating contexts?
The Economic Im lications. This question about techniques can be expanded into other questions. What methods
should be used to encode record content in the various partsof the system -- input processing, storage, output? Whatshould the file structure be, that is, how will the records
be mapped irto the physical storage and how should they berelated to each other? What should be the form and capabilityof the search language? What post-retrieval analysis andprocessing capability should be provided to the system 'user?
2.

To select appropriate techniques in answer to thesequestions one must face all of the issues of evaluation which
are common to information retrieval research, that is, bysome method the cost cf the system must be considered in- 1 -relation to its performance. Leaving aside the problems ofestablishing the system cost, there remain myriad problems toFor
be resolved in establishing the performance of the system.
example, how does one measure the effect of a system upon its
user? Can recall and relevance or similar such measures beapplied? Should one consider in the performance of the system
the "non-users" of the system, or the questions that for various
reasons are not or cannot be asked of it? There are only certain
aspects of performance that can be measured. The problem thenis to establish a relationship anong them in order to provide
a suitable measure; for example, how can system capacity, actual
use, average response times, response time variations, etc., becombined in a performance measure? These questions can be in turnexpanded into still other questions. However, though they are
but a sample of the questions which do not at present haveoperational answers, they are among the most important and they
form the context for our study.

On-line time-shared computer systems present many desirable
features for library use: parallel operations in multiple
locations, response times to complex queries measured in minutes
or seconds, reduction in the manual labor required for the
maintenance of catalogs, etc. At the same time, man-effort isrequired to design and implement these systems. Further, theoperating costs of an on-line system are high when compared tothose of a computer system organized for conventional batchprocessing. That is, a batch processing operation can always
be organized in such a way as to be less expensive to operate
than an on-line system, on the condition that no cost be
assigned to the processing delay incurred by the batch system.

An estimate of the operating cost of an on-line system
for a library can be obtained by considering the major cost
factors of the system: the terminals, the mass-storage forthe bfbliographic records, and the computer time used. The
costs of terminals and the interfacing equipment necessary
to attach them to the central processor vary widely depending
upon type of gear, total number of terminal units, data transferrate, distance fram the processor installation, and the like.
For purposes of rough estimation, however, a reasonable system
operating cost lies in the region of $2000 per terminal per
year. Due to the size of the bibliographic record and thecharacter of present auxiliary mass storage devices, the costof storing the data equtya1ent to that of one catalog card
ranges from $0.10 per record per year, to $0.60 or more. The
cost of the central processor allocated to the terminal network
during the time it is in use is similarly highly variable. At
present, these costs range from $2 per terminal hour to $10 andbeyond.

if

A library with 100,000 master bibliographic records to beaccessed in an on-line mode might need 20 terminals or more withan average utilization of 1200 terminal-hours per terminal per
Thus, the following minimum costs for such a configurationyear.
can conceivably be incurred:
Terminals

20 @ $2000/yr.

$ 4o,000

Mass Storage
Capacity

100,000 recs.
@ $0.30 ea.

30,000

CPU Time
Charges

20 term. x $12001
term. yr. @$2.00/
term. hr.

48 000TOTAL$118,000/yr.

Due to the fixed sizes of the various central processors
and auxiliary storage units now available, it is most likely
In any event,that the actual charges would be higher still.
it is clear that such costs are significant. There will bereal monetary impact on libraries resulting from improvementsin the processing capacity of the computer system employed tomaintain and search the bibliographic files.
Should libraries undertake the development of such systems?
The question has in a sense become moot in that there are alreadyso many on-line system development efforts. Rather than attempt
to answer this very broad question, the File Organization Projectis directed toward an analysis and understanding of specificissues of organization and search if an on-line system is used.
Such understanding should then contribute to the establishmentof the most approprate blend of camputer services for a given
library.
B.

RESEARCH METHOD

The focus of our efforts in the File Organization Projectis to develop a facility for research and testing, one within
which experimentation with the many issues of on-line fileIn this first phaseorganization and search can be performed.
of the project we have been concerned primarily with the designof the facility and the implementation of its initial components.
In addition, to ensure the development of an adequate facility
we must establish central examples of the crucial issues which
are to be investigated. Thus, the research method has had thefollowing points: to define precisely the issues which will be
studied, to develop the facility within which such issues canbe studied, and to carry out the initial research on theseissues.
MM.

3

Our approach has been to organize both the tasks necessary
to the establishment of a usable facility for testing, and thetasks directly assigned to attacking the experimental issues insuch a way as to keep the tasks manageable and within reasonableboundaries, hopefUlly without sacrificing the quality of theultimate results of the project. The elements of the facility
have been decamposed into separate units for ease and speed ofaccomplishment, particularly where programming is involved. The
research issues have been compartmentalized and broken down into
isolable units so that their respective dimensions and sub-problems are delineated for the staff member assigned as problem
solver. An insight into the effectiveness of this approach canbe gained by a simple enumeration of the significant findings
and achievements of the first year.
C.

SIGNIFICANT FINDINGS AND ACHIEVEMENTS
1.

Facility for Experiment.

General. Equipment was obtained and an initialsoftware system was programmed and operated on a test basis todemonstrate capacity to search (by author only) a file of 75,000
catalog records on-line. The system was expanded to supply
access to the central processor fram two different mechanicalterminals at two remote locations on the Berkeley campus. Massstorage equipment with capacity in excess of 200 million characters was proaured. The necessary adaptation units for interfacing with remote terminals were installed at the campus
computer center to support the activities of the project. Plans
were laid for extension and upgrading of the system to handle
the complex requirements of visual (CRT) terminals, now on order.
a.

Programs. File structure design was specified
and programmed for an initial configuration of multi-levelindex files linked to the master data base, using manufacturer
supported software. Search strategy development was initiatedand preliminary programming of Boolean search programs commenced.
b.

Data base. Plans were completed for acquisition
and incorporation of an existing machine file and for the incorporation of bibliographic records requiring original conversions.
We expect that within a year our data base will contain inexcess of 120,000 records.
c.

A logical record format for input conversion of catalog
records not in the existing machine file was designed and a
program written to accomplish conversion to a unitary storage
format.

A storage format was designed having special features tohandle special characters and diacritical marks in any language
likely to appear in textual data, yet retaining convertibility
to and from the Library of Congress' MARC II communications
format.

Analytic Issues. Analyses were initiated on a numberof the most pressing issues, in particular the optimum lengthof search keys, e.g., author name. This analysis will be usedin establishing index files and in guiding users in the minimization of keyboarding time and effort. This analysis represents
the first step in deciding how to allocate storage for both themaster file and the index files, based on the various criticalfactors such as frequency of access, equipment access time, andstorage costs.
2.

User aids were under development, in particular an algorithm
which will serve as the core of a sub-system C.esigned to process
and assist the terminal user in overcoming the effects of spelling
errors. Analysis of these aids will follow their development.
D.

FUTURE DIRECTIONS

The general purpose of setting up the facility is to perform analyses and experiments in on-line file organization andsearch of bibliographic records. Experimental facilities areoften needed to collect data and test hypotheses. In the present
instance such a facility is especially important in order torecord information about factors such as terminal user behavior
and the frequency of recurrence of requests, as well as to test
our analytical models Against actual system operations.
This facility will enable us to experiment with users who
have real needs for information and thereby to relate ourresearch to its operational implications. To accomplish this,we will place terminals in areas where they can be used bylibrary patrons.
Both the placement and the period of use of
these terminals will be determined by the experiments planned.
It is anticipated that these terminals will remain in place forat least three months to allow time for patrons to learn how
to use them.
A number of conditions must be fulfilled before such "live"
testing can be undertaken. The three most important are therealization of the basic search and retrieval system organizedaround the visual (CRT) displays and the basic internal record
formats, the internal (to ILR) experimental use of this system,and the mcdification and improvement of the system prior to its
experimental use by library patrons. We expect that at leasta year and a half will be required before making the system
available for experiments with users outside the Institute of5

Library Research. This is because of the need for time to makethe system relidble and its use easy enough for it to be acceptedby library users. Thus, the second phase of the study will consist primarily of a continuation of the "setting up" activities
concerned with the facility and the associated support components,
such as software for data base preparation and for file handlingand access. In addition, a considerable amount of development
and analysis is needed with respect to the following:

To expand the internal processing format to accommodate
augmented bibliographic records for non-monographmaterials.
To implement a data compression method to convert backand forth from internal processing format to massstorage format.
To expand the formats to encompass Cyrillic records.
To implement a reasondbly efficient program system as
a foundation for the ready development of userinterface programs.
To implement user assistance routines such as those
providing proper name equivalence classes.
To implement storage allocation algorithms based uponquantitative analysis of the file parameters.
Such a list can be indefinitely expanded. However, allof the dbove have been given at least initial effort during
the current phase aLd we hope to accamplish work on all ofthem during the coming phase.

II.

A.

FACILITY ESTABLISHMENT

GENERAL

A facility for experimenting with the on-line organizationand search of bibliographic records, requires equipment, programs, and data bases. Work has been performed during the firstphase to plan this facility and to begin development of it.
Inaddition, some analysis using the facility has been initiated.
An overview of the facility and its major components is schematically depicted in Fig. 1
EsuLipmalL We plan to implement the facility on two
closely related sets of equipment: the IBM 360 model 40 computer system with mass storage and cathode ray tube display
terminals at Berkeley; and the IBM 360 model 75 with similar
peripheral equipment at Los Angeles. At present we are utilizing
two mechanical terminals in Berkeley. The Los Angeles system
has IBM 2260 displays.
1.

2.

Programs. Initial programming has been performed on
many aspects of the system. We will continue this programmingand implement it on both computer systems. The following are
examples of programming that has been done or that is planned:

Monitor system. At pre:Lent, monitor systems toprovide terminal operations are operative at both Berkeley and
Los Angeles. The systems are not designed to the same conventions, however. In order to implement common programs td
operate within these systems, we plan to modify the Berkeley
monitor system to bring it into conformance with the more advanced Los Angeles monitor.
a.

b.

Data base programs. Programs are under development
both to provide the data base (both records and file structure)and to search it. These programs are now being used for monographic files and will be used for journal article records.
Initially the file access programming has been restricted tothe development of a basic search alld retrieval system utilizing
temporary formats for monograph and journal article records.
Following this, we will extend the system to utilize our internal
storage format. This format has been fully defined for monographic records. However, it will be extended to include othertypes of records in the file structure.

Programming of file handling operations will continue
throughout the study. As soon as the basic search system isin full operation, however, we will make it available forinternal (to ILR) use in the data base development effort. We
will augment the existing computer programs, dividing theeffort between the internal file handling operations and theuser interface procedures. To guide part of this augmentation,
7 -FIGURE 1:

SCHEMATIC DIAGRAM OF PROJECT FACILITYCOMPUTER CENTER_

IDATA BASE CREATION & MAINTENAMMO

Data
Base
(Mass
Storage)

LC MARC tapes
Santa Cruz tapes
etc.

Computer

L

FILE HANDLING & SEARCH(ON-LINE MODE)

Mlimo

1111

OMNI.

=YON

?bpec. Proaessors/Utilities (Level II)

kr

,,
tr,

REMOTE LOCATIONS(e.g., limit

Terminal

IUter

Terminal

IUser

Terminal

IUser

DATA BASE
PRODUCTS,ANALYSES,ETC.

(BATCH MODE)

liograph es
etc.)

Files çorData Ei-changqCommiications Format)- 8 -we are now developing a quantitative analysis of the effect offile structure on retrieval operations. We will then develop
programs to set up the alternative file structures which ouranalysis indicates to be of interest.
As soon as our basic search and retrieval system is fully
available, we will monitor the performance of the system users.
In this monitoring process, we will keep a precise machinereadable record of the two-way interchange between the user
and the system. We are now planning how to keep and identify
this "historc record so that useful analyses can be performed.
As an example, one such investigation to be carried out isthat of determining common errors which are made in keying
In addition to using the terminal systembibliographic data.
to obtain data on error, we will study our off-line data baseinput operation to analyze the nature of errors which are madein conversion keying. Following this we will develop automatic
correct4.on routines and study their behavior. While the correction routines would have to be combinatorial in nature fthey are intended to be exhaustive, it is our hypothesis thatcertain types of errors occur with considerably higher frequency
than do others, and therefore useful heuristic routines can be
developed even though they may ba partial in their effect.
Such analyses are expected to lead to the development ofroutines which can assist the non-expert user in obtaining the
Therefore, we anticipate programming
materials he desires.
these routines and incorporating them as part of the searchsystem available to the general user.
We anticipate that large amounts of programming will be
required to provide the user interaction programs. To support
this programming we are developing the LYRIC processor (atLos Angeles).* This processor will facilitate the rapid development of the user interface programs. PILOT, a similar processor,
is being programmed in PL/1 at UC, San Francisco. Although we
anticipate that LYRIC will be preferable, we will experimentwith both processors.
Analytic programs. To support our research, many
special-purpose programs will be required. The analysis of error
mentioned above will require programs of this type. Others will
be concerned with analysis to investigate the issues discussed
in the Introduction, such as record encoding, file structure, etc.
These will continue to be defined Lnd developed throughout thec.

study.

Data Base Progress and Plans. The data base development, initiated during the current phase, will be continued.
The monograph records will comprise the largest portion of thedata base. As an initial data base for the project, we have3.

*See Appendix III.

L.

9

utilized 75,000 catalog records which were available in machine
form from the Library of the University of California Santa Cruz
campus, and 40,000 records of the Mathematics Citation Index
available at Los Angeles. These records have been utilized intheir origiLal format. However, we have been developing record
formats to be used on a continuing basis for the facility. The
record formats have been developed to be compatible with the
Library of Congress' MARC II format in order to provide thegreatest likelihood that the records and programs will be ofuse to others. The results of this work are presented in
Section III.

We will continue to develop our data base in order toobtain a file of a size that can support meaningful experimental
work with regular library users. Our objective is to obtainan on-line file of at least 500,000 bibliographic records.
During the second phase of the project we hope to complete theestablishment of a file containing a minimum of 1/4 million
records, from three sources: by obtaining MARC II records, byoriginal input, and by converting the Santa Cruz records.
We have developed procedures for original input of monographic records. We are considering a production plan which
utilizes both on-line terminals for the search of the existingdata base and off-line terminals for the large scale keying oforiginal input. We have defined a statistical quality control
procedure for maintaining the accuracy of this input process.
During Phase II we plan to test, modigy and re-test bcth theextraction and the original input procedures.
Because there are a number of existing bibliographic files
in different machine formats, we have been investigating the
problem of utilizing such files through the method of routines
for automatic conversion of such files to a common format. Our
first task has been to develop computer routines for converting
the Santa Cru'd., format (a relatively simple format) to the MARC II
format (a complicated format). Through of dubious practicalvalue unless the file to be converted is of considerable size (webelieve it to be in excess of 100,000 records), the problem is of
considerable research interest since its basis is the recognition
by computer of the components of a bibliographic entry.

Though data base development is not one of the central
concerns of our study of on-line file organization and search,
there are significant problems involved in such development,
and the inescapable requirement to develop a data base has
immersed us deeply in these problems. BecAuse many of these
problems are of general interest but are not dependent upon
the rest of our work, we have dealt with our solutions to themseparately in Section IV.
4.

Ana1ysis of File Structure. An analysis of filestructure was initiated recently. Our objective is to create- 10 -

an analytical model of the retrieval system, which will delineate the relationships between access time, the cost of
direct-access storage and the strategy for allocating space.
Initially this study will be limited to trade-off analyses
between a few variables. We will then test the model byorganizing the file in the manner indicated and check the outcome of the predictions. This model will, for example, provide
information on when it is more economical to divide a logicalindex file into two or more physical index files, in massstorage. To support this work, we have observed the relationshipbetween the length of the beginning portion of an identification
key (e.g., author name) and the degree of uniqueness thisidentification provides in the retrieval of records from theSanta Cruz file. For certain purposes, the key might ultimately
be composed of portions of several fields of the record (seeChicago search code*).

In order to make it possible to experiment with alternative
file designs, we must have flexible programs which are capableof maintenance and retrieval without re-programming even though
the file structure is modifiede To accomplish this, we willcontinue the attempt to develop a structure which is capable ofcontaining a wide range of content, and from which any recordor sequence of records can be retrieved by using a common retrieval routine. The programs have been designed to be parameter-driven, so that changes in file structure may be easily
accommodated.
Other Analyses. We have barely begun to identify andstudy the central issues associated with on-line organizationand search. Among these, the grouping of records or parts ofrecords by similarity or other relations pertaining either tosubject content or to user need, has received growing emphasisin the Project.
5.

We have programmed the first of a set of routines for theequivalence class coding for author names, title words and othersubstantives (reported in detail in Section III).
Associative indexing by statistically-generated means, asa technique for providing improved retrieval, can be effective
for material which is indexed in depth, i.e., with a large average number of assigned index terms per document. We plan toinvestigate the utility of such associative techniques on monographic records, which by present cataloging practice have a much
lower average number of assigned index terms per item - i.e., subject headings per document. As part of this study, the subject
headings in the file will be analyzed for characteristics such as
distribution by date of publication, co-occurrence, and the number of headings per document.
*Payne, Charles T. "Tagging Codes." Chicago University of
Chicago Library, Feb. 1967. (unpublished report) various pagings- 11 5Our work on formats tailored to the various system functions,
i.e., input, processing, etc., leads naturally to concern for datacompression techniques. Our concern here is to provide efficient
representations of the records when they are in the mass storage
device. We plan to initiate a task to analyze the applicabilityof known data compression techniques in terms of their encoding
and decoding effort together with the resulting amount of spacerequired to represent bibliographic records.
B.

EQUIPMENT

Characteristics.
In selecting a system for our experiments with bibliographic data, we must trade off the following
six, desirable characteristics:
1.

The maximum amount of large-capacity, random -accessstorage capacity that can be attached to a central processor.
a.

b.

The least difficulty in attadhing remote terminals,
of both the mechanical and cathode ray -Wipe (CRT) variety.

Machine logic capable of handling the maximum
number of individual characters in order to facilitate handlingof the wide range of typographic dharacters encountered in
bibliographic data.
c.

That to the utmost extent possible, system software
especially in the realm of physical I/O, be provided by the
manufacturer of the machine.
d.

That this full system be available on a time-sharedbasis during as much as possible of the regular working day.
e.

That it be the most common system in use forbibliographic data, in order to share software.
f.

In choosing between the two systems available to us at theBerkeley campus computing center (an IBM 360, =del 40 and a
CDC 6400), the IBM 360 provided the better compromise among thisset of desired characteristics.
Main Frame. The central proceSsing unit (CPU) of our
360/40 has a 128K byte memory.
Of this, the Operating Systemtakes approximately 30K. The remaining core storage is divided
into two partitions.
In one of these (approximately 20K in size),
utility programs are run (such as tape copy operations). The
remaining 80K bytes are available for other applications. It
is in this larger second partition that we operate when on-line.
2.

Mass Storage. The concept of the on-line system necessarily involves the use of random-access mass storage. Threetypes are generally available for 360 attachment: drum, disk,
and magnetic strip (also called data cell). Drum storage wasrejected became of its limited capacity. Magnetic strip storage- 12 3.

.1.1

was considered, but rejected for use during Phase I of the project because of persisting hardware failures. (These problems,
we understand, are currently being corrected.)
Given our requirement for very large randam-access storage,
the IBM 2314 disk storage facility appeared particularly satisfactory since it is a supported unit in terms of IBM softwareand has high operational reliability. This device provides a
maximum storage capacity of 233 million characters. However,the device's effective capacity is approximately 150 million
characters since some space is required by the system in orderto store the operating system programs and control data. The
access time (i.e., the average time required to position theaccess device to the desired record) for the device is 100 milliThe data transfer rate (i.e., rate at which data isseconds.
moved between the device and core storage) is 312,000 characters
per second.
Terminal Equipment.. In our experiments, we have planned
that inquiries made to the file will be processed over remoteterminal equipment. This means that processing programs must
operate in a tele-processing environment where several terminals
are busy at the same time. Since the transfer rate of charactersover phone lines to a terminal is relatively slow, the computer
central processing unit (CPU) while servicing one terminal iswaiting virtually all of the time. It is this waiting state orunused CPU time that makes time-sharing with multiple terminals
an attractive alternative, since with the attachment of a few
additional terminals the CPU is utilized more fully5 with little
noticeable increase in delay time to individual terminals.
4.

Current1y, we have two mechanical terminals attached to thea Teletype model 35 ASR and an IBM 2740 communications
360:
terminal. Data transfer rate to each terminal is approximately
12 and 15 characters per second, respectively. Each has a private line connection over voice-grade telephone lines. The
teletype can read and punch paper tape, a feature useful insituations where data can be prepared before going on-line, inorder to obtain the maximum input rate when on-line.

Mechanical terminals, as we have indicated, produce hard
copy output which in some instances may be desirable. However,in our early work it has become evident that the slowness of amechanical terminal as it types out, character by character, theresults of an inquiry, does not make it particularly attractive
Therefore, we are now
for most library retrieval applications.
planning to connect CRT terminals for inquiry and display ofsearch results. As a shared task among several ILR projects
we have completed an exhaustive inquiry into the characteristics
of currently available CRT's. A. major drawback at this time isthat, with the exception of extremely expensive terminals, themaximum number of displayable characters is under 100, a number
not nearly sufficient to meet our capacity requirements for the

13 -display of bibliographic data. Even so, the high volume of data
output associated with bibliographic search makes it desirable toincorporate CRT's as soon as possible, in order to facilitate
testing on a basis superior to that achievable with the mechanical devices.
C.

COMPUTER PROGRAMS

Three levels of programs are required for the facility weare developing. On the first and most general level is themonitor system which provides for remote terminal operation.
The second level programs carry out the various operations offile generation, organization, maintenance, retrieval and display.
The third.level contains programs which analyze the performance of the system, to guide its development. The second
and third level programs do not necessarily operate in an on-line
mode.
1.

Terminal Monitor System.

Introduction. During the course of the year wehave developed an experimental Terminal Monitor System (TMS)
which is designed to facilitate both communication between remoteterminals connected via phone line to the Camputer Center's
IBM 360/40, and data transfer between the 360/40 and the attacheddisk storage facility. All files are maintained on ILR's private
disk facilities and are not accessible by other 360 users.
a.

TMS has a time-sharing design which allows multiple terminals
to operate seemingly at the same time. It allows the user tocarry on a limited dialog with the computer, and will wait for
the user to enter his response before continuing processing.
TMS operates as a user slibsystem of the manufacturer-supported
Operating System/360 (OS) in a partition of core storage ofapproximately 80,000 bytes.
TMS performs five general functions:
Text entry:
later be processed.
(1)

the establishment of new files which canFile search: retrieval and display of records framwithin id.n existing machine file.
(2)

(3)

Text edittm:

addition, replacement, and deletion ofCharacter strings, individual records, and blocks of records
within an existing file.
Campilation of source programs: conversion to executable instructions from source language (FORTRUT, PL/1, and 360
Assembler) entered in the manner of text.
(4)

Interface to special user-written routines:
capability
for terminal user to load and execute special-purpose programs.
(5)

The logical structure of TMS is built around a set of two
supervisory routines which maintain control over a series ofprocessing programs (called processors).
It is the processors
which perform the file handling functions listed above.
Supervisory routines. One of the two supervisoryroutines (TXIO ) has the function of coordinating input/output
operations to the terminals attached to the system. It doessuch things as directing message transmission to the proper
terminal, and the analysis of which user has sent a message
when one is received.
It also performs error recovery andrecording (when a transmission error has occurred), charactercode translation on both incoming and outgoing messages (sinceeach different type of terminal has its awn character code),and message attribute and length analysis.
b.

The other supervisory routine (BASE) is the overseer ofboth TXIO and the set of processors. Its function is to bringa processor into core storage when needed by a terminal user.
After loading a processor, the BASE routine delegates controlof the terminal to the processor, which then communicates directlywith TXIO to accomplish terminal input/output. If more than one
terminal requests the same processor for simultaneous use, theBASE routine does not load another copy; both terminals use thesame copy. The arrangement which allows sharing of a processor
in this way is called "re-entrant coding".
The amount of corestorage in use at any one time is thus a function of the number ofterminals on-line, the space required by each terminal's processor, and the degree to which terminals are sharing processors.
Processing programs are of two sub-types: 1) utility processors, which perform general functions and are used by manyusers and 2) specialized processors written by terminal users
for their own file handling applications.
Utility processors. Currently there are seven utility processors. The specific instructions for existing processor
use, showing the formats of both TMS messages and terminal user's
responses,appear in the Terminal User's Manual (See Appendix II).
The first three processors are used for general file handling
operations. The last four are used in the development of new,user-written processors. The processors are:c.

Text processor. Enables the terminal user to create
a new file. Each record in this file is 80 characters in length,of which 76 characters may contain user data and 4 characters
the key by which a record can be retrieved.
(1)

Search processor. Used to display one or more records
from an existing keyed file. It has the ability to search on
the full key or on a portion of a key.
- 15 (2)Egit_processor. Used to edit existing files. It is
possible to perform replacement, addition, and deletion functions
on character strings within a record. In addition, one can also
add or delete entire records.
(3)

AsseMbler language processor. Here a source language
file created as text is given over to the 360 Assembler togenerate object (machine language) instructions.
This, and thefollowing PL/1 processor, are used to generate new programs which
may become part of the processor library.
(4)

PL/1 Language processor. Used to generate object codefrom PL/1 source language stored as text.
(5)

A link processor. Has as its input object code (generated from either the PL/1 compiler or the Assembler) and generates a new (or replacement) processor.
(6)

LYRIC.

A processor for user interaction routines,
which is not yet fully implemented (See Appendix III).
(7)

S ecialized processors. Currently there areapproximately ten specialized processors. The most importantof these, the file handlir programs, are discussed below.
d.

All processors exist as load modules, and are maintained as
individual members of a partitioned file. Each processor isplanned to be re-entrant, so that multiple use does not require
separate copies of a given processor. A terminal user writinga specialized processor must adhere to several conventions: He
must observe standard calling conventions, and must provide error
recovery which returns control directly to the calling program(the BASE routine). Terminal I/0 is accomplished through calls
to special interface routines which are link-edited into theuser's routine during execution of the LINK processor.
File Handling Programs.
Two classes of programs havebeen written which have as their focus the development and useof the project's data base. The first class is concerned withthe generation and maintenance of the master file and its
associated index files. These programs are batch-oriented andare not operated as part of the Terminal Monitor System. The
second category consists of programs which are terminal based(i.e., processors) and which are concerned primarily with retrieval from the existing bibliographic files.
2.

File generation programs. Fig. 2 shows the file
generation process. The names in parentheses identify computerroutines required. These file generation programs provide forboth original input and extraction flam existing machine files.
- 16 a.

FIGURE 2:

FILE GENERATION PROCESS//&ntaCruz Fl eriginal
orma
/-

./

Original InpAtt
ILR Input Fo

KTranslation
File
TRANSCOF)

1(
Translate

Input
Conversion& EditRoutinesanta
Cruz Fi e(ILR In ut Format)(INFOCAL)

nc orreDiagnostic
PrintoutedTransact onFile (I R Input Format)

d &

Proof
Read

No
Error

Certify

Correcte
Transact on(INFOCAL

CreateUpdatedTransaction
File

ecs.)

Certified &orrectedTransact
ecordsILR Input Format)
Errors Flund
Diagnostic
Marked forRe-keying

Off-line
Correction
Typing(Processing
For_ t)(Tape or Disk)or

On-line(TMS: EDIT)(Carryover)

TO UPDATEOF MASTER FILE

The most important of the programs written for this purposeis the one called INFOCAL, which converts data presented in input
format to internal processing format. INFOCAL is written in PL/1.
For continuing
Unfortunately, it is so large it is very slow.
operation, it appears that an assembly language routine will beThis program is designed to do several things before
required.
a record is actually placed in the master file. First, every
new record entering the system may, as an option, be printed out
on an edit listing, so that it can be proofread for errors inSecond, the program does a certain anount of syntacthe text.
tical checking of field codes and other control information,
At the saneand prints diagnostic messages on the edit listing.
time the program also places a copy of each new record in atransaction file written on disk storage as a keyed file.
Corrections are written on the edit listing by the proofreader and changes to the file may be keyboarded either on-line
via remote terminal, using EDIT, the edit processor available
under the TMS, or off-line for processing in a batch-oriented
update routine. If a record has no further corrections, it may
be certified either off-line or fram the terminal as ready to
be transferred from the transaction file to the master file.
On the next successive execution of the INFOCAL routine, records
marked as certified will be written out and are ready to be
added to the master file. Routines to accomplish this maintenancecycle are now in the design stage.
Records carried over from the existing transaction file,which are not yet certified, must go along with new records into
the new (updated) transaction file.
The file translation program, TRANSCOF, converts the existing
Santa Cruz data base fram its original record format to the ILR
input format (that is, the record structure accepted by INFOCAL).
This routine, still under development, is being written in assembly language as a result of our operating experience with PL/1.
We planned to produce our input format so that if there arechanges to the internal format, only INFOCAL will need tobechanged. Additional discussion of TRANSCOF is provided in
Section IV.
Specialized retrieval processors. In order tobegin experimentation with terminal-based search algorithms,
we placed 75,000 records fram the Santa Cruz file on the 2314
disk facility. These records are still in the original fornat,
not yet having been converted to ILR internal format. To retrieve
records, we have written a primitive processor (SCAT) which allows
a terminal user to type an author's name over the terminal, andto receive a list of documents indexed under that name. Although
we have not generated them, other search keys could be included
readily. However, we did not include them in this initial test
because Boolean cambinations of the terms would not be possible
with SCAT.
- 18 b.

For the user, the search technique is quite simple (and,of course, limited). The contents of the file can be displayedby typing an author's name. Retrieval will be made of allrecords for which the left-most characters of the key match
those characters input over the terminal by the requestor. For
example, if the requestor typed the string "GARN" when asked toplace his request, the program would retrieve all records whose
Currently, three authors
key begins with these characters.
match: "Garn, Stanley M.", "Garner, Wendel R.", and "Garnett,
If he had typed "GARNETT", he would, of
Arthur Campbell".
course, match only on "GARNETT, ARTHUR CAMPBELL".

The master Santa Cruz file contains full bibliographic
entries, When printed out on the terninal each citation occupies
from five to twelve print lines. It becomes evident very quickly
that, given a mechanical terminal, some means must immediately
be constructed to enable the user to limit the amount of material
which is typed out before him. Display of a full record of
average size (400-500 cha-acters) consumes about 1-1/2 minutes.
For the future, it would
Ten of these would requile 15 minutes.
thus appear that mechanical terminals will not be suitable foron-line retrieval of bibliographic data.
Because of such time delays, we have initially provided two
mechanisms for limiting output. One of these is to request theuser to indicate how many entries he will accept in response tohis search request. The other is to allow him to display index
entries only. An index entry contains only one line of data
which gives the full key (full author name) and the address ofthe master file entry (its sequence number). Upon completionof the index file search, the user may then request full citations by record number.
Although the SCAT processor is quite limited, it has provided us with immediate experience which has been usefUl in
determining the types of retrieval capability to be incorporated
in future search programs. Further, we now have had experience
with multi-level file structures of the type we will be using
throughout the study.
3.

Analytic Programs.

General purpose. The third level contains the
programs concerned with the analysis of the system - its content,
structure and operation. These "third level" programs are not
a component part of the system providing file maintenance andsearch; rather, they operate independently to obtain information
to guide the project.
a.

Although no programming on this level has yet been accomplished, we anticipate that some of these programs will be used
- 19 -

'`For example, one kind of general program willquite generally.
be for extracting information from a history file containing
the records of past system use. This history file will bemaintained as a basic part of our terminal handling program.
In this way we will accumulate a history file which can be used
to make the same sequence of inquiries against file structures
which have been developed using different organizing strategies.
In these reruns the results of the searches will not go out over
the terminals. Instead, we will record the time for the storage
access portion of the query processing in order to obtain comparable timings fur the different structures. We will use thisapproach to study factors such as prediction of relative frequency of access to the records in the file, as a basis forreorganizing the files in order to minimize average access time
for the individual query.

To support our research, many
Special purpose.
special purpose programs will be required. Examples of these
programs are those to analyze the frequency of co-occurrenceof index terms in the data base; the time pattern of requests
for file information; and the effects of a file compression
technique on storage space and encoding -.decoding time.
Programming of such routines will continue throughout the
project.
b.

D.

DATA BASE DEVELOPMENT

MonograDh Data Bases. To begin the study, a brief
survey was performed to determine the availability of librarycatalog files which might be used. While there are numerous
such files, those of UC Santa Cruz library, Stanford University
Undergraduate Library and LC MARC I* seemed most complete andaccessible. Arrangements were made to obtain all three files.
However, the Stanford file has not yet been obtained.
1.

Before our work with the MARC I records had begun, theLibrary of Congress announced the MARC II format. Therefore,
wa decided to organize the major work of the project around thisnew format and in the interim until it is available to concentrate on the use of the Santa Cruz file. A brief descriptionof the Santa Cruz format is provided in Appendix V. Section .IIIof our report provides the results of our work which concerns
the design of these records. Section IV provides a descriptionof our resolution of the problems associated with the development of a large database.
Other Data Bases. In addition to the task of acquiringexisting files of monographic materials, a further goal of the
project is to study the problems of integrating and searchingfiles of records for other materials, e.g., journal articles,
2.

*The Institute at Los Angeles was a participant in the MARC Pilot
Project experiment.
- 20 -

music, etc. The availability of bibliographic machine files
on the level of the journal article is placing increasingpressure upon the library community to provide computer-based
access to these articles. Thus the issue of organization andsearch of such files is as pressing as that for monographic
material. At the same time, it does not appear to require a
uniquely different approach to its solution. Therefore, we
are including journal article maohine files in the study.

Although these records necessarily vary in content fromthat of monograph catalog records, compatible record formats
may be developed. This is a goal to which the Library ofCongress MARC Project has addressed itself in the developmentof a common machine format for bibliographic and other librarymaterials. That is, the MARC II format is designed as a dataexchange medium, with a structure intended to have "wide applicability to all types of bibliographic data" and to be "hospitable to all kinds of bibliographic information." For "any given
type of item, e.g., serials, maps,, music, etc., the componentsof the (MARC II) format may have specific meanings and uniquecharacteristics."
Whether it will be possible to so integrate the different
records remains to be seen. However, we are working on the
integration of a citation index file which is a non-monographdata base. Work has been carried on at UCLA to extend theCitation Index in Mathematics, which was initiated at Princeton.
The nature of the UCLA data base and its preparation is described
At this time, we have no significant resultsin the Appendix V
to report.
.

E.

FILE STRUCTURE

Inverted File Structure. In the first phase of thisstudy we began a detailed examination of file structure. Early
work in this area indicates that the way in which files ofbibliographic information are structured profoundly influences
both the time required to gain access to a record in the fileand amount of storage space consumed by the files. In a large
file it is evident that one cannot conveniently make a sequential
search of the entire file in order to satisfy a unit request for
information. As a result, files usually have been structured
in what is called an "inverted structure". In this structure
all logical records are held in a "master file" with auxiliary
files (called index files) created which allow quasi-random
access into the master file.
1.

*Avram, Henriette D., John F. Knapp, and Lucia J. Rather. The
MARC II Format: A Communications Format for Bibliographic Data.
Washington, D.C. Information Systems dffice, Library of Congress,
1968. p. 10.
7 21 -

It.is possible to define a master file structure from which
records could be retrieved by some given portion of it, calleda key:
for example, author. However, since in most computer
systems a record hes only one access point, its address, wemust define additional files which provide multiple accesspoints into the master record. These additional files, theindex files, contain records which point to and allow us tolocate and access the information contained in master records.
In oux system the full bibliographic record comprises themaster record. The content and format of each master record
is described in detail in Section III. The full set of masterrecords will be called the master file. The content of anindex record consists of the index term (the access point), andpointers, one to each master record which has been indexed underthat particular term. One record in an author index file might,
for example, be:

1

SMITH, JOHN W.

Al

A2

A

A4
3

j

In this record, SMITH, JOHN W., is the access point to theindex records and Ai, A
A and A4 represent pointers to four2'

3

different master records which have been indexed under thisaccess point. Because index terms have varying numbers of
documents indexed under them, the length of the logical index
record is variable.
For our initial operations, this is the type of structure
we have been using. However, this approach is not adequate toour purposes in developing a system for experiment. During thecourse of the study we will want to maintain maximum flexibility
in the operation of the system. We will want to split logicalrecords, create new index files, partition them, relocate records
and files, and make new groupings of records.
Such manipulations
will allow us to explore the relationships which exist between
access time and storage cost. For this reason we are developinga more general system employing the basic file structure described belay. As a part of this we are writing a set ofutility retrieval routines which operate independently of filecontent.

Random-Access Device Characteristics. In our experiments
we are using a large-capacity disk facility to store the bibliographic data files used. Since the way the file is structured
is, in large part, determined by the methods with which data
can be physically stored on a disk, it is necessary to discuss
briefly the ways in which information can be stored on andretrieved from a disk device.
- 22 2.

Storage modes. Data is recorded on a disk In
recording surfaces called tracks, all of which are of equal
capacity in the 2314 facility. Movement of data written ontoor read from a track is done in units called "block" (alsoreferred to as physical records*). Normally, the size of a
We have now
block cannot exceed maximum track capacity.
experimented wdth two general types of file: 1) keyed files
and 2) non-keyed files.
a.

A keyed file is one in which each logical record can be
retrieved by supplying some unique portion of it called a key.
In general, each record may have only one key which must be
identified at the time the file is created. Non-keyed files may
be retrieved only by address. This address may be the actual
position of the record on the device (that is, the track number
where the record resides), or it may be a relative address (thatis, its position relative to the beginning of the file). Actual
addresses are rarely used, since this technique makes a fileunmovable. Relative addressing allows the file to be moved toany position within the device.
Whether keyed or non-keyed, the very large size of our files
and the relatively high 'cost of direct-access storage, make it
desirable that the data be placed on the disk in as compact aform as possible.
Disk space is used mast effectively when logical records
are blocked to maximum track capacity, and all blocks are of
fixed (i.e., equal) length. This is because fixed-length blocks
involve less overhead both in terms of access time and storage
space than do variable-length records. Blocking to maximum
track capacity also involves less device overhead since eachblock must have a given amount of system control information
stored with it.
To illustrate: Track capacity of a hypothetical device
is 3000 characters; and each block must have 50 characters of
If we allow only one
s!rs tem control information stored with it.
block per track, we must talocate 50 of the 3000 characters forcontrol information. This leaves us 2950 characters for theblock of data. If we allay two blocks per track, the system
requires 100 characters with 2900 remaining for the two blocks(1450 characters in each). Fig. 3 shows figures for differing
lengths.

*As distinct from logical records; i.e., logical units of data.

- 23 -FIGURE 3:

BLOCKING STRATEGY

Track. capacity = 3000 dharacters
System overhead = 50 chars./block

Total
Allocationfor System
Overhead

Total Space
Remainingfor Data

Size of
Each
Block

1

50 char2950 char2950 dhar

2

100

2900

1450

3

150

2850

950

4

200

2800

700

Number of
Blocks per
Track

I

_

To summarize: All blocks, whether in a keyed or non-keyed
file, should be bcth fixedrlength and blocked to maximum track
capacity.

Since most bibliographic records are variable in length,
and we wish to use fixed-length blocks, we are adopting atechnique which translates variable-length logical records into
fixed-length blocks of maximum track capacity. A different
strategy is required for keyed files than for non-keyed files.
Non-keyed files. This structure involves sequential
placement of the variable-length logical records into a block
sized buffer in main storage; when that buffer is full, theentire block is written to disk. In order to retrieve a logicalrecord we need to knaw: 1) in which block it begins, 2) its
relative position within the block, and 3) its length. In theexample in Fig. 4, a total of sin: complete logical records(denoted by Ln) and part of a seventh record have been mappedb.

into two physical blocks.

FIGURE 4:

Block 1

Block 2

1---L

BLOCKING A NON-KEYED FILE

L1

L

(cont'd)

L

3

L

2

3

4

Partial
RecordPositional
Scale
1

1

1000

2000

3000

1

1

1

4000

1

1

1

1

1

5000

1

1

1

6000

The first logical record (L1) is of length 2000 and can be
retrieved by first reading block 1, then locating position 1
Similarly, the second logical record (L2)within the block.
can be addressed by Block 1, Position 2000. It has a length
value of 1500 Characters. Note that it is necessary to breaksome records across track boundaries, so that one logical record
may actually be contained in two (or possibly more) blocks.
Therefore, when a logical record is broken across track boundaries,
a signal must be inserted to indicate whether or not the record
continues into the next block.

Thus, any variable length logical record can be retrievedby supplying its block (or track) number, its position within
the block, and its length, which together constitute the addressof the record.
Keyed files. To map variable-length keyed files
into fixed-length keyed format*,we separate each logical record
into two components. The first component, a fixed-length, keyedrecord, points to the variable length data portion. For example,
the two records:c.

DATA 1

KEY 1
KEY 2

DATA 2

IIIMIMIREC1*The operating system data management facilities currently require
the index entry in a keyed file to be fixed-length, specifiable by
the user.
- 25 -

would each be split into two records.
keyed records:

KEY 1

ADDX1

KEY 2

ADDX

The first set is the2

ADDX is a pointer field that contains the address of the variable
length data portion.
The second set of records contains the data portions:

1.4

DATA

ifATA

2

ADDX

ADDX

These are placed in a variable-length, non-keyed file and treatedas discussed above.
Multilevel File Structure and File Linkage. Central toan understanding of a generca retrieval routine are the conceptsof record segmentation, file segmentation, and linkage. An index
file may, for example, be composed of two distinct files which
are linked together. The mechanism which links two segments
together is called a pointer. In the simple inverted filestructure, the poiaters in the index record connect to addressesof master records. In the same way, two distinct physical files
may be linked to form a single index file. Similarly, a masterrecord itself might be divided into two or more parts, one partcontaining those elements of the logical record which are usedfrequently (e.g., main entry, title statement, imprint and callnumber) and the other portion(s) of it which are required withlesser frequency (bibliographic notes, etc.) Thus it is possible
in each case to have a record which has both content and a pointerto another record, or portion of a record. We shall call theindividual records which are linked together (by pointers)
linkage segments.
The linkage segment is conceptually independentof its logical content. This file structure is similar to thatof a linked list file. In the conventional version of the latter
file structure, the master records themselves are usually notsegmented.
3.

Why should one wish to subdivide a master record? One cananswer this question both at the level of our requirement to- 26 -

develop a system for experimentation, and on the level of thedesign of an operational on-line bibliographical retrieval
system. Taking the latter first, it is most likely that thevariable length records will be segmented into fixed block sizes
in order to simplify the problems of input/output and storage
allocation. If one places the most frequently used parts of therecord in the first segment then following segments will notneed to be obtained as often as they would if the allocation tosegment were independent of use. Thus, the capacity and responseof the system would be improved. This improvement would be evenmore dramatic if two random-access storage facilities are used,with the high frequency elements allocated to the one which is
fast and expensive, while the lower frequency elements areallocated to the other slower and less expensive unit. Suchstorage of segments on the basis of access time of the device
we call "horizontal segmentation".
One can also perform "vertical segmentation", that is,within the file, the records themselves may be arrayed byfrequency and divided into groups containing the most-used andleast frequently-used records. Here Again one could place themost frequently used records on the fast device and the leastused on the slaw device.
The answer for our facility derives fram the comments just
made.
These remarks indicate a conceptual solution to the
problems of file organization. However, it is a solution whichraises a host of questions about the relation of segment sizes,
element frequencies, access time and cost, and storage costs.
These questions can only be answered by analysis and experiment.
Therefore, we wish to set up our computer routines such thatvarieties of segmentation can be accommodated in order to beable to carry out relevant experiments.

For our initial experiments we have chosen a simple filestructure. It consists of:a.

A master file consisting of records stored without keys.

Index files for author, title and sUbject.
file consists of two sub-files.
b.

Each index(1) a keyed file (called the "access file") in which theaccess point is the key and the data field contains a single
pointer to the second sub-file.
(2) the "address file", a non-keyed file containing avariable number of pointers to the master records which havebeen indexed under the term represented in the access file.
To illustrate: Four master records have been indexed under
the term CARTOGRAPHY.

-27-

The entry in the access file for this term would be:

ADDX

CARTOGRAPHY

where the term CARTOGRAPHY occupies the key portion of therecord, and ADDX is the data portion containing the pointerto the corresponding record in the address file. Logical records
in the access file will be of fixed length, while those in theaddress file will be variable length.
The address file
entry for this term will have four fields
ONIMIN11
containing the addresses of the master records. The format will
be:

I

All.

A

ADDX
where the shaded part indicates other data contained in the
A , A4 are pointers whichA
track, R is the record,and A
l'

2'

3

contain the addresses of the four master records.

Uniqueness of Identification as a Function of Key Length.
A significant issue in the study is the number of characters which
are required to uniquely identify a record held in the system.
This is important for several reasons. nrst, the more characterswhich must be input through a terminal, th ::? longer the encoding
process becames and the less capacity the terminal will have forhandling requests. Second, the mcre characters which must betransmitted by the terminal, the greater the chance for keying
error becames. At best, these errors will be automaticallycorrected,* but invoking correction routines will reduce thecapacity of the system even further. At worst, these errors will
initially cause improper retrieval and thus a partia/ failure of
the system. Finally, the expected key length for uniquenesscould be used as a basis for index record segmentation. The
objective of the system, therefore, should be to provide searchbased on enough characters in the request key to provide a
reasonable amount of uniqueness of the resulting search output,
but with as few characters as possible in order to reduce theprdblems of keyboard time and keyboard error.
4.

We have analyzed the amount of uniqueness of author names*For example, by use of equivalence class coding techniques
discussed in Section III.E.
- 28 -

as a function of the number of characters of input provided.
The program used distributes the number of catalog records per
unique search tag of given length for author keys. The results,
taken from a sample of 20,000 author names, are presented inFig. 5. We plan to run this analysis on title and subject key
fields as well, in order to determine the proper length for thosesearch keys. At present we are implementing our search routines
so that a search key of any length can be used by the searcher.
Consequently, our search strategy is independent of search key
length, which enables us to experiment with keys of various
lengths.
The segmentation of an index file may be performed on thebasis of key length. In that case, the distribution of thelength of the index terms, or keys, determines the effect ofsegmentation. In preparation for the analysis of the effects ofsegmenting the index files, we counted the lengths of the callnumber field, the author field, and the subject heading fields
in the Santa Cruz machine file. Fig. 6 contains the distributionsof the field lengths for those elements.*
Linkage Control in Retrieval. The retrieval routine
needs to provide access to logical records from two types of
keyed and non-keyed; in addition it must be able tofiles:
distinguish between data which is content and pointer data. If
a record contains pointer data the routine must know whether or
not to go on retrieving subsequent records in the linkage sequence.
In some cases presentation of information contained in the pointerfield will be enough to satisfy the control routine.
5.

The pointer field establishes a linkage between two logicalrecords. The pointer field will contain the following information:A.

Type of file pointed to; whether it is keyed or non-keyed.

B.

Type of data in the record pointed to:1) Pointer data only.
2) Content data only.
3) Pointer and content (pointer first).
4) Content and pointer(s) (pointer information
carried as a type of content, e.g., key
itself encoded as the address).

C.

The name of the file in which the record resides.

*Source: Compiled from computer runs made November 1 to November 6, 1967 against Santa Cruz catalog on SCAT 10, 11, 12.
Compiler: Ralph M. Shoffner. Capital letters are counted as
two characters. Note that none of these counts have been verified and they may contain errors.
- 29 -FIGURE 5:

UNIQUENESS OF AUTHOR IDENTIFICATION

Number of Characters in Identification Tag

2
3
4
5

6
7
8
9

10
11
12
13
14
15
16
17
18
19
20
21
22
23

24
25
26
27
28
29
30

7

8

2866
971
492
333
184
148
98
60
78
49

3175
1030
518
335
183
150
92
63
73

39
27
29
24
20

16
8
9

50
42
28
26
21

19
18
11
8

3635
1140
544
347
176
152
96
63
73
45
29
30
25
22
18
14
11
9
14
9

10

11

12

13

14

15

16

4364
1290
605
358
200
152
89
62
66
46
34
29
26
19
18
13
14
13
13
12
9
8

5399
1515
655
343
201
164
91
65
58
42
26
23
23
19
11

6319
1628
683
346
203
142
88
55

6895
1732
699
323
204
134
82

7288
1765
692
322

7496
1768

7656
1770

678
324
194
128
82

670
323
190
127

56

55

47
38
21

47
36
19
21
21
11
13
7
12

42
32
15
18
23
10
14

51
41

51
39

30
15
18
22
9
13
8
9

30
15
16
21
11
11
6
9

18
9
14
10
7

16
9
10
7
8

11
9
8

4

5

4

8

5

3

6
4
3

5

5

3
4
3

6
7

6

5

3
5
5
3
5
3
3

4

3

1

5

11
12
13

18
26
13
12
10
12

201
128
81

6

12

79

9
8
6
9

9
9
9
2
4

7
6

5
5

5
5

5
5

1
4

1
4

1
4

1
4

2

2

2

2

2

2

1

2

2

2

2

5

4

3

2

2
3
1
0

3
2
2
1
0

2

3
2

2

2

0
0

0
0

1

3
4
2
3
3
1

9777

10000
1
2
0

Notes:

Table entries are the number of unique identification tags
having a given number authors with the same identification tag(specified by the row value) for the given tag length (specified
by the column value). The identification tag consists of the
beginning characters of the author field in the order surname,forname, etc.
20,000 authors in alphabetical order
Source:Naresh Kripalani, March 6, 1968
Compiler:
30

FIGURE 6:

DISTRIBUTION OF NUMBER OF FIELDS OF LENGTH N
Length1Call No.

%

46-56

0
0
0
0
0
0
0
0
0
0
0
1
4

0.0
0.0
0.0
0.0
0.0
0.0
0.0
0.1
0.5
3.2
8.2
15.8
21.0
11.3
8.3
8.8
7.6
6,4
3.6
1.5
1.4
1.2
0.7
0.4
0.1
0.0
0.0
0.0
0.0
0.0
0.0
0.0
0.0
0.0
0.0
0.0
0.0
0.0
0.0
0.0
0.0
0.0
0.0
0.0
0.0
0.0

Total

54,122

100.1

5

1
0
0
0
0

6

5

7
8

14

2
3
4

9

10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35

36
37
38
39
40
41
42
43
44
45

79
312
1870
4770
9234
12322
6614
4877
5180
4434
3717
2095
846
801
656
422
218
77
6
0
0
0
0
0
0
1

Author
0

16
36
33
37
50

136
64

155
103
249
221
444
953
1399
1908
2339
2709
2740
2759
2698
2768
3014
3165
3145
2972
2837
2664
2745
2826
3145
3311
2965
2713
2277
1919
1542
1230
968
969
644
573
512
472
439
2513

71,377- 31 -

%

0.0
0.0
0.1
0.1
0.1
0.1
0.2
0.1
0.2
0.1
0.4
0.3
0.6
1.3
2.0
2.6
3.3
3.8
3.8
3.9
3.8
3.9
4.2
4.4
4.4
4.2
4.0
3.7
3.9
4.1

44
4.6
4.2
3.9
3.2
2.7
2.2
1.7
1.4
1.4
0.9
0.8
0.7
0.7
0.6
3.6

100%6

Sub ect
0
0
4

65

172
462
924
1154
1197
1434
1662
1676
1651
1778
1976
2072
2330
2346
2369
2552
2835
3327
3095
3186
3499
2979
2844
3046
3036
2656
2458
2332
2422
2372
2085
1691
1588
1674
1314
1196
1224
1190
1031
1003

g44
87,216

%

0.0
0.0
0.1
0.1
0.2
0.5
1.1
1.3
1.4
1.6
1.9
1.9
1.9
2.0
2.3
2.4
2.7
2.7
2.7
2.9
3'3
3.8
3.6
3.7
4.0
3.4
3.3
3.5
3.5
3.0
2.8
2.7
2.8
2.7
2.4
1.9
1.8
1.9
1.5
1.4
1.4
1.3
1.2
1.2
7.

100.2

Length information: if a content record is pointed to,
the rumber of characters in the record; if a pointer record(typos 1 and 3), this element contains the number of fields inthe pointer record.
D.

In addition, if the pointer is to a non-keyed record, thepointer field mast contain the record address; that is:E.

Relative block number, andF.

Relative position within the block.

If the pointer is to a keyed file the pointer record must
contain:
G.

The key value.*

The format of a pointer field is as follows:(NK)

POINTER TO A NON-KEYED RECORD:

A

B

T

D

Number/Pos.
Length

File
Name

BLOCK

-

1

0

4

2

B
Number/Length

File
Name
0

12

(K)

POINTER TO A KEYED RECORD:

A

8

6

1

2

4

KEY

6

The length of pointer NK is always 12 bytes; the length of pointerfield K is always 6 plus the key length (KL) of records in the.file to which this points.
Thus in this arrangement we have a linkage established by
the pointers. The access record points to the address record;
the address record contains multiple pointers to master records.
This linkage is shown schematically in Fig. 7.

*The key length for the file is obtained fram an independent
table called the file table.
- 32 -FIGURE 7:

SCHEMATIC OF MULTI-LEVEL FILE STRUCTURE LINKAGE

ACCESS RECORD(Level 1)

2

ADDRESS RECORD
A(R )

A R

A (R2)

A(R4)(Level 2)

WM./WMMASTER RECORDS
R(Level 3)

3

R2

R

1

Possible retrieval linkages are:any combination of Master records:
e.g.:R, + (R

2

R

3

R )

Using the master and index files described above, it ispossible to show the steps in a utility retrieval program. The
program is supplied with the key name of an access point, alongwith a code indicating whether the name is of the author, title,or subject class. A search on key is performed in the indicatedaccess file
If and when a match occurs, the data from theaddress is read into core storage.
- 33 .

At this point it is possible to provide the user withinformation which may be usefUl in satisfying the search request.
It is possible, for example, to indicate how many master records
have been indexed under the name, or to list the addresses ofthe master records. Otherwise, the processing continues and,using the addresses of the master record(s), the blocks containing them would be read into core storage, and the informationtransferred to the requestor at the on-line terminal.

-34-

III.

A.

THE BIBLIOGRAPHIC RECORD

GENERAL

Although record format and file structure are different
problems, there are overriding considerations which caused us to
devote considerable effort to the identification of record content
and form in the first phase of the project. Among these reasons
were three which are goals of the Project:
To develop methods of converting machine files of bibliographic data both from manual catalog data and existing machine(1)

files;

To devise techniques for processing the content ofmachine records via search routines, using actual data bases forrealistic testing;(2)

To make the data base being established as a researchvehicle for the Project, serve as a usable, available source ofrecords both to other organizations and in practical applications,e.g., a computer-produced book catalog. This is particularly
important in view of the large costs associated with the development of the data base.
(3)

Although we often refer to bibliographic records as if they
were homogeneous objects, they vary greatly both in 3:..)rm and content. Particularly in a data processing system, records whichrefer to the same external item (such as a book) may assume radically different forms as they are transformed through the stagesof input, processing, storage or output. Fig. 8 shows the major
functional components of a data processing system, each of which
may have different, often conflicting characteristics dictating a
different desirable form of record, independent of its content.
The use of a single record form throughout the system would meanthat none of the conflicting requirements would be effectively met.
In addition, the form of a record is very important to the fileorganization project if we are to meet our goal of providing asystem within which experimentation can take place. A computer
works entirely with the form of the data. .For example, if it is
to be possible to allow the author field of a record to containeither the author's name or a code standing for his name in an
author authority index, the form of the records must be such that
this difference can be signalled to the computer programs so thatappropriate conversion can take place before processing or output.
Thus, where a highly flexible system is the objective, record form
and file structure are interdependent.
In a similar manner, we cannot divorce the record form fromits content. That is, we wish to develop a system capable ofhandling records with content ranging from that of the traditional
catalog card through those of index entries for journal articles,
35

FIGURE 8:

FUNCTIONAL SYSTEM COMPONENTS RELATED TO DIFFERENT RECORD FORMATS

Printouttions

to those of augmented records containing abstracts, user comments
and perhaps, full text. In order to assure that our system isproperly designed, we must consider this range of content as weestablish the various record :orms.

In considering the form of the records to be handled, thequestions must be broad and their answers must be quite precise
if we are to achieve an open-ended facility that is capable ofreasonably efficient operation even though in an experimental
environment. Thus, we must determine not only how to tell whatparticular kind of record we have at hand or where a particularfield begins and ends, but also which typographical symbols wo
have chosen to represent, how they are represented, and what will
be done should we choose to represent other symbols at a later
date.

The final part of this section is given over to a discussionof one aspect of bibliographica3 "closeness", that of the similarity of personal names. This is an extremely important type ofsimilarity in the handling of bibliographic entries, in that theauthor is normally the major access point to a work. Our work is
directed at overcoming errors due to unintended spelling variations
in personal names.
B.

RECORD CONTENT

The content selected for the data base does not constitute aradical departure from current bibliographic standards. That is,
the major portion of the data base is anticipated to have thecontent of the traditional catalog card. In addition, some records
will have additional index record and citation content. A few
remarks will summarize the reasons for the decision to employ theconventional catalog record in a basically unmodified way. To
reconstitute the basic information content of machine bibliographic
records in advance of data on the use of current record content
would change the focus of the project.* Our decision was to orient
the building of a data base toward materials of general and current
usefulness to the research library so that our research results
could be of potential value at an early date. That goal in turnmade it desirable that the content of these records be capable ofrepresentation in a standard format to facilitate interchange ofdata among libraries. The decision was made, therefore, to conform to the path being taken by the Library of Congress in its
MARC Project which is developing a standdrd bibliographicalrecord content and format, beginning with monographs, and specifically intended for the data exchange function.

*For the opposite approach, see the reports from MIT's Project
INTREX, in particular the Augmented Catalog Data Base.

-37-

Since the end of the MARC Pilot Project in June 1967, LC has
moved through a phase of refinement and testing of the original
MARC I record, preliminary to implementation of a machine cataloging distribution service to libraries. A "new model" of the format
was issued in the form of specifications for the MARC II format in
the Spring of 1968. Parallel to these developments, ILR had begun
its File Organization Project in mid-1967.
The impact of this decision during the current phase of thestudy has been to place considerable stress on the project staff
attempting to keep the Project work abreast of the MARC format
development. We have found that the problems of coding using the
MARC II format are of two sorts - matters of data availability andmatters of difficulty of interpretation. The matters of data
availability are those pertaining to the less predictably usefulelements, e.g., to record whether the book contains an index toits own contents, or whether it is a festschrift. Without retrieving the book, editors coding catalog records cannot make suchdeterminations. The other difficulty arises from interpretation
problems, such as identifying sub-types of name headings in the
MARC indicators, assigning of the language codes, identifying someof the sub-types of subject headings, determining when the main
entry should be regarded as the publisher, and other areas wheretrained judgment and perhaps access to the book are mandatory for
accurate data encoding.
At this time, most of these problems appear to be resolved.
However, while accepting the defined content of the MARC recordin genera12, there are some codes or elements, defined by LC,
which we will not attempt to supply and there are others which wewill supply which are undefined by LC, at this time. These content
exceptions are listed briefly in Figures 9 and 10. It should be
noted in Fig. 9 that a distinction is made between a data element
and the corresponding code (e.g., field tag, sub-field delimiter,or whatever) which identifies the data element. For example, ILR
will not supply the identifying code for Book Number (e.g., in the
LC Call No. field) but the data will be present since it is embeddedin the field. In most cases of exception to MARC II, ILR is deferring both the MARC II data element and its associated identifying
code (if any), e.g., the Search Code, Tag 042.
C.

RECORD FORM

The Need for Multialt_Ezgalts. We have defined for theFile Organization Project an input record format and an internal
processing format to accept the corresponding input data. The
processing format in turn is convertible both to and from MARC II,
that is, when the record is to be output for transmission of our
locally-converted records on magnetic tape, or when LC-produced
records are to be accepted into our data base. In addition, weare planning to develop a mass storage format using data compression
techniques. The first question that is sometimes asked is "Why do
you need many formats? Why not just one?"1.

- 38 -FIGURE 9:

MARC II ELEMENTS DEFERRED IN FILE ORGANIZATION PROJECT DATA BASE
Defer
Identif.
Code Only

Data Element

Defer Both
Data Element& CodeFIXED LENGTH DATA ELEMENTS FIELDCountry of Publication
Intellectual Level
Festschrift Indicator
Index Indicator
Fiction Indicator
Biography Indicator

X

X
X
X
X
X

VARIABLE FIELD DATA ELEMENTS
002
014
019
020
070
071
080
220
360
670

Legend Extension
Search Code
Local System No.
BNB Classification No.
NAL Call Number
NAL Subject Category Number
UDC Number
Translated Title
Converted Price
NAL Agric./Biol. Vocabulary$Book Number portion of callnumbers
Thickness$- 39 -

X
X
X
X
X
X
X
X
X
X
X
X

FIGURE 10:

FILE ORGANIZATION PROJECT DATA ELEMENTS NOT DEFINED IN MARC IILEADER
Agency Code for Originator of Machine Record
Date of Machine Record Format Translation
Agency Code for Processor of Machine Record
Type of Source of the Catalog Data
Agency Code for Source
Agency Code for Adaptor of Catalog Data

FIXED LENGTH DATA ELEMENTS FIELD
Literary Group Filing Control
Cancel Title Added Entry Indicator

VARIABLE FIELD DATA ELEMENTS
052
091
570
580
64o
741

Cataloging Source Legend
Copy Statement (Local Card) (Proposed)"In Analytic" Note
"Full Name" Note
Book Title (as subject)
Title Added Entry (Periodical) (Proposed)

9-- NUC Card Number
Superintendent of Doc's. Number-12 Firm Name Heading (Proposed)-31 Anonymous Classic Heading (Proposed)
9--$

Copy No. (in Holdings Field)

40

record forIn many information system applications, only one
mat is defined". For example, a library converting catalog data
searching or bookfor any particular purpose, such as acquisitions
catalog printing, might operate with essentially one format. The
identifying codes used constitute a single structure applicablethroughout the cycle of computer storage and retrieval.
In contrast to that approach, the Library of Congress andothers are experimenting with multiple formats, viz., in the LC
input record, mnemonic character groupings are used as acronyms
to substitute for the field names (e.g., "MPS" stands for MAIN
ENTRY of the type PERSONAL NAME, sub-type SURNAME). The internal
processing codes, however, are the standard MARC II numeric tags,e.g., "100." Other libraries, such as Stanford and Toronto, aretesting mnemonic notation for use as query tags, i.e., as identifiers used to interrogate the data base.*
It is our view that although the formats must be convertible
one to another in order to perform functions such as input andsearch, the format requirements are dependent upon the particularfunction being performed. It would be cumbersome to impose thesame notational solution, e.g., use of an identical mnemonic tag,throughout all the system's functions. Where large files ofbibliographical records are being processed, the single format
approach is also costly. Instead, workable formats must be constructed to meet the requirements of each function separately.**

The following are the functional requirements for record
formats that we have identified.
(1)

Input format.

Ease of manual preparation and handling of the source
data (selection, creation of coding sheets, etc.)(a)

Speed and accuracy in human editing of the source
data. The codes should be easy to remember and simple and convenient to insert.
(b)"Query Language for the*See, for example: Bregzis, Ritvars.
Reactive Catalogue." In: Tonik, Albert B., ed. Information Retrieval: the User's Viewpoint - An Aid to Design. Philadelphia,(Fourth Annual
International Information, Inc., 1967. pp. 77-91.
National Colloquium on Information Retrieval, May 3-4, 1967).

**For a report on an approach to library file handling, with manyof the record format features of which we find common agreement,
Cox, N.S.M. and J.D. Dews. "The Newcastle File Handlingsee:
System." In: Cox, Nigel S.M. and M.W. Grose, eds. Organizationand Handlin_ of Biblio ra hic Records b Com uter. Hamden, Conn.,
Archon Books, 1967. pp. 1-21.

As much streamlining as possible of the keyboarding.
The keying should be rhythmic. Awkward strokes should be avoided.
(c)

Independence of the format codes and the typographical dharacter codes from the character set of the equipment, sothat in principle any device can be used.
(d)

(2)

Processing format.

Facility in addressing individual data elements,grouping of data elements into segments, etc.
(a)

Flexibility of structure, so that either fixed orvariable fields can be efficiently handled by software.
(b)

(c)

Repeatability of any data element and its format

identifier.

Versatility such that the format can encompass
within a consistent structure, the varying content of records for
all types of library materials: monographic, serials, journal
articles, etc., in either conventional or augmented form.
(d)Minimization of character-by-character scanning,e.g., when preparing foreign language fields containing diacriticals
for output display.
(e)Provision of capability for generation of sort keys
with the minimum number of additional codes placed in the record.
(f)

(3)

Mass storage format.
(a)Compactness of representation.

(b)

Fast translation from and to the internal processing(c)

The effects of translation errors are localized orformat.

correctable.
(4)

Communications format.

Processable on computers with different memory andlogic organizations.
(a)

(b)

Low programming requirement to utilize the record.

There are many other requirements which the formats shouldmeet. However, these will be sufficient to demonstrate the conflicting requirements from function to function. For example, there
is a conflict between the compactness of the mass storage format
and the facility in addressing individual data elements of the- 42 -

processing format. Also there is a conflict between the processing
format versatility and the ease of coding the input format, sincein general the fewer the things to be remembered the easier the codIt is true that even in developing formats for the separate
ing.
functions it is unlikely that our initial formats will remainunchanged throughout the study. However, regarded as a system offormats, they are likely to be more satisfactory than a single
format would be.

Irrespective of the discrete functional demands placed on thesequence of formats, they all serve a basic purpose in common:
they emboay coding to identify record element content by "kind"
and, where applicable, by the role played by the data contained
as a value in an element. A name is an example of a kind of data
element. A name may be one of several sub-types and can act in
one of a nunber of roles in relation to a document. For eachrelationship it is assigned to a different field which identifies(E.g., kind = personal name;at once both its kind and its role.
role = added entry, alternate author of document.)
In summary, the coding in the formats performs four importanttasks. It supplies:

Information about the document itself (title, subject
headings, all the conventional bibliographic data).
(1)

Information about the file (whether the record is new
to the file, which file it belongs to, record status, etc.).
(2)

Information about the record (to uniquely identify it,
to characterize its composition, to distinguish it from othertypes of record content, etc.).
(3)

Information about fields or about other codes (e.g., thata name in a field is of a certain sub-type; that a tag is to beprocessed under a ,.:ertain condition, etc.).
(4)

Input Format. A summary of the approach underlying thedevelopment of the input format at ILR can be found in a previous
Institute publication.* The present format is an extension andelaboration of work reported therein.
2.

Background. The input format described here, andthe conversion procedure outlined below (in Section IV) represent
an approach chosen in the earliest stage of the data base preparation task. The initial work has resulted in a draft coding manuala.

*Cartwright, Kelley L. and R.M. Shoffner. Catalogs in Book Form:
A Research Study of their Implications for the California State
Library and the California Union Catalo, with a Design for their
Implementation. Berkeley, Institute of Library Research, University of California, 1967. pp. 30-35.
- 43 -

We anticipate that the input format
for use in manual editing.*
and the editing procedure will evolve towards greater reliance on
computer-assisted field identification. As our experience isevaluated, conclusions will be drawn about the feasibility ofplacing more emphasis on such computer-set codes. The ultimate,of course, would be optical scanning of the unedited cards or astraightforward keyboarding of them. There would be no intervention by a buman editor until after processing by a computer program.
This program would format the record and then print out the resultsfor post-edit inspection by a human and correction as needed.
Corrections to errors in coding committed by the edit program
algorithms or the input device operator would be input to thomachine file in the normal fashion. We are currently working
toward this goal with the automatic translation of the Santa Cruzrecords. These records have only the most general level of fieldcoding.

Coding scheme. Since most of the catalog data isvariable in length, the bulk of the logical fields have been definedas variable for the input format as well as the processing format.
The advantage to this is that in conversion from 3x5 cards a morenormal text typing rhythm is permitted than if a fixed length fieldapproach were used.
b.

More than 120 data elements and fields have been identified
in the MARC II format, and equivalent ILR input tags have been
assigned for most of them. These are translated into the MARC II
tags and indicators in the ILR processing format.
The maaner of identifying the variable fields for input is a
key feature of this format. A mixed technique has been employed:1) insertion of signals which explicitly label the beginning andend of each field, and 2) depending on the predictability of theoccurrence of fields,.the insertion of an identifying tag. The
first is a uniform symbol for fields which are always or nearly
always present, and which always occur in the same sequence. The
second is a unique symbol applied to those fields which occur infrequently or in isolated sequences.

The first type of coding is applied to the body fields (author,
title, etc.). In this case, no field code may be omitted, eventhough the field contains no value in a particular record. The
ftblank" field is purged from the input record at the time of its
transformation into the storage record. Ten fields were chosen forcoding by this technique on the basis of previous experience andA symbol easily insertable in the dense text of thesampling.

*Cunningham, Jay L. Instruction Manual for Editorial Preparationof Catalog Source Data. Preliminary Edition. Berkeley, Instituteof Library Research, University of Calif,,rnia, 1968. 172 p. Because this manual is a draft which is expected to be revised it
has been provided only in limited quantity.
- 44 -

catalog card was selected: the slash mark, "/", is used to signal
the beginning of each of the "body" fields. A blank field, e.g.,absence of a publisher name, would be signalled by two slashes,"//"

The second type of code is two-character tag composed of a
uniform special character plus a unique alphabetic. When a field
is not present, no tag is inserted. Two series of these codes
were defined. The more frequently occurring fields are coded witha combined asterisk plus lower case alphabetic (for ease of input
typing). The less frequently found fields are identified by an
exclamation point plus a single character alphabetic. The "*" or"I" is needed to distinctly identify the tag, otherwise the edit
program could not distinguish reliably between the code letterand a text letter.
A further conciseness in input coding derives from the convention that the editor need write only the lower case alphabetic
code on the sheet, in the case of the "asterisk" series of tags.
For distinctiveness, the code letter is written in red. The input
device operator must recognize the letters written as tags andpreface each "red letter" keyed with an asterisk. For the otherseries of tags, the editor writes out both the exclamation point
and the code letter.
One problem with the brevity inherent in this system is its
lack of mnemonic value. The input tags used in the LC MARC
production system, for example, are more easily remembered than
combinations such as asterisk plus a letter. However, we havefound that a simple checklist supplied to the editor as a readyreference tool suffices for the majority of tags not quickly
learned through repetitive use.
Coding sheet. Information will be input to thecomputer from a coding sheet on which a catalog card is reproduced
or attached. The principal reasons for the use of the coding(1) it provides space in which to record certain insheet are:(2) it provides
formation which is not explicit on a catalog card;
space in which the editor of the data can write information which
represents additional data required, or modification of information already on the card; and (3) it provides checklist remindersof some of the coding conventions and options.
c.

In Fig. 11 we present an example of the coding sheet developed
for the project. The fixed field alternatives and certain otherThe
codes are printed at the left and at the bottom of the sheet.
tags and field names used in the ILR input format are lib-bed inAppendix V.
Processing Format. The details of the processing format
for the File Organization Project data base are described inAppendix IV. The salient features, and certain significant
3.

- 45 -

100011
FIGURE 11:
CODING SHEET - MONOGRAPHS

ild)ash, 10
non-uniform
Qua7tzation of signals id
Calif.,TRW
Space Techedondo Beach
teps

A644

rec:ord_6
Date 2

Date 1"7I

12 daLes - 2d is t6i-minal.10emb#OLOW Information measureseat. 02; Si:vials andsigaaliagdilW nektroavyaz ay. 10, p. 1012, 1957.

IDate not knownnnbc21----1DAgits missingbs(ST

44CTranslation of Kvantovanie signalov s neravnomernym shagom from Blektrosvyaz, no. 10, pe
10-12, 1957.

4,

DAYE TYPE:

bri/
os1964.
Technical LibrarySTrans-

28

I803m",

bc'I2 dates - 2d is CD
130

Glogy

IReprodireprt.-no dig. out
1Single dt - no digits out

de!

B1BLIO. LEVEL:
Analytic
Col1eGtive

arill'Monograph-1-1-ratttrlar 4SerierrrVbpace

HOLDINGS:

Shelf
Cop.No.
ThisLoc.
Loc.
_ - _ .
_ .Camnusi-Br.

Call No.

dd V Series

cc!
ed]

bzi

d

jb

eb1 0/Local orig.

jc

1NUC

_Other

Total
Copien11 firIL

if

CAT. SOURCE:(CentralTechaology Labor-

Aageles.%Techaical Libatories, 1m0., Lesrary.vrratslatiolaio. 80.).4

ri

jd

ILC call no. bracketed
j e
-

JJ

ar- 10RIG. AT,GOVT. PUB.:
Federalkb) ____1Cal. StateADAPTED AT

Interm.t'1
ker-7-10tber govts.
kciI,

MICRO-REPROD.:
Microfilm
Microfiche
Micro-Opaque(kd

jCal. Co./muni.

mai(CONFERENCE PUB.

nat

'MAIN ENTRY IN BODY

1---

CONTENT FORM:
Bibliogs.
Iii;)

hci
hed

hel
hf1

hi(

hj(
hki

LITERARY GROUP:

1Complete/coll. works
Fa)pD----ISolected workf,
1Prolific
pc
E--Catalogs
Indexes
Abstracts
Dictionaries
Encyclopedias
Directories
Yearbooks
Statistics
Handbooks

iMo. I Da. Yr.,

CANCEL TITLE A.E....DICT. CAT.
CANCEL TITLE A.E....DICT.WIV.

E.5171

raf

044

4/LANG.

100 TTiA N'YLATION YESTYPE OF MA1N ENTRY
ua
IT----1-MAIN ENT. IS SUIV.
1MAIN Ei;';'. IS

PUii

11.E1

ADDED E2,TRIE3 TYPE:

Serivs traced same
Subjects & subdiv.

Author edor Vtle
Series traced diff.

W

ELL .2 2

J.5

11111

400_400.__J1111

differences in form and structure of this format in relation to
the input format on the one hand, and the MARC II Communications
Format on the other hand, are outlined briefly below.
Input and processing format differences. There are
two main differences between the input and storage formats:a.

Certain data elements and codes are set by
Default.
default in the edit program and thus do not appear on the coding
sheet or in the external input record. An example is "Type ofRecord = a-- language material, printed."(1)

Coding notation. A briefer but flexible form of fieldcoding (as described above) is used for input rather than threeor four character numeric tags as in the MARC system. The threedigit MARC II tags and associated codes are set by the INFOCAL
edit program, which translates the input codes into corresponding
internal processing format codes, as it compiles each field byconcatenating the various input codes and elements that make up
the internal record. The input record image is, in general, nota mirror image (identified by different field codes) of the resultant internal record in processing format. This is because of1) the default values, mentioned above, 2) separation of partsof certain tags and elements on the coding sheet for purposes ofeditor convenience, which are brought together by the edit program
for the internal record, and 3) the more complex structure of theinternal coding, due to the fact that a record directory method isused to organize the processing record.
(2)b.

Processing format and MARC II differences.

Figures 9 and 10 in Section III.B. listed the few minor
differences between the File Organization record content andthat of the MARC II record. These differences were primarily thoseof inclusion/exclusion. The great majority of data elements that
constitute the content of the File Organization Project record willbe identical to those in the MARC II reord.
(1)

There are very few differences between the File Organization record and MARC II in terns of coding - i.e., the same fieldand sub-field tags, delimiters and indicators which identify andIn a few cases ILR will
characterize the data element content.
not be able to supply a code for a data element identified in
MARC II, and in a few other cases, we elected to go slightly beyond
the Library of Congress in identifying a field to serve special
requirements.
(2)

In respect to record structure, the File Organization record is quite similar to MARC II - a fixed length Leader is createe',with quite similar purpose and content as MARC II. A Record directory controls the access to the remainder of the record, including
the Fixed Length Data Elements field, which is treated forprogramming purposes as if it were a variable field length field.
(3)- 47 -

In all cases, our goal has been to conform to the utmost
extent to the MARC II design, in order to minimize software differcreation and records
ences in routines to handle records of our own
In thisoriginating elsewhere, e.g., in the LC distribution service.
MARC II
regard, we expect to do no reformatting of the content ofaddition of data willrecords received from LC. A certain minimum
local call numbers,
be necessary, e.g., a local master record number,structureetc. Lastly, a certain amount of reformatting of record
marks fromis planned, e.g., to move the codes for diacritical
This will facilithe text of the field to a special field header.
terminals having
tate preparation of the field for display on CRT
experiments in which
limdted character sets and allow us to begin
language text inthe very highest quality of display of foreign
bibliographic records is not a significant factor.

These differences can be reviewed in more detail in theprocessing format specification in Appendix IV.
In summary, these differences in formats are not considered
to be departures from the concept of a standardized bibliographic
specifically intended to be adaptable
record. The MARC II design isdata and coding.
to local needs, and hospitable to non-conventional
Standardization does not imply a rigid, uniform set of data elements
constituting a monolithic structure. Instead, through the use ofof structure,
the mechanism of optional tags and fields, modularity
comply withand a hierarchical coding definition, libraries can
MARC yet tailor it to specific requirements. The acceptance oflevel in thethis approach is already perceptible at the national
MARC IIefforts of the three U.S. national libraries to support
certain of their elements
as a common standard by accommodating
and codes
to a common definition, while defining other elements
for their awn particular applications.
D.

THE REPRESENTATION OF TYPOGRAPHICAL CHARACTERS*

Overview. Once the input format and the storage format
designed, the problem of keyfor bibliographic records have been
keyboards of stanboarding textual material not represented on thedard devices must be faced. Catalog records contain tremendous
data (usually
linguistic variety, because certain portions of thein the language ofat least the title) are by convention recorded
must
the text or its translation. Thus the conversion system
alphabets and theembody procedures for keying both non-Roman
diacritical marks, symbols, and other special characters, some ofwhich may occur in any alphabet. Moreover, provisions must be
when they
made for efficlent representation of such characters
operating code set of the particuare not defined in the internallar computer used.
1.

*This section was prepared by Thomas Hargrove, of the- 48 -ILR staff.

In line with the goal of minimization of the overall cost of
conversion, the basic approach taken was not to secure speciallymodified input transcription devices which would contain a limited
number of symbols and special characters beyond the standard set.
It was felt that this approach is too costly at this time andrepresents only-a proximate solution to both the input and thestorage of special catalog data. Rather, a more universal concept
was developed, wherein the standard keyboard of a readily available
This
keypunch or other input transcription device could be used.
would require only a small amount of extra operator training andwould be virtually unlimited in its potential for expansion tocover symbols and codes not foreseen at the beginning of the conversion.
General Objectives. The objectives of the physical data
representation technique are:2.

Input device independence. To handle the extensive
set of alphabets and characters, a method must be established torepresent those characters by a notation that can be implemented
on any equipment.
a.

Compact internal character set.
The codes used on
the input device must be transformed into a concise internal set.
This will be based upon the specific arrangement of the computer
upon which the system is being implemented, but need not be logically tied to a given machine.
b.

For data exchange purposes,
Standard interchange code.
such an internal code set could be translated to the ASCII extended
character set being proposed for standard library use by the Libraryof Congress.
c.

Ease and economy of codinE. The technique developed
at ILR assumes two aspects to the representation problem: conversion of individual special characters, and conversion of specialalphabet streams (e.g., text in Cyrillic). At input, it is desirable to have a mnemonic code for those individual characters noton the particular keyboard employed. Yet it is desirable also tohave concise codes. For a special character sequence, a control
or escape code is needed to show the beginning and ending of thesequence and a shift to the original or to another sequence.
d.

Each of these requirements poses special difficulties in thedesign of an economical special character coding technique.
Transliteration problem. The conversion of specialalphabets (e.g., Cyrillic ) could probably be efficiently handledby having special masks placed on the device keyboard and trained
language operators type the foreign language records using thespecial alphabet control signal. The entire record would be stored
in an internal configuration translatable to the external display
repertory desired. For example, on a CRT device, the record storede.

- 49 -

in the coded equivalent of Crrillic could be transformed automatically through a table for display in transliterated or Romanized
form on the screen. A printed transliteration table would beavailable at the console for the user to convert from the transliterated form into his own language. It is recognized that thisapproach is fraught with several dangers since there is no unified
agreement in the linguistic world on standard transliteration or
Romanization schemes.

A remaining task is to establish the precise mapping of the
input character codes into the particular computer character set
available in the Project facility. Numerous smaller problems havealso been identified, such as case change, spacing, and a signalthat a character used as a format code has occurred in the character stream as text ard should be so treated.
The remainder of thissection summarizes the rules for input
keying of either individual special characters or of character
sequences.
Basic Assumptions. The assumption underlying the proposed
input codes is that all special characters can be keyed from aphysical point of view. Such an approach night be called the"printer's" point of view, in contrast to a computer point of viewof subsequent encoding of characters economically in storage, orto a user's point of view of the semantic or logical meaning which
the characters serve to express.
3.

A printer, without knowing the subject, is able to set characters by knowing only their physical sequence and alignment in
A keying operator, similarly, should be able to keyboard
text.
codes for the physical shapes and alignment of the same characters.
In this approach, the codes typed would not depend on the operator's
having to know the meaning, or how the character would be coded incomputer storage. Instead, the codes should be those which makeinput easiest for the keying operator, with the highest accuracy.
This proposal considers all special characters as establishedshapes, in a given horizontal position, with a defined verticallocation, and physically imposed upon one or more textual positions.
The approach is aimed at providing the benefits of:a.

short codes for the most frequent characters;

minimal look-up for less frequent characters by useof combined codes of basic shape+location+applicability, in lieu of
looking up arbitrary, non-structural codes;b.

c.

method for adding as many codes as needed;memorizing by keying operator of most frequent character codes plus the few basic qualifying codes.
- 50 d.

SRecial Character Encoding Situations. For the purposesof input a special character representation scheme must, inprinciple, addrees five separate but interrelated situations that
may occur in textual material:
4.

e.g., Cyrillic andSpecial (non-Roman) alphabets:
any other alphabet not represeated on standard input devices.
a.

Special marks: within any given alphabet, e.g.,
diacritical marks (acute/grave accent, etc., and combinationsof these, digraphs, etc.)b.

Font change: within any given alphabet, foni, canSignificant
be either significant or aesthetic (for emphasis).
font change is one involving semantic change, e.g., bold face todenote vector quantities in mathematical notation, italics todenote variables, etc. Aesthetic font changes usually add emphasis(e.g., italicized words), provide reader aids (e.g., boldface subheadings in text), or have other not strictly semantic purposes.
The latter have been excluded from the technque devised, but could
be handled in principle.
c.

Special symbols: characters not on the standard keyboard, which may or may not have conventional meanings. They may
be language-dependent4.or context-dependent for their meaning (e.g.,of in British money; 4 has a special meaning on an LC catalog card.)d.

S ecial positional confi uration:of any character
E.g., exponents and subscripts have a conventionalin any language.
position and may have smaller point size in a string of text.
e.

Several of the above conditions may apply to a given character.
The single special character in a stream of text may be considered
as a special case of the problem of handling the entire stream ina non-keyboardable alphabet.

To devise a systematic coding scheme to cope with these text
situations, we redefined them into the following categories:(1)

The alphabet can be either:

Regular mode - the Roman alphabet as represented on the
keyboard of the particular device used, is regarded as the nominal
or default case; orSpecial mode - alphabets other than Roman, e.g., Greek,
Cyrillic, etc., not represented on the keyboard of the device used.
(2)

A character stream in a given alphabet can be either:
Unspecified length; orSpecified length.
51

A s ecified len th character stream is initiated by ashift code signalling either a:(3)

Fixed len th code (predefined) standing for a special
character; or a
Variable len th code (e.g., Ebbreviated words predefinedin a program table

Breaking the textual situations dawn in this manner allows
us to think how the keying might be best performed in a continuous
fashion with the least stop-and-go motions by the device operator.
The data can be regarded as contiguous blocks or strings withina characte/ stream. Unit codes are applied by the operator tocontrol each block that departs from the default situation ("Regular
To illustrate, a catalog record in which a number of spemode").
cial text situations occur can be keyed as a series of contiguous
blocks forming the total record:
FIGURE 12:

Start-of-record:
REGULAR
Unspecified...
Length Set

KEYING BLOCKS OF TEXT

shift toANY
Specified

return to
REGULAR
Unspecified...

shift to
SPECIAL
Unspecified...

Set

Set

Set

shift toANY
Specifiedshift to
SPECIAL
Unspecified...

Set

Set

ANOTHER SPECIAL
Unspecified Set,
or RETURN to REG.
by.Termination
Symbol or End-ofRecord

Specifically, the text categories defined above would beapplied as follows:
For normal Roman alphabet text not modified or interrupted
by change in alphabet, by diacritical marks or by any characters
not present on the keyboard, no special action need be taken by
the device oper4or. Each record would be assumed to start its
text in "Regular" mode.
(1)

When any of the three categories of text situation occurs,
as a departure from the Regular mode, the device operator can still
continuously input data, in any alphabet, with or without diacritic
marks or special characters, merely by input of a shift code when(2)

- 52 -

departing from the normal situation. The "Regulcir" alphabet
defined for input keying may be somewhat Lxpanded for storage
For example, the S/360 internal set has 256 possible
pul-poses.
codes, of which a number are already designated for graphics notpresent on the standard keypunch. It would not be necessary tostore the input codes in the exact form in which tney were keyed.
Moreover, the convertibility is three-way: the input codes may
be transformed to the most convenient and efficient storage codes,
and the storage codes may be again transformed to available output
devices, e.g., printers.
While keying in "Regular" mode, each character represents
itself exactly as denoted on the keyboard. At the point of interruption, subsequent characters stand for either a character inanother alphabet, a diacritical mark, or some other special character not denoted on the keyboard.
Two types of departures from the Regular mode can occur:
Unspecified-length shift to Special mode, followed at some finite
point by either a deshift to Regular or a shift to another Specialset.

For example, a book title might have a Greek word embedded
in an English phrase, followed by an alternate title in Cyrillic.
The deshift would be signalled by either a termtnator code
or automatically by the end-of-record mark input by the device
operator, if appropriate.
Specified-length shift codes can be of two sub-types:
Fixed length codes, which are either one or two-character1)
format codes recognized by the input edit program as field signals,delimiters, etc., or are these sane characters themselves precededby a reserved symbol to show that the character is in this case to
be regarded as text data, not as a code.
Other than these occurrences, a number of doublet codes havebeen provisionally defined to represent diacriticals, e.g., "OE"
for acute accent. Triplets, etc. could also be specified, in orderto make the system expansible.
The second sub-type is variable-length specified shifts,
as in the case of groups of characters predefined in a programtable to represent abbreviated words, either for the purpose of
economy of keying frequently occurring words in text, or for corapact storage, or both.
2)

The Identification of Shape, Size, Attitude, and Relative
Position. Diacritics in particular manifest many combinations ofshape, point size, inclination or attitude, and position relative
to the character they modify. For brevity, we shall discuss thesecharacteristics using the single term "location".
5.

- 53 -

Special characters
Order and location of characters.
or Special uses of any character are considered for keying purposes
to be in the same position or in juxtaposed positions of the data,left to right.
a.

A

Data

1 2 3 4 5 6 7 8 9Position No.

For purposes of input, all special ctharacters are coded inrelation to other characters by both physical shape and horizontal
and vertical locations. The same shape in different inclinations
is considered a different shape, for coding purposes:
1d differs from ?

adiffers fromadiffers froma

The same shape character in different horizontal and verticallocations is coded to express both uniqueness of shapes and position in relation to a character to which it is either attached or
juxtaposed.
Before

With

After

-d

a

d-

-d

El

d-

_d

d

d_

High
MiddleLow-character

All characters including diacritic marks at the same
position are considered vertically in a regular location, or inhigh, middle or low locations (drawn large for illustration):b.

Location:

RegularHigh
MiddleLow(Circumflex is High)(Slash is through Middle)(Letter 0 is Regular)(Dot, or period, is Low)

Alphabets, digits, punctuation, and additional characters,such as +, 4, are at the Regular location.
(1)

Diacritics and special uses of any character, such as
exponents and subscripts, are at High, Middle, or Low locations.
(2)

-5)4-

6.

Initial Codes for Provisional Im lementation. A doublecharacter code set for modern European nharacters was definedusing the ten digits to represent the most frequent diacriticalmarks. The most frequent letters specially formed with diacriticalmarks, and other separate diacritical 'darks would be represented
by the letter codes. This double-character code set would serve
By being fixedas an extension of the Regular character set.
length, the diacritical mark codes can be used in the midst of any
special alphabet set.

The location of a diacritic in relation to a letter is impliedby its code. The implication of this is that any special character
which exactly coincides with a Roman letter and a diacritic markmay be coded either as a letter plus a diacritic mark, or as asingle combined code. The frequency of certain diacritinally marked
letters in certain languages might make single combined codes very
useful. However, for the present, the letter-plus-diacritical markcode is proposed as a scheme applicable to any particular language.
Unfortunately, a 36-characte/ double code set is not enoughto encompass all ne diacritical marks. LC lists at least 64 special characters beyond the normal keyboard, in its ASCII extension
proposal. These could be assigned single coue characters in two
separate sets, but for simplicity a two-character code is assigned
provisionally to the less frequent diacritics. The order of the
LC list does not reflect th, frequency of occurrence of the marks
themselves. LC's Information Systems Office is making a statistical
study of the frequency of occurrence of diacritics, and when thiskind of data becomes available, it will guide the refinement of thesystem proposed. In advance of such quantitative design information,
we have used the diacritic set proposed by Palmer (see Fig. 13),) guide the establishment of the initial set
and our own research(see Fig. 1)t).

When the frequency distribution of the diacritics and othersymbols is better established, we will implement the LC list as
needed, to assure full convertibility of the textual content of thedata base to a MARC II character set standard.
Codes for special characters and for special uses are made up
of a combination of a flag plus "0" plus digit(s) or letter(s).
The most frequent codes have defined shape, location, andDiacritic characters,
number of positions applicable in one code.
for example, occurring most frequently are coded with briefest codes(given vertical position assumed in each case as the default position).

E.g.:acute (high, one-position) is coded OA

micron (breve, short)(high, one-position) is coded 0
cedilla (law,- 55 -

tt)

is coded OC

FIGURE 13:

THE TENTATIVE HARVARD LIST OF DIACRITICSILR 029
Codes

Harvard Listof Diacritics*

Comments'acute

OE
0A
OF
OU
ON
OC
OK
OG
019

3 cedilla' hacek° (as in Swedish g, Czech ta. )underline0/(as in Danish16)' grave

A circumflex
dieresis or umlaut"tilde...

(letter 0 )...O.M- (as in B)

0/

Anglo-Saxon D differs**Display size adapts
to letter(for Polish /)(letter 0):1S2W(comma under letter as in
Rumanian)(dot below letter)(dot above letter)t (used in Polish &Lithuanian)ID, ikused in Hungarian)OR-macron

,OL
.0L
.0H

01W

Angstrom
Digit= No. of lettersunderlined.
**Display size adapts
to letter)Inverted cedilla

micronNote:

Unique letters after 0 were chosen as non-shift, andto coincide with frequently associated letters where
possible, with allowance for specially reserved gymbols.

*Source: Palmer, Foster M. °Conversion of Existing Records
in Large Libraries; with Special Reference to the Widener
Library Shelflist." In: Harrison, John and Peter Laslett,
The Brasenose Conference on the Automation of Libraries.
eds.
Held at Oxford, Eng., 30 June-3 17117776. London, Mansell,
1967.

p. 74.

**In display the slash through "Z" of Polish font will be
shorter than the larger slash through "0" of Scandinavian font.
Slash, as cross-over, can be keyed as @/ in conjunction
with the 01119 code for multiple applicability.
- 56 -FIGURE 14:

ALPHABETICAL INDEX OF DIACRITIC CODES
Code

Mark

Name, Defined Loc.

0A

Grave (high)OB

Breve, micron (high)OC

Cedilla (low)

0D9

at

Acute (high)OE
OF

Disgraph, up-to-9

A

Circumflex (high)OG

Angstrom (high)OH

Undefined high location

0I9

Underline, up-to-9 (low)
Ligature, up-to-9 (high)

0j9

OK

./

Caret, Hacek (high)OL

Undefined low location

OM

Undefined middle location

ON

Tilde (high)

0.9

Specified number to 9 of non-keyablecharacters

00

Slash (through middle)

OP

Slant-in-font series (Italic)
Weight-in-font series (bold)-

OR

Macron (high)OS

Special alphabet shift

OT9

Multiple-applicrldlity (to 9)

OU

Umlaut, dieresis (high)OV

Reserved-further coding

OlW

e.

Inverted Cedilla (low)02W

Double acute (high)03w

Reverse comma (high)04W

Double underscore (low)05W

Candrabinde (high)

06W

-D

Demi-bar in Ang. Sax. (middle)

OXIDE

High or middle or low illegible

0..

Unspecified number of non-keyablecharacters
57

FIGURE 14 (Cont.):

ALPHABETICAL INDEX OF DIACRITIC CODES
Code

Mark

01Y

t

Dagger

02Yt

Double dagger

03Y

01E.

04Y

Pound sign (English)

Inverted question

05Y

I

06Y
07Y

Names Defined Loc.

Inverted exclamation
Thorn (Icelandic)b

08Y

Musical flat
"Undotted i"

09Y

4

Single left quote (high)

010Y

44.

Double left quote (high)

011Y

/7

Double right quote (high)

012Y

<<

Left Cont. quote

013Y

>>

Right Cont. quote

OZ

Astride symbol00

Termination of special alphabet(de-shift)

FIGURE 15: PROPOSED SINGLE KEYING CODES COMPATIBLE WITH
TRANSLITERATION SCHEMES FOR MODERN CYRILLIC*
RmaiemUpper
Case

LetterOm). Ital

AppliedMechanics
Reviews

U.S. Library

Journalofof Congress

Symbolic

a
b
v

a
b
v

a
b

Logic,

Lower
Case

A

A

B

E

a
6

V

B

e

a
b
v

G
D
E
2

'r

:

g

g

g

g

a

d

d

d

ra

E
g

e

e

e

i

ti

g

6
o

d
e

X

AK

ox

zh

A

i

x

Z

3

3

z

z

z

z

I

II

i

i

i

i

J

I/

I

1

j

j

K

K

u
a
K
A

k

k

k

k

1

1

1

1

na

m

L

JI

.

v

2

M

Ai m

ma

ma

N

II

m

n
o
p

n
0

n

n

n
o
p

P

P

r

r

r

0

0 o

P

II

o

R

P

p

r

S

C

c

a

a

8

S '

T

17

m

t

t

t

t

u

u

u

u

f

f

f

f

kh

a

h

h

C

Y y
0 0
X x
U

te

;

o

c

3

q

m

dh

6

3

w

111

us

6

A
A

A

w

4

III

u#

&eh

oh&

A6

4

U
F
H

5

II

1

5

s

Ed

u

b

b

3

3

Y
6
7
6

H) m

9

SI

Q
one 1
zero 0

#

A

I

i

6

e

y

i

6

yu
Ya

Y

6

el

iii

fi

a

6

f

Y
6
7
8
9

4
1 one
o zeroCAPITAL letters keyed with Underscore (0-5-8) before code.
Difference in fonts implied in code for Character Set.
Other characters (such as V, transliterated by L.C. as 4Y) can becoded as elements of another character set, acting as a miscellaneous, overflow, or Slavic character set containing all Cyrilliccharacters not already coded in the Russian character set.
*Source: Mathematical Reviews,(Am. Math. Soc., Lancaster, Pa.)v. 30 (1965), p. 1207.

- 59 -

The code for the diacritical would be keyed following thecharacter which it modified or with which it is associated. Special conditions for which this rule is ambiguous and requires
further specifications are provided for in the keypunching manual.
Particular Scheme for Cyrillic Alphabet. For Cyrillic,
it is proposed that a 36-character code set (26 Roman alphabet and
10 digits) to be used for keying and storage is more convenient,economical and unique for operator recognition purposes than transliteration schemes of a linguistically controversial nature. Withunique codes stored for every Cyrillic character, printout can bemade into Cyrillic, or into any transliteration scheme desired.
What is left undefined is a complete transliteration scheme. It
is emphasized that it is the alphabet (made up of graphemic units)
which is being coded, not the "language" in transliteration. A
Russian word that already appears in Romanized character formcannot be coded so that it can be printed back out unambiguously
in Cyrillic, due to the non-agreement on transliteration. Roman
letters, witn or without diacritical marks, are keyed as Roman(regular) character set, whatever the language the letters themselves transliterate.
7.

It is also proposed that the Cyrillic coding scheme include
two single-character sets: one single-character Russian (East
Slavic) Cyrillic set - includipc; three pre-1918 characters - andone single-character (Other Slavic) Cyrillic set to handle theoverflow, rather than to provide a double-character set for all
Cyrillic characters.
Fig. 15 is a proposed code for Russian Cyrillic character
It adopts as much as possible the characters common to manyset.
transliteration schemes, several of the more commonly used of which
are shown in juxtaposition.
E.

LOGICAL SIMILARITY OF BIBLIOGRAPHIC RECORDS

Groups of records in retrieval processes
General Remarks
may be defined according to similarity of subject content, or in oneof several other senses. One condition for search execution (thestage which performs the actual retrieval and assembly of therecords or elements a user wishes to inspect) is the definitionof rules to aid him in deciding what will be acceptable to him interms of closeness of match of the file responses to his request,
as he carries on his dialog. Howiver, a user must formulate arequest based on the clues available to him when he initiates his
probe of the file, before the individual records or groups ofrecords which satisfy these clues can be presented to him. Therefore, the most should be made of these clues in order to makesearching in an on-line environment satisfactory.
1.

.

Automatic Error Control in User-File Interaction. One
characteristic of these clues is that they will often have error
in them and thus an exact match on the request is not always the- 60 2.

"Near misses" or assistance in correcting"best" retrieval.
the error should be provided. Accordingly, a task was defined todevelop automatic techniques for error detection and correctionof request messages at the terminal. Because of the linguistic
and statistical nature of the task, a consultant with experiencein this area was employed to start the work. A technique which
we denote as "name compression by equivalence class algorithm"
has been developed. The background and reasoning for the algorithm
itself is described in a separate paper by James Dolby, included
as Appendix I to this report. The purposes and provisional resultsof tests of the algorithm are summarized in the remainder of thissection. Although the specific intent of this algorithm is error
control, it has usefulness in file search beyond that. For thisreason a more general perspective on the concept of "closeness"is presented first.
Closeness in Name Searching. A measure of closeness canhave several dimensions, e.g., specific subject similarity or
associativity, likeness of one or more properties such as set
inclusion (several books by one author), and a number of other
It is possible to apply such a measure to relations
relations.
between records in the file, and between records and requests
made of the file. And it is possible to applj such a measure toparts of logical records, e.g., author names. We can consider
two major classes of file organization and search, and of therequests that each can handle. The first class is the general
problem of match between requests and file responses which arein some sense close to the request. The other class is a special
case of the first, in which closeness is defined as exact matchof the response to the request. The exact match problem, as thesimpler of the two, is the logical place to start the research.
3.

At the time of search specification, the user has some initialclues which may or may not be well-formulated. These clues, suchas author name, subject terms, etc., are used to make up a searchrequest. The request will be expressed through search keys transmitted through a terminal device. If the clue is an author name,the user will key in all or some portica of the name. Two problems
arise: no response may be obtained from the file corresponding
to the key as initially expressed, either.1) because no record
at all is present which identically matches the request as expressed, or 2) the key may contain errors (e.g., form of name atvariance with the form in the machine file, simple misspellings,
phonetic misunderstandings, name changes, etc.), thereby causing
either no response or "false drops." If the search key was
accurately known at the time of input but response is nil or
minimal, then other search aids such as associational and relational techniques can be invoked (i.e., searching under relaxed
conditions), and the search can proceed or terminate when somekind or amount of useful search output has been obtained. Alternatively, the user might proceed as if the search key were inaccurate and expand his field of search as follows.
- 61 -

If the exact spelling of the name used
"Noisy" Matches.
as search key is not accurate ("guess-match"), an error correction
technique becomes immediately useful. The facility should be able
to provide capability to be invoked automatically or at the optionof the user. It might be executed automatically if the user
thought he had input the correct spelling of the author's name,but received a negative response from the first search of the auThat is, the system should notthor index file, for example.
be allowed to "give up" just because the user missed on his first
try. The search control program would, upon notice of a negative
search, retrieve synonymous names, if the name input in the request was spelled closely enough to that of one or more names
existing in the file according to some probabilistically determined(The precise nature of the "equivalence class" facility
threshold.
It will probably be tested at first byis yet to be determined.
establishing a file of fixed-length coded classes. An alternative
method would be to generate interpretively the names which are
members of the class, at the time of input. Another combination
might be to invoke the algorithm inside the machine but have alist of classes at the console as a user aid.)
4.

The possibilities for the desired name would then be displayedfor consideration by the user. This is a kind of grouping of partsof records (e.g., a cluster of similar surnames of different authors)
one of which may turn out to be the one which the user is seeking.
The names are regarded as related in that they have close spellings,
a linguistic property exploited in a number of name compression
systems.

In this sense the error control feature acts as a filter on
the request, self-activating upon certain conditions, on demandat other times. Although searching via author keys may not be themost important file access point,* keyboarding of proper names isthought to be the area most vulnerable to input errors. It is
anticipated that the equivalence class technique can be extended
to non-name searches, e.g., words in title and subject index files.
Error correction will be useful in any of the usual searching situations, i.e., for the person who cannot remember the spelling of an
author name for a book he has seen before, or to handle requests
based on bibliographic references which may contain spelling errors(the "bad citation" problem).
*Recent research reveals that for the type of catalog search fora book with which the searcher has had previous contact, only alittle over 20% of a sample of users surveyed could remember authorname clues. However, it was pointed out that of the searches undertaken using author name as an access point (or author + title), thecause of failure was lack of a method of manipulating incorrect
author or title information in order to make it operative in catalog searching.
See Vaughan, Delores K.
"Effectiveness of BookMemory Data for Conventional Catalog Retrieval."
In:
Chicago.
University. Graduate Library School. Requirements Study for Future Catalogs; Progress Report No. 2. Chicago, Mar. 1968.
(NSF
Grant GN 432), p. 53.
- 62 -

5.

Equivalence Class Algorithm.

General objectives. The handling of large bibliographic files presents two levels of error control, as pointed
out by Dolby.* They are 1) error detection and correction during
the input (file generation) cycle or as a result of feedback byusers to correct the file; and 2) the system's reduction of theeffect of the user's own errors, committed as part of his search.
The use of equivalenct: classes is not restricted to error control.
The general objectives of a compression scheme may be:a.

for automatic cross-referencing among similar names,
where there is not an c:rror involved, but a file authority situation.
(1)

to confirm or establish a "guess" match th,"ough "noise"i.e., misspelling.
(2)

to help eliminate misspelled words from the new records
updating the file.
(3)

to save keystrokes both to speed up the request andreduce keying error.
(4)

to achieve data compression 1-2r se, in storage (reduce
disk space for an index file and for programming convenience inprocessing fixed length entries).
(5)

Our motivation in developing the equivalence class algorithm
is primarily as a user aid in relation to (1), (2), and (4) above.
1LR objectives. Work to date in 1LR has st7 ased
the first of two areas of immediate concern: proper names asmatch elements in catalog searches, and regular vocabulary suchas words in titles. The approach taken in the work on authornames was to find a way of gathering together "like" names systematically so as to identify similar spellings (and possible misspellings) but without over-identifying the list of names in aclass. That is, a balance must be found between errors of exclusion and errors of inclusion. By a 'balance" is meant a minimalerror of exclusion while at the same time achieving a code thatwill produce the minimal amount of identification that will match
names in a given group of "close" name-forms. Such lists offamilies of close names are termed "equivalence classes", and themost familiar forns are found in the conventional telephone directory, e.g., for variant spellings of the name "SMITH".
b.

The algorithm is intended to function both 1) as an error
filter in on-line interrogation of a file, and 2) as a general
aid in search elaboration and request reformulation. This tool
will provide a capability for access to the file through a*See Appendix I.

- 63 -

mechanism not now feasible in the limited, passive cross-reference
structures in conventional card catalogs. Not only will the user
be able to come in through ro.Ates not presently available, but hewill be able to receive support from the machine in dynamicallycorrect_ng his spelling errors when he is attempting to probe thefile and converge on a particular datum, whether it be title ofbook, a name, or other information. On the other hand, he will
be able to expand his search in ways not presently available, e.g.,
to track down names or records when the similarity among them is
not explicitly recorded in conventional cataloging information.
Develo ment of a name com ression scheme. A spelling
equivalent abbreviation algorithm for personal names may be designed
to produce variable-length or fixed-length canonical forms, i.e.,class codes. Variable length coding has been extensively tested
but is generally rejected for use in operating systems due to theadded difficulty in programming in comparison to fixed length fields.
Also, code compression is enhanced by fixed length codes where nointerword storaae space is required.
c.

Since no definitive data was available yet on the problem ofdeciding exactly how long a fixed-length code should be, or on thenature of errors introduced by truncating variable-length classes
to a given fixed length, the approach taken was to create two versions of the algorithm for surname equivalence classes: both avariable-length algorithm and a fixed-length algorithm.
First a "hand-drawn" protr.)type variable-length algorithm waswritten, based on study of the set of classes given in a local
telephone directory. The rules for this algorithm are listed inFig. 16. The algorithms used in other compression schemes werethen synthesized and modified. The resulting algorithm was visuallytested on the equivalence classes in the phone book.

Results of variable length algorithm on telephone
directory. Visual analysis of the results of the variable-lengthalgorithm suggested that reasonably accurate matches could beachieved without excessive over-identification, that is, inclusionof widely variant names in a class that should have been excluded
from it.
d.

A sample of the telephone directory names and the variablelength class into which they were compressed by the first versionof the algorithm is shown in Fig, 174
The variable-length algorithm achieved a score of below 5%
under-identification error and over 77% preservation of distinct
identifications when evaluated against the original phone bookSpecifically, the phone directory contained 451 equivasystem.
lence classes. The initial version of the algorithm only split
22 (4.9%) of the 451 classes and preserved 349 (77.4%) out of the451.
- 64 -FIGURE 16:

A SPELLING EQUIVALENT ABBREVIATION ALGORITHM FOR PERSONAL NAMES

Dolby Version 1 - Variable Length

McG to Mk, Mag to Mk, Mac to Mk, Mc to Mk.

1.

Transform:

2.

Working from the right, recursively delete the second letter
dt, ld, nd, nt,
from each of the following letter pairs:
rc, rd, rt, sc, sk, st.

3.

ConsoTransform: x to ks, ce to se, ci to si, cy to sy.
nant -ch to consonant -sh. All other occurrences of c tok, z to s, wr to r, dg to g, qu to k, t to d, ph to f
(after the first letter).

4.

Delete all consonants other than 1, n, and r, which precede
the letter k (after the first letter).

5.

Delete one letter from any doubled consonant.

6.

Transform pf# to p#, #pf to #f, vowel -gh# to vowel -f#,consonant -gh# to consonant -g#, and delete all other(#.is the word - beginning and wordoccurrences of gh.
ending marker,)

7.

Replace the first vowel in the name by the symbol "*".

8.

Delete all remaining vowels.

9.

Delete all occurrences of w or h after the first letterin the word.

(NOTE:

vowels are defined as a, e, i, o, u, y.)FIGURE 17:

EQUIVALENCE CLASS COMPUTATION (MANUAL)
Version 1

A portion of a list of personal-name equivalence classes
from the Palo Alto-Los Altos Telephone Directory, arranged
according to the variable length compression code (with thevowel marker * treated as an A for ordering). (1)Variable-length"Dolby Code"
*BL
*BRMS
*BRMSD*D*DMNS*DMNSN
*DMS
*GN
*GR
*KN
*KNS
*KR
*KR
*KS
*LBRD
*LD
*LN
*LSN
*LVR
*MS
*NGL
*NL
*NRS
*NRSN
*NS
*RKSN

Names Belonging to Class
Abel, Abele, Abel?, Able
Abrahams, Abrams
Abrahamson, Abramson
Eddy, Eddie
Edmonds, Edmunds
Edmondson, Edmundson
Adams, Addems
Eagan, Egan, Eggen
Jaeger, Yaeger, Yeager (2)Aiken, Aikin, Aitken
Adkins, Akins
Acker, Aker
Eckard, Eckardt, Eckart, Eckert, Eckhardt
Oakes, Oaks, Ochs
Albright, Allbright
Elliot, Elliott
Allan, Allen, Allyn
Ohlsen, Olesen, Olsen, Olson, Olsson
Oliveira, Olivera, Olivero
Ames, Eames
Engel, Engle, Ingle
O'Neal, O'Neil, O'Neill
Andrews, Andrus
Andersen, Anderson, Andreasen
Ennis, Enos
Erichsen, Erickson, Ericson, Ericsson,
EriksenNotes: (1) A small number of directory entries that do notbear on the immediate problem have been deleted from the list:
Bell's see also Bells: Co-op see also Co-operative; Palo Alto
Clinic see also Palo Alto Medical Clinic; St. see also Saint; etc.
(2)
Names whose compressed codes do not match the one
given in the first column (and hence represent weaknesses in
the algo,Ath._ and/or the directory groupings) are underlined.

- 66 -

errors (i.e. , close
An analysis of the under-identification
included in a given class or classes
names that should have been
according to the phone book) is inthat should have been split,
the Dolby paper in Appendix I.

Results of testing op catalog data. A program wasby computer.
then written to implement the variable-length algorithm
in the Santa
This version was tested on a sample of 50,000 names selected names
Fig. 18 shows someCruz machine-form author file.
for families ofand the "canonical forms" which were computed
close names.
e.

second, fixed-length verIt was then decided to construct akinds of errors that mightsion of the algorithm to discern thevariable-length classes tobe introduced by truncation of therules for Version 2 issome standard length. The initial list of
yet been implemented,
presented in Fig. 19. This version has not
carrled out in the next phasebut computer testing of it will beversion incorporates modificaof the project. The fixed-length
the program for thetions suggested by analysis of results of"hand" analysis ofvariable-length classes together with separate
program the improved
the effect of truncation. We intend to comparatively
with otherfixed-length algorfthm and evaluate it
equivalence class schemes such as SOUNDEX.*

Effect of truncation to create fixed-len th classes.
algorithm revealed that
Initial analysis of the variable-lengthof under-identification
simple truncation vfll not generate errors
i.e., inclusion of
but will lead to further over-identification,The
disparate names in a class, that should have been excluded.
characters of the classes
simple truncation to the left-most seven
introduced no losses fromproduced from the phone book names,
However, reduccombining too many classes (over-identification).
jump in the cumulativetion to a code length of four causes alosseT(T.e., too many names being included
over-identification
be included due to wide variations
in a given class that should notof five characters for a straightin spelling). An optimal length
attainable, however,forwardly-truncated class code appears to be
Since it is desirable
in advance of any testing on a large file.
class, a method other than simple
to obtain the shortest possible
solution is further refinement
truncation was sought. A possible
with a final fixed length ofof the algorithm itself, ending up
proposed acts via selective removal of somefour. The procedure
vowel-marker deletion fromof the remaining characters, such as
vowel-markers'in thethe longer words and insertion of additional
indicates that an improvevery short words. Preliminary analysisof the version 2 algorithm
ment is achieved. Manual application
produced fixed-length compression codeson the phone book classesf.

*Becker, Joseph and Robert M Hayes. Information Storage andRetrieval: Tools, Elements, Theories. New York, Wiley, 1963.
pp. 143-144.

-67-

FIGURE 18:
EQUIVALENCE CLASS COMPUTATION (COMPUTER)

Version 1
Variable-length1121by Code"Names Belonging to Class (1)*BRN

O'Brian, O'Brien*NL

O'Neal, O'Neil, O'Neill*BD

Abbatt, Abbot, Abbott, Abetti, Ebbit,
Obieta*BR

Aubert, Auboyer, Aubrey, Aubry, Ibert,
Ybarra*BRM

Abraham, Ibrahim*BRMS

Abrahams, Abrams*BRMSN

Abrahamsen, Abramson*DRS

Edwardes, Edwards, Idriess*LPR

Allport, Alpers, Alpert*LS

Ellis, Alas, Eales, Eells, Elias, Ellis,
Else, Elst, Elwes, Olds, Olesha*LSN

Allison, Alston, Ellison, Elsen, Elson,
Elston, Ohlsen, Oleson, Olsen, Olson*MR

Amery, Amory, Aymar, Aymard, Emery, Immer*MRN

Amerine, Amrine, Emerson, Emerton, Emmerson*NG

Ewing, Ienaga, Inge, Iongh, Ong, Yanaga,
Yang, Youge, Young, Younge*NN

Anand, Annan, Anthony, Antin, Antoni,
Ennin, Onnen, Unwin, Yenawine, Yohannan
Airey, Ard, Arey, Arrow, Auer, Aury, Ayer,
Ayers, Ayre, Eayrs, Ewers, Ewert, Eyre,
Ihara, Irie, Iyer, Ore, Orr, Orrey, Orth

*R

*SBRN

Ashburn, Asbburne, Osborn, Osborne,
Osbourne

B*K

Bach, Back, Baikie, Bakke, Beach. Beachey,
Beck, Becke, Beke, Bewick, Biek, Boak,
Bocci., Bock, Bodky, Boeck, Boeke, Bok, Buck

T1) Selected from Santa Cruz machine author file.

-68-

FIGURE 19:
ABBREVIATION ALGORITHM FOR PERSONAL NAMES

Version 2 (This version incorporates refinements proposed as a result of applying Version 1 of the algorithm to about 50,000
author names selected from the UC Santa Cruz machine catalog
These rules should be regarded as interim, in thatfile.
further revisions may be suggested after additional analysis.)
For each sur-name:1.

Remove all blanks, hyphens, and apostrophes.

2.

Transform McG, Mag, Mac or Mc appearing at the beginning of a name, to Mk.

3.

Working from the right, recursively delete the second
letter from each of the following letter pairs:
dt, ld, nd, nt, re, rd, rt, sc, sk, st

4.

Transform: x to ks, ce to se, ci to si, cy to sy;
consonant-ch to consonant-sh, all other occurrencesof c to k, z to s, wr to r, dg to g, qu to k, t to d,ph to f.

5.

Delete al3 consonants other than 1, n, and r which
precede the letter k (after the first letter).

6.

Delete a final e if it occurs.

7.

Delete one letter from any doubled letter.

8.

9.

Transform pf# to p#, #pf to #f, vowel-gh# to vowel-f#,
consonant-gh# to consonant-g#, and delete all otheroccurrences of Oh (# is the word-beginning andword-ending marker).
Transform v to f.

10.

Replace each of the first two vowel strings by "*".
Here we consider a vowel to be any of the characters
A,E,I,O,U,Y.

11.

Delete all remaining vowels.

12.

Delete all occurrences of w or h after the first
letter.

13.

If the name is longer than 4 characters, drop one
final s.

14.

Truncate to six characters from right end.

15.

Reduce name to 4 characters by first removing therightmost * if length?. 5. If still not reduced to4, remove second * (can't be more than two *) and
If resultant length isthen trutcate if need be.
- 69 -FIGURE 19(Cont.):ABBREVIATION ALGORITHM FOR PERSONAL NAMES

Version 2 - FixtlIpamth

now less than or equal to 4 letters, retain one *,don't remove the *.

16, If the name is less than 4 characters, pad withblanks at the right, to get a uniform length of 4
characters total.

of length four, resulting in 361 distinct classes or 80% of the
Improper splits (failure
451 original classes in the directory.
of the algorithm to identify a name with its proper class) occurredin 24 or 5.3% of the classes.
Comparison with Present Manual Catalogs. It may still
be wondered why the equivalence class idea is advantageous. The
conventional card catalog provides a rather rigidly constructed"syndetic" apparatus to guide the user to his "target" - reference
structures that lead, for example, from forms of name requested
to the specific form of name used in recording particular file
items, and from specific known names to a list of pertinent
from "Dannay, Frederic see
related names (e.g., to a pseudonym:
entries under Queen, Ellery; Ross, Barnaby;" etc.)
6.

The subject cross referencing system is even more highly
constrained - the dictionary of synonyms and other link terms
is dispersed throughout the catalog; only the user who has accessto the "authority list" can even try to assemble systematicallyall the possible terms under which he might wish to search. Instances of related terms are often explicitly listed as "seealso's", but the distance separating any two similar terms or
names may be great, and thus printing out terms on either side of
the requested one is frequently unproductive. Some method must
be employed that overcomes the linear array of the file.
The manual file, e.g., the telephone book or card catalog,
usually makes limited, rudimentary efforts to help the user correct"See" and "see also" references are inserted for
his own errors.
variant spellirgs of names in the same and different languages.
These are usually permuted general class references, e.g.,
ACCESS 1:
SMITH

see also

SMYTH
SMYTHESCHMITHACCESS 3:
SMYTHEACCESS 2:
SMYTH

see also.

ACCESS 4:
SCHMITHsee also

SMITH
SMYTH
SCHMITHSMITH
SMYTHESCHMITHsee also
SMITH
SMYTH
SMYTHE

andCATHERINEFor sovereigns, princesses of sovereign houses, and saints:
Bohemian:
Dutch:English:

see

- 71

Katerina
Katherina
CatherineThis apparatus not only is cumbersome to use but complicatedto input and maintain. It also introduces a degree of redundancy
into the file which can be precluded by mechanisms such as equivalence classes.
Example of Use of Equivalence Class in On-line Mode. It
is useful to run through a hypothetical exercise to see how thename compacting algorithm might be of service in an on-line mode.
7.

The user desires to check a name in the file to see whether
the name exists, and if it does, what relation it bears to thefile content, e.g., as author of a book, as subject of a biography,
etc. He is assumed not to have a "verified" version of the spellingof the name in mind, i.e., no knowledge of the exact spelling of
the name he wants as it exists in the file and even if he did, itwould not necessarily correspond exactly to the particular form
in which the name and its associated elements are established asa catalog heading.
A necessary stage in the protocol uf fiaterrogating the file
is to decide what to do as the next step. Let us assume that thefirst action is to key in the full or partial surname. (In our
first experiments this will be the first n characters of the surname, depending on the kind of rules that are constructed from ananalysis of uniqueness of identification mentioned in Section II.
In later experiments, it would be interesting to test whether arelatively free-form mmemonic* could be employed, so that the user
could input n charac+.,rs according to general rules and have thecomputer perform the match with the equivalence claEs mechanism
rather than the user selecting an assigned code from a list.)

For example, if the surnane key is used and the user
thinks the name is GARNETT, A.C., he would key in "GARN". Assume
that a degree of match is found at this level of identification.
The computer could then display a sequence of messages describing
the file contents, in response: e.g.,a.

"There are 14 books by authors named Garn, Garner,
and Garnett, in the file you have queried.

INDICATE WHICH AUTHOR YOU WISH TO DISPLAY.GARNER.GARNGARNETT"

If the user requests a display sequence for "Garnett", the following message might be output:"Option 1:

There are 7 books by authors named Garnett in the
file.

*Jackson, Michael.
pp. 26-28.

"Mnemonics."

Datamation-72-

v. 13 (Apr. 1967),

Option 2:

There are 5 books by authors named Garnett with
A. for one of their initials in the file.

Option 3:

There are 3 titles by authors named Garnett, A.C.,
in the file.

Option 4:

There are 2 titles by author whose name is Garnett,
Arthur C., in the file.

Option 5:

There is 1 title by the author whose name isGarnett, Arthur Campbell, in the file.
Please select option you wish to explore further."

Assume the user goes on to select "Garnett, Arthur C." as therequested author. In this case he need only further select fromtwo titles.
By such a process of sequential query of the file contents,
the user is able gradually to converge on the name or document
he desires, or confirm that it does not exist, assuming he canmake these decisions from the file fmformation alone. Of course,
in large catalogs it may be necessary to employ gradually (in an
iterative fashion) more and more information about the name or
document(s) sought, such as titles of honor, birth/death dates,
publication date, and other distinguishing data. Such data would
only be displayed when needed, to permit the user to make distinctions among dense clusters of identical names which are identified
positively only by such subsidiary elements.*

The actual machine match at each stage would be made on the
portion of the name required to respond to the user's specification.
He may wish to expand or contract his notion of matching. Should
he decide to exercise the option which would lead to a display of
a run of very close names, for example, the action of the algorithm
controlling this would be based on density factors of the actual
file (number of identical names which are only distinguished byaffixed information, the number of titles per unique author name,
If there are hundreds of entries under a run of similaretc.).
surnames (e.g., Allen) the search control could step throughpresentations of various levels of summary information.
The other half of the author searching procedure isconcerned with expanding the field of search when no match isencountered. Let us assume that the surname match has failed andb.

*This procedure for search purposes is analogous to the "no conflict"policy employed in cataloging, where no identifying informationneed be affixed to name headings except where needed to distinguishbetween two different persons with identical names.
- 73 -

1,1

hence we are not interested in the initial spelling but rather
wish to interrogate the file for similar surnames. For example,
assune a search on the class of names beginning with "Aarons".
If the user did not know which spelling to guess at he could
reverse the process and key in the equivalence class code "*RNS"
from a list at the terminal, or if he did wish to guess at a
name, he could key either a full surname or an abbreviated key
for it, e.g., ARON. The response in the case of the full surname
might be "no match" but that there are other spellings. The
Our research so far has
dialog would then proceed as above.
served to point up the complexity of even the simplest cases.

-

IV.

A.

DATA BASE DEVELOPMENT

GENERAL

Recent Data Base Pro ects. There has been much recent
activity in the development of machine-form bibliographic files.
For example, the Library of Congress provided a limited amountof English language cataloging in machine-form to a group ofparticipating libraries through its MAchine Readable Catalog(MARC) Pilot Project. Harvard University has a closely related
project to provide non-LC bibliographic records in the MARC
format. In addition to these directly related projects, anumber of universities have begun developing machine data basesof bibliographic records, among which are the Santa Cruz campusof the University of California, Stanford, MIT, the Universityof Chicago, and the University of Toronto.
1.

In early 1967 the Institute of Library Research completed
an intensive investigation of book-form catalogs for the
California State Library.* During the study, a format convertfble to the LC MARC I format was developed and used in an analysisof the effort required to obtain a machine-form data base frombibliographic records. It vas fram this basis that we began ourinvestigations of data base development.
Current Effort. In developing the data base for thefacility, bibliographic records for both monographs and journal
articles are being incorporated in the system as the initial
forms of library materials to be represented by machine records.
Our first effort was devoted to establishing a set of procedures
to obtain the initial monograph data base for the study. Under
the funding granted, it was anticipated that a data base containing at least 200,000 titles in the Raman alphabet could be
To obtain a file of this size, it was recognizeddeveloped.
In addition, wethat some original input would be necessary.
decided to make an extensive study of the procedures by which
existing data bases could be integrated into our system alongwith our original input. F.Tom this study, we devised a method
for extracting the relevant information fram same of thesesources in order to reduce original input as much as possible.
The incorporation of these materials requires computer programs
which we are specifying and coding in order to translate theexisting machine data bases.
2.

Catalogs in Book Form: A*Cartwright, K.L. and R.M. Shoffner.
Research Study of their Implications for the California State
Library and the California Union Catalog, with a Design for their
Implementaion. Berkeley Institute of Library Research, Universityof California, 1967.

- 75 -

Where original input is to be accomplished in the development of the data base, the source of this material will be therecent cataloging output of the nine campuses of the University.
In order to minimize delay in the establishment of our data base,we started up the process for original input in parallel withthis programming. We took care to ensure a minimum likelihoodof redundant conversion of the various machine-form information.
It was known that the Harvard project is concentrating on materials of 1967 and subsequent publication date. The MARC records
were begun in mid-1966 and do not include retrospective materials.
As a result, virtually no materiels prior to 1966 publication
date are included in the MARC I file. The materials of the
Stanford University collection and the University of California
Santa Cruz collection are both primarily undergraduate collections.
The remaining materials available tend to be concentrated in
specific subject areas.
As a result, we anticipate that the majority of machinerecords for materials of publication date later than 1967 will
be fully covered by the combination of the Library of Congressand Harvard University projects. Therefore no original encodingof materials for this publication region has been undertaken.
With respect to sone of the materials of 1966 and later publication, it is our expectation that at some point, the records now
existing in LC MARC I files will be supplied to users in MARC II
format by the Library of Congress. In anticipation of having
both these upgraded MARC I records and current MARC II records
from LC, materials with a publication date prior to 1966 only,
are being used for original encoding in our project.
Overlap with the undergraduate collections previously mentioned can be avoided in two ways: first, by selecting material
which appears to be relevant to a research library collection but
not to a basic undergraduate collection; and second, by checking
questionable materials against the printed catalogs and lists ofthese collections which are available.
In our input procedure, the encoding of the bibliographic
data to identify the logical content of the record and the keying
to provide the record in machine form are performed as two separate processes. The encoding process requires training inlibrary cataloging in order to identify the logical content of
the bibliographic record. The keying process does not require
such training. Rather, the central issues in this process are
Thi3
the speed and accuracy with which it can be performed.
performance is enhanced given that virtually all decisions havebeen made about what is to be keyed.
By separating the keying operation and by making all information required by the camputer input program explicit in thedata stream, the keying can be performed on any device such as
keypunch, paper tape typewriter, on-line typewriter, or standard- 76 -

typewriter wlth optical scanning equipment. This means, forexample, that the computer program is organized to function
wdthout regard to the end of a tab card in a keypunch operation.
The separation of these processes is particularly usefulsince there exist commercial firms specializing in the keying
aspects of data conversion, to which this task can be contracted.
By separating this part from the other procedures which require
knowledge of bibliographic records, we can utilize these services
with most efficiency.

An important aspect of any production procedure is thecontrol that it provides for maintaining the appropriate costand quality of the product provided. We have been studying thetrade-off of cost and quality and we have established a statistical quality control procedure to maintain adequate accuracy inthe conversion process without excessive expenditure for thiscontrol.
B.

STRATEGIES OF CONVERSIONThis section is addressed to two closely related conversion
strategies which are often mentioned but which have not received
carefUl consideration. They are the use of existing machinedata bases and joint eff,rts jn which the conversion is shared
among several libraries. A critical aspect of both of thesestrategies is the amount of overlap that exists between the
catalog databases. In the material that follows we consider
this overlap as a variable and show the impact that it has on
the effectiveness of these strategies.
Utilization of an Existing Data Base. If a data basealready exists in machine form, it is possible to search it inorder to extract fram it matching records to be inserted into
gt
the new data base being assembled. There are three elements
which are important to the unit cost of the records obtainedby this approach: the cost of the search; the expected numberof records that will match and thus be usefUl; and the translation
costs to convert the record to the format of the new data base.
1.

The search cost is dependent upon the kind of arrangement
or degree of order of the two files - the old and the new, theIn general, thesize of the files, and the match method used.
least expensive method is to have the two files sorted to thenotsame order and then merge them. Although sorting time isrecords, only aa strictly linear function of the number ofsmall error is introduced by considering it linear. Used forthe purpose of a search, merging time is approximately a linear
function of the number of records in the larger of the two files
dharacterized as aby this approach. The search cost can besort cost per record times the number of records in the new file(assuming its initial order to be different from that of the new- 77 -

data base), plus a merge cost per record times the number ofrecords in the larger of the two files. The translation costdepends upon the degree to which the format and content of theexisting and the new records are the same. In any event, thetranslation cost is a linear function of the number of records
translated. That number can be characterized as a percent of thethe total number of records to be included in the new file.
We now develop an algebraic expression for the cost perconverted record. Lebthe number of records in the new database be represented as "Vnew". Assuming that it is the largerof the two data bases (true for our case) the search cost is:$search = ($sort + $merge) Vnew

(1)

where "$sort" is the sort cost per record and Imerge" is thesearch-by-merge cost per record (batch mode).
From this, we obtain as the next expression:$ conv= ($search + $trans=$searchi(Rhit'llnew)

Rhit'Vnew)/(Rhitslinew) (2)$trans= ($sort + $merge)/Rhit + $transwhere"$trans" is the translation cost per record;"R.
" is the ratio of the records V in the new file whichnit
are found in the existing data base; andconv" is the cost 221:record of obtaining the new database by searching, extracting, and translating records from theold data base.

If our purpose is to choose the approach which requires
the least cost to obtain the new data base, we may compare thisapproach to that of a straightforward conversion of the newdata base, which makes no use of the existing data base. As
before, we can consider the cost as a linear function of thesize of the new file and express the per record input cost as"$new".
We can now set up an inequality such that if theexpression is true, we could choose to use the existing database in the development of the new one:$new

N $ conv

Ammt($sort$merge)/Rhit- 78 -$trans(3)

From this, it is clear that in considering these alternatives
an immediate check can be made to confirm that the cost oftranslating an existing record is indeed less than that of theoriginal conversion of the desired record. Variations in format
or content could be great enough in themselves to reverse thedirection of the inequality. Assuming, however, that this is
not the case, we can proceed to rearrange the terms (since allterms are positive, the direction of the inequality will bepreserved):R$sort + $,merge )/($ new - $ trans

hit

)

(4)

This expression simply says that for search to be justifiable,
the ratio of hits must be greater than/equal to the ratio of
the search cost (sort-plus-merge) to the difference between thestraightforward conversion cost (non-translated file) and theconversion cost by the method of translation of an existingmachine file.

Another approach to the use of the existing data base is
to allay the files to remain in their original (perhaps unsorted)order and to make a randam search for each desired record. For
example, such a search could be performed over a terminal toan on-line file. In this case, our general analysis is the same
except that we have a search cost (on-line mode), expressed asa linear function of the number of records, which replaces thesorting and merging cost. This can be substituted in expression(4) as "$srch":
R

$

hit

/($

srch

_ $

new

)

tr

s

(5)

Thus, the analysis is applicable to either situation. Further,for on-line search to be less expensive than the sorting andmerging approach, the following must hold:$srch <.($ sort + $merge ).

(6)

Joint Conversion Efforts. As mentioned earlier, it isoften the case that a joint conversion effort is proposed as adesirable strategy. Given that all the records fram a group of
separate source files are to be converted, there are still manyquestions that must be answered in order to determine how to
go about the conversion. First, is there enough overlap between
the files to make it worthwhile to try to avoid multiple conversion of the "same" record? This is the question we wereaddressing in the previous section, in which we formulated an
equation which can be used to compute the minimum amount ofoverlap needed to justify a procedure to prevent duplicate
conversion. If the overlap is sufficient, which records should
be converted first? Can some files be converted independently?
79 2.

Can some files be converted in parallel, or is there a single
desirable conversion sequence?

Careful analysis shows that this is a less complicatedFirst, the total
series of questions than it first appears.
anount of direct conversion effort will be the same regardless
Therefore, the factorof the sequence in which it is performed.
which affects the total conversion cost is the search for tbose
records already ccnverted to machine form. Therefore, the proper
conversion strategy should minimize the total search cost, incurred during the course of ccnverting the whole "new" file,either by eliminating the search altogether, by reducing thenumber of searches, by reducing the search cost per record, orby a combination of these.
Some approaches which eliminate or keduce the number ofsearches will do so at the risk of a certain amount of unintended
duplication of records converted. We will first consider only
approaches that do not risk duplicate conversion.
Conversion with guaranteed control of du.lication.
The major source of search cost is search time, whether in a computer or a manual system. Let us use the following notation:a.

II= the number of records in file i;= the ratio of the total number ofrecords in file i, which are insome other file as well;R.

19j$isrch= the ratio of the total number ofrecords in file i, which are also
in file j;= the cost per record of searchinganother file for duplicates whileconverting file i.

The total search cost is thus:
$srch

±
i=2

V1 $ isrch.

(7)

Note that the summation index begins at 2 because there is noneed to perform a search when the first file is being converted,since no machine file exists yet.
The search cost Der record is dependent upon the file size,that is, upon the number of records al-7eady converted, over which
the search for duplicates takes place. Although there are manydifferent ways in which that search can be performed, we assume- 80 -

it will be one of a binary nature in which the search time is
proportional to the logarithm of the file size. Thus,
$

,

isrcn

-

V

= k log2 (

i-1

i-2
2:
j=1

j=1

V.)

R.
h=j+1 J'
E:

(8)

0

This formulation of search cost is useful primarily for random
search.

Determining the sequence in which a group of source files
are to be converted has two parts: 1) the general case of
selecting the next file to be converted; and 2) the special
case of selecting the first file.
The rule for the general case is straightforward. The file
to be converted next is the one which has the lowest ratio ofrecords which are not yet in the converted file, to total
records in this file. This rule balances out the executionof the maximum possible searches of the converted file at its
current size (as would be provided by selecting the largest
remaining file for conversion next), against the addition to
the file of the least number of records possible (as would be
provided by selecting the remaining file with the fewest records
not already converted).
A, batch oriented search is formulated in a manner similar
to (8), but it is linear:
S imerge

= g (

21'

j=1

i-2

1-1

j=1

h=j+1

V. - lE

2:

R

V.)

(9)

'3

where limerge" is the total cost for the search of all of therecords in the new file i.
The use of this search method changes the rule for selecting
the least cost conversion sequence. Because it is a batchprocess, the cost is not sensitive to the total records in thefile to be selected. Therefore, the appropriate rule is toselect for conversion next that remaining file which has the
fewest records that have not yet been converted (i.e., themost overlap with the other files). This has the effect of
keeping the growth in total size of the converted file atsmaller increments and thus the merges as amall as possible.
Now let us consider the special case of selection of thefirst file which will be converted without pre-sorting andwithout search since there is no data base already in machine
In the case of a random search it will be the largest
form.
file for which the overall duplication rate with the otherfiles, R., satisfies the expression defined for R.nit:- 81 -

Rhit$srch/($new(10)$modif)flA

qmodif" in the equation refers to either the cost of converting
records fram manual files or the cost of translating from anexisting data base, when search is used.

By this rule, the most searches will be avoided for a file which
would have been converted with search at some point in thesequence. This search avoidance reduces the cost more thanconverting the large file increases it.
For a batch processing procedure, the first file is selectedaccording to the general rule, i.e., the smallest file first.
The reasoning for this was given above.
Independent/parallel file conversions. To this point
we have discussed only the general characteristics of the files
to be converted. On this level of analysis it would not be
possible to define a method by which independent conversionof several files in parallel could take place. The key issue
in defining such a method is to be able to predict on the basisof some set.of readily observable characteristics of the record
to be converted, whether or not it is unique to this file.
If
this prediction can be made to a high enough level of accuracy,
then ipso facto it would not be worth it to perform searches
for these probable unique records. The duplication rate needed
to make search and modification (integration of duplicates withvariant elements such as call number) less expensive than directconversion without search was shown to be:
b.

Rhit

($sort$merge)/($new$modif)Recall that search is not economic if the hit ratio is less than
So long
the ratio of search cost/conversion cost differential.
as the error of estimate is not so great that the actual hit
rate satisfies this expression while the estimated hit rate does
not satisfy it, then these records having law expectation of
duplication can more economically be converted independently,
without search.
By this procedure, then, the records in all the files,
which are predicted to be unique could be separated and converted in any order whatsoever.
Parallel conversion could be accammodated by the strategy
of segmenting the files into sub-groups, such as letter groups
by main entry or title, and allocating different sub-groups
for conversion at a given time. This technique, however,
implies later search and reconciliation of the file segments.
- 82 -

As a result, it does not attempt to minimize the total conversion effort as do the approaches we have presented here.
An Example: The Catalog Suppleumnt. To develop thedata base for our facility by original conversion, we are using
the catalog records of the University's Catalog Supplement.
These are records of material which were new cataloging (notnecessarily new publications) to the campuses during the period
1963-67. We have made extensive studies of these records andare thus in a position to provide examples of the use of theexpressions defined in the earlier parts of this section. This
discussion, it should be noted, is intended only as an illustration, however, since many of the estimates are not verified.
Assume the parameters have the following values:
3.

$sort

= $0.05 per record$merge =

0.01

=

0.75

"

0.25

"

$

new

$modif$ srch=0.15

For a manual approach to the search, FLn it must satisfy thefollowing relation:

R(0.05 + 0,01)1(0.75 - 0.25),

(12)

0,12

(13)

hit

For an on-line search approach, it must satisfy(0.15)/(0.75 - 0.25)
Rhit

ie

(15)

0.30

- 83 -

Total estimated file sizes, by campus, are as follows:

Berkeley2149000
Davis
131,300
Irvine53,500Los Angeles
276,200
Riverside
99,500San Diego
128.,700
17,500San Francisco
126.9100
Santa Barbara
Santa Cruz*
Total - 1,046,80o
At this time we do not have estimates of the duplication rates
among these files. However, we believe that it can be reasonably
hypothesized that duplication rate and file size are negatively
corrrelated. That is, the larger the file, the less the duplication rate.

Using this hypothesis and the selection rules definedpreviously, we would first convert the largest of the files,
that fram Berkeley. The next file would be the one estimatedto have the greatest overlap among the nine. It would besearched against the machine-sorted Berkeley author file, after
which we would convert the unique titles and input added locations for the matches. The next highest overlap file would
follow, and so on down to the last remaining file with thegreatest residual uniqueness.
In this situation we would not take San Francisco as thefile with the highest duplication rate, even though it is thesmallest file. Because it represents a special collection(to support a medical school) our hypothesis of size and duplication rate is not likely to hold. Therefore, we would convert
it next to last.
All other campus files would be chosen inascending order of file size.
There are two characteristics which we believe will be useful for predicting duplication rate.
If so, they will bear on
the search/convert decision. They are the language of therecord and the indicated date of publication of the material.
Our hypothesis is that duplication rate is highest for English
records, next highest for records in other Raman languages,
and lowest for non-Roman languages. Also, the older the material,
the less the duplication rate. If the record was seen to havea particular language and a publication date that falls before
a certain date, the card would be converted without search of
the machine file, in the expectation that duplication would bevery low. Otherwise, it would be searched against the data base.

*The Santa Cruz file is now in machine-form and was not estimatedat this time.

-84-

.; I

In order to catch unintended duplication, a check could bemade automatically as part of the file maintenance process (ormanually, off-line) to compare possible matches to see if they
are truly duplicates. If found to be matches, post-conversion
modifying would then be done.

To check these hypotheses, we took a sample of approximately 7000 records and counted the duplication rate as a
The results arefunction of language and publication date.
given in Fig. 20. It should be noted that this is an overall
duplication rate: it is computed as a ratio to the resulting
file after duplicates are removed, rather than to the inclusive
source file with duplicates intact. The results for English
conform very well to our hypothesis. It would appear that 1962
would be a reasonable cut-off date for the use of on-line search,on an experimental basis.
No consistent pattern appears for publication date forthe non-English materials. Given the outcome of the work withEnglish, more study of these materials may be needed. At any
rate, it appears that for English language materials there isa high likelihood that if we used 1962 as the cut-off date
for searching, not to search items with a date prior to 1962
would give around a 30% chance of converting an item twice.
Whether this is acceptable will depend upon verification of
the various estimates which were used in the computation of
R

hit

C.

TRANSLATION OF EXISTING MACHINE FILES

General. In order to investigate the automatictranslation to our format of a file in another format fordata base development, we obtained copies of the magnetic
tapes of the catalog of the Library of the University ofCalifornia Santa Cruz campus.
1.

The reasons for the decision to use the Santa Cruz filerather than the LC MARC I file, arose from our analysis of
the UC catalog file.
First, the use of Library of Congress MARC I tapes for
extraction was likely to be useful for less than 10% of therecords in our source file. This is the case because the
MARC I file contains English language materials, the preponderance of which were published since mid-1966. At present,
probably about half of such material which has been published
has been included in the MARC I file. Therefore, we decided
to concentrate on the development of translation proceduresfor the Santa Cruz tapes since: 1) they contain approximately
90,000 catalog entries and 2) the Santa Cruz file should haveconsiderable duplication with at least two of the other campuses.
- 85 -FIGURE 20:
DISTRIBUTION OF DUPLICATE TITLESAS A FUNCTION OF PUBLICATION DATEA.

TITLES IN THE ENGLISH LANGUAGE

Year ofNo. of Publi- 1967 1966 1965 1964 1963 1962
Copies cation

1955 1950 Prior
to
to
to
no
1961 1954 1950 date

.12EL7eat.

.83

.53

.49

.20

.56

.67_

.82

.81

.86

% crf year

.17

.18

.19

.10

.14

.17

.12

.18

.10

at year

.10

.10

.04

.14

.10

.06

ar year

.10

.06

0

3

of year

.03

.

.

03

.

8

% of year

, 02

.

.03

.

4

ar year

.03

ion

.0 2

.0 2

.

.02

S-SZtagfir

Total %
% of year:

--

.01

% of year

100

100

100

100

99

99

100

99

100

100

This is the percent of titles (in the language indicated) in the sample that were published in the given
year, having the given number of copies.

- 86 -B.

TITLES IN LANGUAGES OTHER
THAN ENGLISH THAT USE A
RONAN ALPHABET

1955 1950 priorYear oftotoNo. of Publi- 1967 1966 1965 1964 1963 1962 to
1261 1254 1250
Copies cation
1
1

2% of yr

.83

.79

.781

.83

.79

.83

.79

.92

% of yr

.12

.16

.14

.11

.13

.15

.14

.07

no
date

100

,

% of yr

.03

.03

.03

.03

.03

% of yr

.02

.02

.03

.03

.05

.02

.005

.03

.06

.01

3

4
,

'0 of yr.01

5vc. of yr
,

6

70 of

8

.

v° of yr

/0 of yr9

Total %

C.

100

100

100

100

101

100

100

100

100

TITLES IN LANGUAGES THAT USE A NON-ROMAN ALPHABETINCLUDES TRANSLITERATED TITLES

1955 1950 priorYear ofnototoNo. of Publi- 1967 1966 1965 1964 1963 1962 to1961 1954 1950 dateCopies cation
100
100 100 100
.96
.95
.91
.93
.98
% of yr1,% of yr

.09

.07

.02

.05

.04

100

100

100

100

100

2

Total %- 87 -

100

100

100

100

The automatic translation of records is of considerable
importance, not only for the conversion of the Santa Cruz
materials as such, but because it represents a special solution
to the general problem of converting fram a less specificformat (such as the Library of Congress MARC I format or the
Stanford University Undergraduate Catalog format) to a more
specific and complex format (such as the Library of Congress
MARC II format). To the extent that automatic conversion is
successful, materials which have been converted in less specific formats will be more immediately useful in systems basedon the MARC II format. Also, the continuing process of input
in the MARC II format requires that many of the encoding decisions be made by professional cataloging personnel. A further
benefit of automatic translation could be to reduce the amountof trained personnel effort required for original input. This
would be acccmplished by reversing the production sequence.
The new process might be termed "camputer-assisted editing"(see Fig. 21). The keying of the catalog data is first performed, then the computer programs incorporpting the algorithms
assign the basic identifying codes to the records, and finally,
the trained personnel would review and correct or extend thecomputer actions.
The approach of the algorithms will be to identify thefields and sub-fields of the bibliographic record on a "bestguess" basis. By this approach, it seems reasonable toattempt to reduce the total manual effort to one-half that
required by present procedures.

- 88 -FIGURE 21:
CONVENTIONAL CONVERSION COMPARED TO AUTOMATIC FORMATTRANSLATION AND COMPUTER-ASSISTED EDITING(I A.F.T.

C.A.E.
START

SourceCatalog
Card

SourceCatalog/ManualPre-edit

MachineecordsKeyboard

J

Keyboard

Computer
Edit toProcessing
Format(Computer

ManualPost-edit/
correction

Edit

41110

Proofread

(

START

UpdateStop

9 -ComputerTranslation
to Input
FormatSanta Cruz File Translation Program. The generalfunction of the program (TRANSCOF) is to convert the codes fordata in one format to codes in another format. The change is
one of form, not content. Specifically, this program translates the Santa Cruz Library System input format into the ILR
input format.
2.

Some of the problems of format conversion are:Determination of the rules which categorize thedata into a given format, at the field level.
(1)

Identification of the "structural properties" ofthe original data (such as that of LC cards). Structuralproperties are the physical characteristics according to which
the data as strings of symbols may be said.(with some degreeof probability) to group itself according to logical content
Very little quantitative information on theand meaning.
nature of the field content of catalog records is available.
To obtain indicative data, a rough sample of 1000 LC cards
was analyzed: 997 had the characteristic that the author
surname (which is a logical content meaning) was uniquely
separated from the remainder of the name by a comma (a physicaldharacteristic). Thus, one trivial but usefUl structuralproperty of LC card data is that the surname is separated framthe given name of an author by a canna with a probability of
about 99%, yielding a reliable 'way of confirming sub-type of
personal name.
(2)

Structural properties with a high favorable prdbability
are being used in the development of algorithms which establish equivalence between the data elements of one format withthose of another. For example, using the structural property
illustrated above, we developed the following algorithm:
Given the code in the source record which identifies
the field containing main entry author name, we caninfer that the symbols preceding the first camma ina left-to-right character scan constitute thosecharacters which belong to the element 1surname', andthose subsequent to the comma belong to the category
See Fig. 22.
'given names'.
Where structural properties are too ill-structured to describe or have an unfavorable probability, indirect methods must
be developed for the division of a field in one format into
the several corresponding elements of another, more specificFig. 23 depicts the initial version of a sa-routine
format.
to identify the elements in the title field, a more difficulttranslation.
To illustrate further, consider the problem of subdividing the element 'subject headings' into sub-categories- 90 -FIGURE 22:
FLOW CHART OF PERSONAL AUTHOR FIELD ALGORITHM

concerning personal author as found

Given: All of the information
on an LC card, except information such as 'editor', translator',
etc., which is given by a code letter in col. 69 of the Santa Cruz

(1) Which portion of the string
input card. To be determined:
constitutes the name of the author, (2) which portion is the date
date(s), (3) whether or not the title 'Sir' (which is the most common title) is present.

The following is a generalized outline of the logic involved andof the subroutine
does not faithfully represent the actual coding
PERSAUTH.
Scan for theword 'Sir'
imbedded in
the string

(1)
START

Iheck col. 69 of
he orig. input

Is
'Sir'

ard for the
resence of aode which inicates tnat t eperson Is an editor,translator, etc.

set
SIRFLAG= 0

resentIf code ispresent, tackon appropriateabbr, with deIlmiter at
ena or,string.

Scan for 1st
numeral not
followed there-

after by aFirst said
numeral =start of datesmall alphabetic

Does
situation
occur?

Date field isassumed notpresent. Insert delimite
in front ofname field.

Yes

Insert delimi
ters in frontof name anddate fields

Tack on theword 'sir' withits delimiterat end ofstrin

if SIRFLAG =1

If SIRFLAG =1
insert 'sir'with its delimiter between
name and date

- 91 -EndFIGURE 23: FLOW CHART OF TITLE FIELD ALGORITHM
Complete monograph title field.
Given:To be determined: (1) Short Title, (2) long title, (3) author
statement and additional information, (4) edition statement, (5)remainder of edition statement.

START

left to rightscan forpunctuation/)11

Period,Questionmark,close parenthesis

If nonepresent,

Cgonly

--->(

Exit

is present.

Followed
by 2
lanks

s.kparatesQ) fromandnColon,

Yesg

present

sesi-colon

Periodsignals end)"..seParates

IIDfrom.0

of cp

No
furtherfieldsarepresent( Exit

Is

Yes

Is

rd char.
fter perioda blanit anumeral?

Indicates
thestart ofleft torightscan for"by"

If 'by'present

This is di,

has the begin-ning or con8matioil of

it

o end
of string

ates

from

- 92 -Are
next 3chars.
'Rev.'?

ExitFIGURE 23 .(Cont.);
FLOW CHART OF TITLE FIELD ALGORITHM

ii

left toright scan

Followed
by 2
lank

erio

Condition

N itherIf neither,'
nly
and
are
present

By
Yes

Periodsignals
end of

1111MOIIM,11111

left torightscan for
periodNo Period

Periodfollowedby 2
blanksYes

11111
If no period, lastsection
is

Exit

-93-

based on the various classe-3 of subjects, corresponding to the
6xx tag series in MARC II. There is nothing in the structureof information on the LC card which provides any clue to its
meaning or type, beyond the level of "subject tracings" identified by Arabic numerals. Resort might be made in this caseto table-lookup scheme utilizing a dictionary, which identifies
the field type to which a given subject heading belongs. A"tagged" version of the LC subject headings machine file could
be the basiz for such a method.

The TRANSCOF program consists of a main routine called
LAYOUT and five subroutines named PERSAUTH, CORPAUTH,
ES200500, TITLEALG and PPD.
The main program performs three functions, in three
sections:
Reads in the set of 'card images fram a source tapethat make up the data converted fram a single catalog card.
This data is then broken dawn and laid out into the appropriate
categories for processing by the subroutines.
Certain Characteristics of the data are noted and recorded for later use.
Branches are made to other sections of the main program upondetection of variations in the data (such as missing sections)for "troubleshooting" purposes.
(1)

The subroutines are called and various logical gatesare opened or closed in the third section.
(2)

The subroutines apply the algorithms to the appropriate
strings of data which vere constructed in che first part ofthe main program. Data which is input to the subroutines are
restructured into the ILR input format.
The various pieces of the restructured data areconcatenated to produce a continuous string which makes up an
ILR input record.
This is then to be written to tape, and thewhole process is repeated for another record.
(3)

D.

DATA BASE PRODUCTION PROCEDURE

In the following section we present a detailed plan of
the procedure through which we plan to obtain the data basefor the study.
The chart in Fig. 24 summarizes the overall file creation
plan, including the experimental on-line, terminal-based
search operation. Certain aspects of the process have notyet been implemented as of this writing. These components
are labelled "Proposed". Two programs in this plan are
critical and were discussed in preceding sections.
These are
- 94 -

SUMNARY CHART OF DATA BASE PRODUCTION

FIGURE 24:[Start]Translate
Format(Original
Format)

rans
lated
File

At

Iklalput'Format)(Update Index.0"

InitialMaster(P ofz.5),g Format)
A
I

GenerateIndex File
Name
Index

\

\

ataloig(Proposed
Terminal
Search on
Name

1

No
Match of Names
Match of Names

Print
MachineRecordDiagnostic
Listing1CompareCat. Data

No Match
Records

f

S.

Input
Edit
Program

Match
Trans.
File

Modifie
MasterFile(Proposed
UpdateC. recte /Modified
ile
rans.

[Stop]- 95 -

01.

.4ma

...1

M.

M1=1

. wo

M. ON IN

M OM

tit

the program for conversion of the Santa Cruz tapes to ILR input
format (TRANSCOF) and the input edit (processing format) program (INFOCAL). A preliminary version of the INFOCAL programis currently running.

Meanwhile, the modules of the system were set up in a"stand-alone" status. To T.,..plement the program for original
input separately, a large sample extracted from the source
file of University of California Catalog Supplement was used.
The material in this sample was edited, keypunched, and processed to shake down the record formats and to serve as abasis for evaluating alternative methods-of converting existingmachine files. That is, we plan to compare the records put
through the Santa Cruz format translation with any matchingrecords which are input as original records.
This will serve
as a basis for improvement and revision of the automatic format
translation program.
Proposed Oa-line Search Procedure. Instead of using
manual methods to verify which catalog entries have been
converted to machine form, we plan to use a terminal-based
search system for this purpose. The search key to be used inthis experiment would be the main entry author name (or aportion thereof) fram the card, matched against the machineindex file of author names. A research objective of thistechnique, as well as to shorten the search process, is togather quantitative data on the optimal length of search key,i.e., minimum number of characters in the name to be keyboardedin the request while preserving uniqueness (see Section ILEA).
1.

If there is a match, the catalog card would be set aside
to be checked against a subsequent printer listing of thefull machine record. If no match is obtained, the catalog
card would be put immediately into the conversion process.
At the same time, the index file would be updated with the
new information indicating that the entry is in the conversion
process but not yet in the system. As a separate manual
process the printed listing would later be checked Against
the catalog entries which were set aside. If a true matchis confirmed, the added location data or other modifying
information such as variant call number would be incorporated
into the existing record.
If the possible match is not true,
the catalog card would then be put into the normal conversion
process as a new record.

Fig. 25 shows the first steps in this experimental
verification search process.
Fig. 26 shows the cards found to be possible matches withexisting machine index file entries being compared with printouts for corresponding machine records fram the computer master- 96 -

ON-LINE SEARCH FOR DUPLICATES

FIGURE 25:

xisting
MachineFile

Existin
File(IIpi1f .
Format)

Input
Edit

ILR
Processing
FormatGenerate
Index

Name
Index
File

Catalog
Card

CompareIdentifier
on Card:Index File>

Update
Index

r_ _

1.0

_
Yes

Hold Match
Cards forCompare with
Machine Listi

Note
Recordumbers ofPossible Matches- 97

Send Nonmatch Cards
for OriginalInput

VERIFICATION OF MATCH

FIGURE 26:

2.0

Record Numberof Matching
MachineRecords
I.,..
.00
Retrieve
Records &rint Diagostic Lis ing\\

ICatalog
Card

F.

CompareCatalog Ca.

Listing1/(

Yesecide How
to Integra e.
Card &
Matching R cord

- 98 -//

If a verified match is confirmed, the editor decides how
to merge the manual record with the machine-readable information.
If no true match is found, the name heading on the catalog cardis entered into the machine index file as a control on the newrecord, which now goes into the original input process.
file.

The next chart (Fig. 27) shows
Editorial Preparation.
the process of editing and initial keyboarding. The cards
flowing into this stage are those which did not match, either
on the initial name (author) search, or which were found on
actual comparison with a printout from the machine file to be
distinctive records, even though the heading matched.
2.

If the records matched on name and also matched on title,
edition, etc. when the card was compared with the printout,
the variant data from the catalog card is input as a correction
to the machine record, as depicted in Fig. 28.
Several thousand records have now been edited in the ILR
Input Format. This wont is under evaluation in order toestablish the adequacy both of the formats and of the organization for providing the input.
Preliminary estimates of the effort requirements will be
used as a basis for evaluating the results of the first year's
editing and for reorganizing the input methods for the remaining portion of the current data base preparation work. Prior
experiments indicate that the input (iacluding all steps ofediting, keyboarding and correcting) of English language
material should be capable of being performed with anapproximate total of 5 man-minutes of effort per title.* Ourrecent work using the ILR input format that translates to full
MARC II format indicates that about 15-20% more effort per
title will be required, at least until the editing and typing
staff reach a high degree of proficiency. It is estimatedthat non-English material will require on the order of between
one-half again to two times as much effort as the English
language material.
Keyboarding. The records are next sent for keyDuring initial tests in ILR the production rateboarding.
achieved was somewhat lower than that attained in previous
studies. This is due to the impact on our coding of the3.

*Cartwright, Kelley L. and R.M. Shoffner. Catalogs in Book
Form: A Research Study of their Implications for the California
State Library and the California Union Catalog, with a Designfor their Im lementation. Berkeley, Institute of Library
Research, University of California, 1967. p. 48-49, 1-31.
- 99 -FIGURE 27:

DATA PREPARATION AND TRANSCRIPTION

Catalog Card....----4.

Prepare

Edit

Edited Coding
Sheet

,.
Keyboard

....

....,

%.

-100-

COMPUTER EDIT, CORRECTION CYCLE, AND FILE UPDATEFIGURE 28:\ Source
,

.

. S.

Input
Edit
Run

Transaction
File

Proposed Automatic Certification
Path

Path

V
MasterFile
UpdateProofread

Exit

iN

Certify

Errors?

Part 2of Listing.

I Indicate
Corrections

\
A

\

//

Keyboard
Corrections

Old
MasterFile
Update Run

UpdatedMaster
File

r
Index
Files
STOP

)

- 101 -.

"Carryover"Diagnostic-) Listings)added complexity of the MARC II format, primarily. Preliminary
estimates secured fram commercial service bureaus for the
initial keyboarding on a production basis are for a rate ofabout 5,000 characters per hour for the learning phase, increasing to a rate of 7,000 characters per hour after that, on
the average. This is based on the input record as describedin this report.

A number of keyboarding conventions are being recommended
for use with the lengthy, complex strings involved in catalog
records. Among these are a "null previous character" signal,i.e., a code that can be typed immediately following a charactertyped by mistake (when so recognized by the device operator),
and a "spacing rule". The device operator will not have to be
especially attentive to inter-word spacing when inputting thecatalog data since an internal computer edit routine will checkthe data stream for instances of more than one space and reduce such occurrences to one space.**There is one exception which is very important if the input
process happens to be implemented on tab cards.
That is toemploy a convention that a punched card can be either filled
completely (i.e., a character in the last column) or partiallyfilled. If partially filled, all trailing blanks (from thelast character punched to the last column available for use)
will be automatically purged from the input stream by the edit
program. In any case, the first normally used column of thenext succeeding card must contain a blank if there is an interword space needed. If not needed, the following card begins
with the next succeeding character of data in the first normally used column, as in the case of a break in a word at theright end of the card being punched.

Strict conformity to this or a similar arbitrary convention is absolutely required to make the input record usable.
Irregularity in spacing can create great difficulty in translation of a machine file fram one format to aaother, particularly in cases where tab cards were used as the conversion
medium.

- 102 -

Proofing and Correction. The records are then input tothe computer, and the internal processing record is compiled by(See Fig. 28). After a number ofthe input edit program.
machine edit checks, validations, etc., are performed, a diagnostic proof-listing is printed out. This printout here is
optional: it may be suppressed if the record is not to be
selected for proofing by the statistical quality control routine.
A line printer will be used rather than a slower, more costly
on-line terminal.
4.

The kind o2 validation checks performed are: check todetermine whether ten slashes have been inserted into the record;Checksignal an error condition if less than ten are present.
detailed aspects of the coding, such as whether author-analytic
added entries contain both an author and a title subfield, as
required by the input instructions. On the double-spaced
listing to be used by the proofreader, the computer prints
out error messages from a repertory of such diagnostic routines.
These messages appear in conjunction with the data containing
the detected error. Examples of printed error messages generated by the edit programs are:
DECKLET SEQUENCING ERROR. The card numbers in the set of
input record tab cards (if that is being used for input) are
not in ascending numeric order.

***cc IS INVALID ADDED ENTRY TYPE***. "cc" is a twocharacter code describing the type and sub-type of a heading.
The code designated was not recognized by the program as avalid code, indicating an editor error or typographical error
was committed.
MORE THAN 3 BIBLIO. LEVELS. EXTRA CODES IGNORED. Only
three positions are allocated in the record leader for storing
the values for bibliographic levels of the record.
TRANSLATION INDICATOR GIVEN, BUT 2ND LANG. MISSING. The
language variable field contained only one three-character
language code: there should be two.
SLASHES FOUND***.
***ONE OR MORE A-FIELDS MISSING.
This message is printed if the program finds more than or lessthan 10 slash marks between the location of the beginning of
the A-fields and the location of the start of the B-fields.
***IMPROPERLY CODED LOCATION SUB-FIELD IN HOLDINGS STATE The sub-field did not contain a five-digit***.
MENT***.
agency code or a five-digit-plus-one-letter sequence.

- 103 -

***WARNING - LC CALL NUMBER OCCURRED MORE THAN ONCE***.
This signals the proofreader that the field code for LC callnumber was used more than once. Could occur when proper codefor LC copy statement was nct used for the second call nuMber.

There are a number of error conditions which cannot be
determined by the computer, and these wauld of necessity haveto be detected in the visual proofing process. In case anerror goes into the master file undetected, a procedure willbe established to input corrections reported by users at alater time.
Two features of the correction cycle are of particularinterest: 1) the re-keyboarding of the corrections could be
performed experimentally via the on-line terminal, in contrast
to the initial keyboarding which is done off-line on devices
such as the keypunch, and 2) the diagnostic liisting of therecord is being experimentally printed out in two parts, one
containing the coding and data displayed in a logical fieldarrgy, element by element, field by field; the second part
listing tbe same data but in order corresponding to thesequence of the physical input record, e.g., a tab card decklet.
This form is an experiment intended to assist the proofreader. The first part of the listing is shown in Fig. 29,
and is used by the proofreader (who will not be the same person
as the editor of any given record) for visual scanning, comparison with the coding sheet, and detecting.of errors.
The second part of the listing (See Fig. 30) will be used
for re-keyboarding: the proofreader will transcribe keying
instructions on the card image listing itself, and the markedup listing will be transmitted to the correction typist, whowill input the corrections into the file. The operator scans
the listing for the indicated errors to be corrected, andre-keys only the amount of information necessary to correct
those errors. This information will consist, first, of theinformation which is in error, and, second, of the correct
information as it should be. Each line of the print-out
will have a number, which is assigned by the computer. Whenthe computer is given the incorrect information which occurs
in that line, it will search the corresponding internal line
to find that information, and then change it to the indicatedcorrect information. This technique avoids the necessity to
re-key either entire print lines or entire physical units
such as punched cards.
Such a "string correction routine" can be set up toprocess errors at the level of single character, words, lines,
groups of lines, or whole records. It can be operated throughany input device, either on-line or off ilne.

RECORD NUMBER

1001501966dmdsebfa690ja/%1(690r%jbBF435.G193cA:4430Amanaw00/111.A52 v.40no 4supp.

/Garrison, Mortimer,Med.

d/Cognitive models and development in mental retardation,

00/%edited by Mortimer Garrison, Jr.

H0

CD

0/New York,

11-1

a
H

/American Association on Mental Deficiency,tzi/1966.

tiH 0

H

CD 1-3/149 p.
1-1/(Monograph supplement to American journal of mental deficiency, v.70, no.4, 1966)*k"Proceedings of a Research Cpnference sponsored by the Woods Sskools, the.Americanissociation on Mental Deficiency and tfie National Institute or Child Health and Human.
Development"/O'"=, 0-3.m

FILE ORGANIZATION PROJECTDIAGNOSTIC LISTING
DATE
159
160

zj

05/02/68
100150100150RECORD NUMBER
01

02100150

8
2.

1966DEDSE8FA690JA/%%6900JB_B_F435._G193C90;430%MANAW00/_R1._A52 V.40110.4

o

SUPP./_GARRISUN, _MORTIMER,U4ED./_COGNITIVE MODELS IND DEVELOPMENT IN M

c-I ti

EIV.

@

1

161

100150

03

H

0
ON
i

162

10015004

H 8

ENTAL RETARDATION,/*EDITED BY _MORTIMER _GARRISON, _JR./_NEW _YORK,/_AME

iii.-31ti IIIRICAN _ASSOCIATION UN _MENTAL _DEFICIENCY,/1966./149 P.//(_MONOGRAPH SUP

HW

FAI.4

84

163

10015005

PLEMENT TO _AMERICAN JOURNAL UF MENTAL DEFICIENCY, V.70, NO.4, 1966)*10_

16410015006

PROCEEDINGS OF A _RESEARCH _CONFERENCE SPONSORED BY THE _WOODS _SCHOOLS,

1-3
n.)
i

165

10015007

16610015008

THE _AMEHICAN _ASSUCIATION ON _MENTAL _DEFICIENCY AND THE _NATIONAL _IN
STITUTE OF _CHILD _HEALTP AND _HUMAN _DEVELOPMENT"...

Following this correction of the initial product, it isnecessary to proofread the corrections made by the computer,in order to determine that these have been made correctly, andthat error has not been introduced in the second keying. Tn
the second proofreading, only the corrections indicated by theproofreader and the corresponding actions taken by the computer,will be campared to one another. It will usually not be necessary on the second proofreading to campare again the information printed by the computer, to the full information on theoriginal catalog entry.
The final step is to update the machine files with the
new record.
E.

ISSUES OF COST AND QUALITY

Data Conversion Quality Control. The data preparation
system is composed of four relatively independent processes:
editing, keyboarding, visual proofreading, and error correction(re-keyboarding).* Fig. 31 shows the flow of both "clean"
records and defective records through the quality control
subsystem. Distinction will be made anong three sources ofan error: editing, keyboarding, and computer processing (program errors, etc.). The detailed attribution of error is for
managerial purposes (e.g., revision of procedures and editor
training), and to prepare billing of a keyboarding contractor,
if the conversion were to be done by an outside service.
1.

Error records (dashed lines) originate mainly in theediting step and the initial keyboarding step. To detect andtally errors introduced by either of these sources, a control
element is introduced into the flow.

*This procedure does not include a control step called
H proofing after editing", wherein the editors exchange
coding sheets among each other prior to initial keyboarding.
This was thought to be an unwarranted extra cost in view ofthe high calibre of the staff available for editing. Also
excluded from the above is the review of trainee editors'
production.

-107-

ft,

QUALITY CONTROL SUBSYSTEM

FIGURE 31::

Error Record
Clean Record

MB

Indicate
Corrections

Re -Keyboard

Keyboard

4

if

Input
Edit

Run
V
END

"Gate"
for
SampleExtraction

Group 1ot Select
for Q.C.
Sample)Proofread(Autoimat1

Certif[..DeferCorrection
1.

Group 3"Clean"Records(Manually C rtified)

MasterFile
Update

-108-

L _

_>.

Group 2
Error
Records

Instead of making this a separate stage, it seemed mostfeasible to couple the control process with the proofreading
step, by modifying the latter. "Modified proofreading" would
include the normal processes of error detection and marking
for the purposes of re-keyboarding. It would also include
tallying of the errors detected and their attribution toediting, keying, or computer progran. An advantage to thisprocedure is a large reduction in the cost of the samplingneeded to establish and maintain the quality control procedure
itself.

The basic decision is that not all records need be
exhaustively proofread immediately after initial computer
input, as has normally been the case in library conversion.
Instead, the editing and keyboarding of all records, and themodified proofing of only some of the records (or, as a separate option, proofing of certain critical parts of eachrecord) is performed as one sequence. The exhaustive proofreading of the remainder of the records (or the remaining
parts of all records) and the correction of all of the errors
can be deferred and performed at a later time. (There will
be a procedure for correction of errors reported by fileusers, of course.)
The number of records to be proofread in the first cycleof the control procedure will be that sample size required
to obtain the desired confidence level of the information to
be fed back in regard to keyboarding and other errors. If theerror rate is high at the beginning of the production operation,
as is likely, then the number of records to be proofread will
be high since one of the variables governing the size of thesamples selected for inspection is the number of defective
records found in the initial sample. (A procedure for sample
size determination is included in Appendix VI.)
The output of the initial cycle would be three groupsof records. Group 1 is those records not selected for
proofreading on the initial cycle. These would be automatically transferred to the master file, and can be inspected
at a later date in a systematic manner for purposes ofcorrection. Group 2 is that set of records, selected for
proofing, which have been found to contain errors and areGroup 3 is that set of records,
now ready for re-keyboarding.
also selected for proofing, which have been found to be errorfree and are ready for transfer to the master file. This is
an iterative process, i.e., Group 2 records are continuously
re-cycled until they are removed from the process by falling
into Group 3.

-109 -A Decision Model for Keying Cost/Qualitz. In planning
for a large scale conversion it may be desirable to contract
out certain of the tasks which are one-time, non-continuing
in nature. The keying of the data is a portion of the effort
which might be done by an outside group. However, a methodfor deciding how to select among alternative conversion systems
and prices for data transcription service is needed. The
requirement is to secure the greatest amount of converted dataat the least expenditure both of time and money. Also, thequality of the conversion is vital. We suggest that a solution
is not found in simply choosing the method or firm which offers
a given level of quality for a fixed price (time and materials).
Rather the variables must be related to each other.
2.

In the circumstances of this project, one of the questions
asked is "What will be the cost impact on the record correction
part of our production, of a given level of accuracy of thedata obtained in the initial keyboarding?" This issue underlies several stages of the process, in particular the levelof effort for re-keyboarding. We have generalized our decision
making scheme in the hope that it might be usefUl to otherorganizations facing similar questions concerning conversion.
The effective conversion cost for a unit record of input
is that needed to attain an error-free record. This accuracy
can be defined by degrees of approximation to perfect data,that is, to 100% accuracy.
Given that the error rate of the initial keyboarding is
low enough, one may choose to accept the records wlthout
correcting them (at this time). In choosing between alternatives
with different costs and error rates one should select the one
with the lowest cost for the accurate records, where this cost
is computed by allocating all conversion cost to the accurate

records.IfIrecharacterizetheerrorrateasEthen
we can1
compute an effective cost per record, $AR, as
$AR =

$SB
1 - Ei/100(1)where $SB represents the initial keying cost per record.
If a higher level of accuracy is needed than can beprovided during initial keying, then subsequent proofing andcorrection will be required. The lucre accurate the initial
keyboarding is, the less the re-keyboarding cost incurred in
the correction module. Therefore, each increment of added
accuracy of initial keying has a value to the organizationin the replaced cost of re-keyboarding the corrections.

-110 -

When we include the cost of proofing and correction in the
total cost for converting data, the cost can be defined as:
K = $1H + $SB + $ED + $ECEi

(2)

where

K = total unit cost to input a record$1H = cost of editing and associated pre-keyboard
processes, per record$SB = cost per record for initial keyboarding*$ED = cost of visual error detection(proofreading)$EC = cost of re-keyboarding, per error"character-block"r.

E. = the number of error "character-blocks"
per record. A dharacter-block is definedas 5 contiguous dharacters which must be
re-keyboarded to correct one or morecharacters within that block.

To state the cost of error in relation to the initial
keyboarding costs, rearrange the equation:$SB = K - $1H - $ED - $ECEi(3)With this equation, we can define a family of keying cost/error rate indifference curves by allowing the total cost, K,
to take on different values. As an example, if $1H + $ED = $0.28
and $EC = $0.05 then the variation in given error rates (Ei)sustained, can be used to establish a curve for the relationof the variation in keying cost such that K is constant:$SB = K - $0.28 - $0.05Ei(4)

Fig. 32 shows an indifference curve for the range ofvalues cd' E. and $SB.

1,*Key verifying is excluded since it doubles the cost of theinitial transcription but does not result in detection oferrors attributable to editing and computer processing - oreven all the initial keying errors.

In Fig. 32, any price quotes for initial keyboarding thatfall in the region to the right of the indifference curve would
be unacceptable in comparison to any prices falling on the curveor to the left of it.
There may be some initial error rate, Ei, that is felt toto be too great to be acceptable even though the records could
be corrected. Thus, we show this "unacceptable region" extending
horizontally at an error rate of 10 blocks rer record. Fig. 32
was shown with the values of $SB plotted in terns of K. The
leftmost indifference curve identifies the best of a set ofdifferent price/error rate alternatives. To see this, consider
Fig. 33, which shows three separate price quotes for initial
keying, expressed as sliding scales based on error level. Partof the decision in this case is clear. Regardless of the specific accuracy obtained, "C" should not be chosen because its
price is always higher than those of A or B. Thus, it would
be irrational to pay price "C" under the conditions shown in
the example.
To determine which of the two prices, A or B, should bechosen, the indifference curve should be moved to the left as
far as possible and yet allow it to lie upon a point within
the acceptable region, quoted for A or B.

The leftmost curve will intersect Price "A" at an error
rate of 2. Thus, it is the preferable alternative.
In most situations a final error rate which is greaterthan zero is acceptable. This does not change the basic
analysis, however. Rather it defines an error rate below
which (plotted above on our figures) additional cost will
not be justified. The leftmost indifference curve whichintersects a price/error rate curve in the given error regionof interest will indicate the best alternative. In theillustration, a final error rate of 2 might be acceptable.
The price curve "A" would still be the one for the organization
to select.
Thus; using the level of accuracy desired in initial
keying, at a given keying cost as the critical vaxiable, wehave established a basis for selecting anong alternative costs
for keyboarding. This error rate is to be determined byvisual inspection of input records selected by the method ofsequential acceptance sampling outlined in the preceding
section.

- 112-FIGURE 32:

RELATION OF INITIAL KEYING COST TO ACCURACY

3

4

5

A7.7/
6

-6R7

,10

4/,

7

/

8
co

.61!/

r4

9

pt4

PI 1

REJECT RECORDABOVE 10 ERRORS

11

12

13

15

-.90

-.80

-.70

-.60

-.50

-.4o

-.30

-.20

PRICE FOR KEYBOARDING ($SB) AS FUNCTION OF
EDITING/PROOFING COSTS & ERROR RATE
K = total unit cost(constant)- 113 -

.-r

-.10

K

FIGURE 33:

ACCEPTABILITY IN TERMS OF ACCURACY AND COST FOR
THREE PRICE QUOTATIONS FOR KEYING

0

1

2

3

4

5

6

7

8

9

10

UNACCEPTABLEREGION

11

12

13

15

-

X

X

11

1

1

-.90

-.80

-.70

-.60

.10

.20

.30

..40

J

1

1

1

1

-.50

-.40

-.30

-.20

-.10

.50

.60

.70

.80

.90

$SB = PRICE FOR INITIAL KEYING(Avg./Record)= indiff. curve= price quote curves- 114 -K

REFERENCES

1.

Avram, Henriette D., John F. Knapp, and Lucia J. Rather. The
MARC II Format: A Communications Format for Bibliographic
Data. Washington, D.C. Information Systems Office, Libraryof Congress, 1968. 167 pp.

2.

Information Storage andBecker, Joseph and Robert M. Hayes.
New York, Wiley,
Retrieval: Tools, Elements, Theories.
448 pp.
1963.

3.

"Query Language for the Reactive Catalogue."Bregzis, Ritvars.
Tonik,
AlbertB., ed. Information Retrieval: theIn:
User's Vie oint - An Aid to Design. Philadelphia, International Information, Inc., 1967. pp. 77-91. (Fourth
Annual National Colloquium on Information Retrieval,
May 3-4, 1967.)

4 Cartwright, Kelley L. and R.M. Shoffner. Catalogs in Book
A Research Study of their Implications for theForm:
California State Library and the California Union Catalog,with a Design for their Implementation. Berkeley, Instituteof Library Research, University of California, 1967.

[

various pagings.
5.

"The Newcastle File Handling
Cox, N.S.M. and J.D. Dews.
System." In:
Cox, Nigel S.M. and M.W. Grose, eds. Organization and Handlin of Biblio ra hic Records b Com uter.
Hamden, Conn., Archon Books, 1967. pp. 1-21.

6.

Cunningham, Jay L. Instruction Manual for Editorial Preparation of Catalog Source Data. Preliminary Edition. Berkeley,
Institute of Library Research, University of California,
1968.
172 p.

7.

Jackson, Michael.
pp. 26-28.

8.

Mathematical Reviews.
(1965), p. 1207.

9.

Palmer, Foster M. "Conversion of Existing Records in Large
Libraries; with Special Reference to the Widener Library
Shelflist." In: Harrison, John and Peter Laslett, eds.
The Brasenose Conference on the Automation of Libraries.
Held at Oxford, Eng., 30 June - 3 July 1966. London,
Mansell, 1967. pp. 57-83.

"Mnemonics."

Datamation, v. 13 (Apr. 1967),(Am. Math. Soc., Lancaster, Pa.)

-115 -

v. 30

REFERENCES (Cont.)

10.

U.

Payne, Charles T. "Tagging Codes." Chicago, University of(unpublished report) various
Chicago Library, Feb. 1967.
pagings.

"Effectiveness of Book-Memory Data for
Vaughan, Delores K.
Chicago. University.
Conventional Catalog Retrieval." In:
Graduate Library School. Requirements Study for Future
Catalogs; Progress Report No. 2. Chicago, Mar. 1968.
(NSF Grant GN 432) p. 53.

-116-

APPENDIX I

AN ALGOPTTHM FOR NOISY MATCHES IN CATALOG SEARCHING

By

James L. Dolby
R & D Consultants Company
Los Altos, California

AN ALGORITHM FOR NOISY MATCHES IN CATALOG SEARCHING*

By
James L. Dolby
R & D Consultants Company
Los Altos, California

A.

INTRODUCTION

A viable on-line search system cannot reasonably assume
that each user will invariably provide the proper input information without error. Human beings not only make errors, but
mechthey also expect their correspondents, be they human or
anistic, to be able to cope with these errors, at least at somereasonable error-rate level. Many of the difficulties in
implementing computer systems in many areas of human activity
stem from failure to recognize, and plan for, routine acceptanceof errors in the systems. Indeed, computing did not become thewidespread activity it is now until the so-called higher-level
Although it is customary to thinklangilages came into being.
of hijher-level languages as being "Imre English-like," theheight of their level is better measured by the brevity withwhich various jobs can be expressed (for brevity tends to reduce errors) and the degree of sophistication of their automatic
error detection and correction procedures.
The processing of catalog information for the purposes of
exposing and retrieving information presents at least two major
areas for research in automatic error detection and correction.
At the first stage, the data bank must be created, updated andmaintained. Methods for dealing with input errors at thislevel have been derived by a number of groups and it seems
reasonable to assert that something in the order of 60 per centof the input errors can be detected automatically (1,2,3).
With the possibility of human proofreading and error detection
through actual use, it is reasonable to expect a mature database to have a very low over-all error rate.
At the second stage, however, when a user approaches thedata base through a terminal or other on-line device, the errors
his own
will be of a recurring nature: each user will generate
error set and though experience will tend to minimize the error
rate for a particular user, there will be an essentially irreAnd
ducible minimum error rate even for an experienced user.
if the system is to attract users other than professional interrogators, it must respond intelligently at this minimal error
level.

*This research was carried out for the Institute of Library
Research, University of California, under the sponsorship of
the Office of Education, Research Grant No. OEG-1-7-071083-5068.
,y47/-119-

In this paper we consider certain problems associated withmaking "noisy matches" in catalog searches. Because preliminary information indicates that the most likely source of input
errors is in the keyboarding of proper names, the main emphasisof the paper will be on the problem of algorithmically campressing proper names in such a way as to identify similarnames (and likely misspellings) without over-identifying thelist of possible authors.
B.

THE STRUCTURE OF EXISTING NAME-COMPRESSION ALGORITHMS

The problem of providing equivalence classes of proper
Library catalogs, telephone directories
names is hardly new.
and other major data bases have made use of "see-also"-typereferences for many years. Same years ago Remington-Rand derived an alphanumeric name.compression algorithm, SOUNDEX, that
could be applied either by hand or by machine for such purposes(4).
Perhaps the most widely used on-line retrieval systempresently in existence, the airline reservation system (suchas SABRE), makes use of such an algorithm (5). The closely
related problem of compressing English words (either to establish noisy matches, eliminate misspelled words, or simply toachieve iata bank compression) has also received some attention(see, for example, (6) and (7) and (8)).

Although the English word structure differs from propername structure in some important respects (e.g., the existenceof suffixes), three of the algorithms are constructed by giving
varying degrees of attention to the following five areas ofword structure:
1.
2.

3.
4.

5.

The character in word initial position
The character set:
(A,E,I,O,U,Y,H,W)
Doubled characters (e.g., tt)
Transformation of consonants (i.e., all alphabetic
characters other than those in 2 above) into equivalence classes.
Truncation of the residual character string.

The word-initial character receives varying attention. SOUNDEX
places the initial consonant in the initialposition of thecompressed form and then transforms all other consonants into
equivalence classes with numeric titles. SABRE maintains theword-initial character even if it is a vowel. In the Armour
Research Foundation scheme (ARF), the word-initial Character isalso retained as is.

Both SOUNDEX and SABRE eliminate all characters in the set
2 above. The ARF scheme retains all characters in shorter words
and deletes vowels only, to reduce the compressed form to fourcharacters, deleting the U after Q, the second vowel in a vowel
string, and then all remaining vowels.
-120-

10.

%NO

All three systems delete the second letter of a doubleletter string. SABRE goes 4. step further and deletes the second
letter of a double-letter string occurring after the vowels havebeen deleted. Thus, the second R of BEARER would be deleted.
SOUNDEX maps the 18 consonants into 6 equivalence classes
as follows:
1.

2.
3.
4,
5.

6.

B,F,P,V
C,G,J,K,Q,S,X,Z
D,T
L
M,N
R

SABRE and ARF do not perform any transformations on these 18
consonants.

Fina4y, all three systems truncate the remaining stringof characters to four characters. For shorter forms, paddingin the form of zeros (SOUNDEX), blanks (SABRE), or hyphens (ARF)
is added so that all codes are precisely four characters long.
Variable-length coding schemes have been considered but
generally rejected for implementation on major systems becauseof the attendant difficulties of programming and the fact thatcode compression is enhanced by fixed-length codes where nointerword space is necessary. Although fixed-length schemes oflength greater than four have been considered, no definitive
data appears to be available as to the ability to discriminate
by introduction of more characters in the compressed code. The
SABRE system does add a fifth character but makes use of theperson's first initial for added discriminatiori.
Tukey (9) has constructed a personal author code for his
citation indexing and permuted title studies on an extensive
corpus of the statistical literature. In this situation theauthor code is a semi-mnemonic code in a tag form to assist theuser in identification rather than as a basic entry point.
However, Tukey does note that in his corpus a three-charactercode of the surname plus two initials is superior to a fivecharacter surname code for puTposes of unique identification.
C.

MEASURING ALGORITHMIC PERFORMANCEOne of the main problems in constructing linguistic
algorithms is to decide on appropriate measures of performanceand to obtain data bases for implementing such measures. In
this case it is clear that certain improvements in existingalgorithms can be made - particularly by using more sophisticatedtransformation rules for the consonants - and that the problemsof implementing such changes are not so great in today's context-121-

as they were when the systems noted above were originally dee.ved. Improvements in processing speeds and programming languages, however, do not remove the need for keeping "linguistic
frills" to e, minimum.

Ideally, it would be desirable to have a list of common
errors in keyboarding names as a test basis for any proposed
algcrithms. Unfortunately, no such list of sufficient size
appears to be available. azcking this, one can speculate thatcertain formal properties of the predictability of language
might be useful in deriving an algorithm. At the English word
lecel, some effort has been made to exploit measures of entropy
However, there
as developed by Shannon in this direction (6,7)
is good reason to question whether entropy, at leas:1 when measured in the usual way, is strongly correlated with actuallyoccurring errors (10).

As an alternative, one can study existing lists of personalname equivalence classes to derive such algorithms and then test
the algorithm against such classes, measuring both the degree ofover-identification and the degree of under-identification.
Clearly, such tests will carry more weight if they are carriedout under economic forcing conditions where weaknesses in thetest set will lead to real and measurable expense to the organization publishing the list. The SABRE system operates understrong economic forcing conditions in the sense that airline
passengers frequently have a number of competitive alternatives
available to them and lost reservations can cause them sufficientinconvenience for them to consider these alternatives. However,
the main application of the SABRE system is to rather small
groups of persons (at least when compared to the number of
personal authors in a typical library catalog) so that errorsof over-identification are essentially teivial in cost to theairlines.

A readily available source of see-also-type equivalenceclasses of proper names is given in the telephone directory
Here, the econamic forcing system is not so strong as
system.
in the airline situation, but it is measurable in that failure
to provide an adequate list will lead to increased user dependence on the Information Operator - with consequent increased
cost to the telephone company. As a test on the feasibility ofusing such a set of equivalence classes, the 451 classes foundin the Palo Alto-Los Altos (California) telephone directory were
copied out by hand and used in deriving and testing the algorithm
given in the next section and the SOUNDEX algorithm.
There remains the question of deciding what is to constitute proper agreement between any algorithm and the set of equiv.alence classes chosen as a data base. At tbe grossest level itseems reasonable to argue that over-identification is less serious
than under-identification. False drops only tend to clog the line.
-122-

Lost reference points, on the other hand, lead to loLt information. Investigation of other applications of linguistic
algorithms, such as algorithms to hyphenate words, identify
semantically similar words through cutting off of suffixes, andso forth, indicates that it is usually possible to reduce crucial
error (in this case under-identification) to something underfive percent, while preserving something in the order of 80 percent of the original distinctions (or efficiency) of the system.
Efforts to improve materially on the "five-and-eIghty" rule
generally lead to solutions involving larger context and/or
extensive exception dictionaries. In this study we shall aim
our efforts at achieving a "five-and-eighty" solution.
D.

A VARIABLE-LENGTH NAME-COMPRESSION SCHEME

In light of the fact that no definitive information isavailable on the problems of truncating errors in name-compression algorithms, it is convenient to break the prdblem into twopieces. First, we shall derive a variable-length algorithm oftile required accuracy and efficiency and then we shall determine
the errors induced by truncation.

After studying the set of equivalence classes given in the
Palo Alto-Los Altos telephone directory, it was fairly clearthat with minor modifications of the basic five steps used in
the other algorithms noted above, it would not be too difficult
to provide a reasondbly accurate match without requiring too
much over-identification. The main modifications made consistedof maintaining tl'e position of the first vowel and using local
context to make transformations on the consonants. The algorithm
is given below.
11112511am_Equiva1ent Abbreviation Al orithm For Personal Names
1.
2.

3.

4.
5.

6.

7.
8.

9.

Transform: McG to Mk, Mag to Mk, Mac to Mk, Mt to Mk.
Working from the right, recursively delete the second
letter from each of the following letter pairs: dt,
ld, nd, nt, rc, rd, rt, sc, sk, st.
Transform: x to ks, ce to se, ci to si, cy to sy.
Consonant-ch to consonant-sh. All other occurrences ofc to k, z to s, wr to r, dg to g, qu to k, t to d, ph
to f (after the first letter).
Delete all consonants other than 1, n, and r which
precede the letter k (after the first letter).
Delete one letter from any doubled consonant.
Transform pf# to p#, #pf to #f, vowel-gh# to vowel-f#,
consonant-gh# to consonant-g#, and delete all other(# is the word-beginning and wordoccurrences of gh.
ending marker.)
Replace the first vowel in the name by the symbol "*".
Delete all remaining vowels.
Delete all occurrences of w or h after the first letterin the word.
-123-

1111..11111

The vowels are taken to be (A,E,I,O,U,Y).
The algorithm splits 22 (4.9 percent) of the 451 equivalenceclasses given by the phone directory. On the other hand, thealgorithm provides 349 distinct classes (not counting thoseclasses that were broken off in error) or 77.4 percent of the451 classes in the telephone directory data base. Thus, we haveachieved a reasonable approximation to the "five-and-eighty"
performance found in other linguistic proble . areas.

To give a proper appreciation of the nature of these underidentification errors, they are discussed below individually.
1.

The name Bryer is put in the same equivalence classwith a variety of spellings of the name Bear. The
algorithm fails to make this identification.
2.
Blagburn is not equated to Blackburn.
3.
The name Davison is equated to Davidson in its various
forms. The algorithm fails to make this identificationand this appears to be one of a modest class of difficulties that occur prior to che -son, -sen names.
4.
The class of names Dickinson, Dickerson, Dickison, andDickenson are all equated by the directory but kept
separate except for the two forms of Dickinson by thealgorithm.
5.
The name Holm is not equated with the name Home.
6.
The name Holmes is not equated with the name Homes.
7.
The algorithm fails to equate Jaeger with the various
forms of Yaeger.
8.
The algorithm fails to equate Lamb with Lamn.
9.
The algorithm incorrectly assumes that the final gh of
Leigh should be treated as an f. Treating final gheither as a null sound or an f leads to about the same
number of errors in either direction.
10.
The algorithm fails on the pairing Leicester and Lester.
The difficulty is an intervening vowel.
11.
The algorithm fails to equate the various forms ofLindsay with the forms of Lindsley.
12. The algorithm fails to equate the various forms of
McLaughlin with McLachlan.
13.
The algorithm fails to equate McCullogh with McCullah.
This is again the final gh problem.
14.
The algorithm fails to equate McCue with McHugh (again
the final gh problem).
15.
The algorithm fails to equate Moretton with Morton.
This is an intervening vowel problem.
16.
The algorithm fails to equate Rauch with Raush.
17.
The algorithm fails to equate Robinson with Robison(another -son type prdblem).
18. The algorithm incorrectly assumes that the interior phof Shepherd is an f.
19.
The algorithm fails to equate Speer with Speier.
-124-

20.
21.
22.

The algorithm fails to equate Stevens with Stephens.
Similarly for Stevenson and Stephenson.
The algorithm fails to equate the various forms of theword Thompson (an -son problem).

Several of the errors noted above are questionable, al
least in the sense of questioning whether the telephone directoryis following its own procedures with complete rigor. Settingthese acide, the primary errors occur with the final gh, thewords ending in -son, and the words with the extraneous interior
vowels. Each of these problems can be re2olved to any desireddegree of accuracy, but only at the expense of noticeable increases in the degree of complexity of the algorithm.
E.

THE TRUNCATION PROBLEM

Simple truncation does not introduce errors of underidentification; it can only lead to further over-identification.
Examination of the results of applying the algorithm to thetelephone directory data base shows that no new over-identification
is introduced if the compressed codes are all reduced to theleft most seven characters. Further truncation leads to thefollowing short table:
Code Length

Cumulative Over-Identification Losses

7
6

0

5

6
45

1

4

Thus there is a strong argument for maintaining at least five
Characters in the compressed code.
However, there is no real need for restricting ourselves
to simple truncation. Following the procedures used in the
ARF system, we can obtain further truncation by selectively
removing some of the remaining characters. The natural candidate for such removal is the vowel marker. If the vowel marker
is removed from all the five character codes, only six more
over-identification errors are introduced. Removal of the vowel
markers from all of the codes would have introduced 17 more errorsof over-identification. Thus we see that the utility of thevowel marker is in the short codes.
This in turn suggests that introduction of a second vowel
marker in the very short codes may have some utility. This isindeed the case. If we generalize the notion of the vowel marker
to that of marking the position of a vowel-string (i.e., a stringof consecvtive vowels), where for these purposes a vowel is anyof the characters (A,E,I,O,U,Y,H,W), and maintain these markers-125-

as "padding" in the very short words, 18 errors of over-identification are eliminated at the cost of two new errors of underidentification. In this way we derive the following modificationco the variable length algorithm:
1.
2.
3.

4.
5.

Mark the position of each of the first two vowel strings
with an "*", if there is more than one vowel.
Truncate to six characters.
If the six-character code has two vowel markers,,remove
the right hand vowel marker. Otherwise, truncate thesixth character.
If the resulting five-character code has a vowel marker,
remove it. Otherwise remove the fifth character.
For all codes having less than four dharacters in thevariable-length form, pad to four characters by adding
blanks to the right.

Measured against the telephone directory data base, this fixed
length compression code provides 361 distinct classes (notcounting improper class splits as separate classes) or 80 percent of the 451 given classes. Twenty-four (5.3 percent) of theclasses are improperly split.
By way of comparison, the SOUNDEX
system imploperly splits 135 classes (30 percent) and provides
only 287 distinct classes (not counting improperly split classes)
or 63.8 percent cf the telephone directory data base.
F.

ACKNOWLEDGEMENT

The author would like to thank Ralph M. Shoffner and
Kelley L. Cartwright for suggesting the problem and for anudber of usefUl comments on existing systems. Allan J. Humphrey
was kind enough to program the variable-length version of thealgorithm for
purposes.

1

-126-

G.

CORPUS OF NANES USED FOR ALGORITHM TEST

List of personal-name equivalence classes from the Palo AltoLos Altos Telephone Directory arranged according to the variable length compression code (with the vowel marker * treatedas an A for ordering).

*BL
*BRMS
*BRION*D*DMNS*DMNSN
*DMS
*GN
*GR
*KN
*KNS
*KR
*KR
*K2
*LBRD
*LD
*LN
*LSN
*LVR
*MS
*NGL

*NL
*NRS
*NRSN
*NS
*RKSN

*RL
*RN
*RNS
*118

*RVN
*RVNG
*SBRN

Abel, Abele, Abell, Able
Abrahams, Abrams
Abrahamson, Abramson
Eddy, Eddie
Edmonds, Edmunds
Edmondson, Edmundson
Adams, Addems
Eagan, Egan, Eggen
Jaeger..., Yaeger, Yeager

Aiken, Aikin, Aitken
Adkins, Akins
Acker, Aker
Eckard, Eckardt, Eckart, Eckert, Eckhardt
Oakes, Oaks, Ochs
Albright, Allbright
Elliot, Elliott
Allan, Allen, Allyn
Ohlsen, Olesen, Olsen, Olson, Olsson
Oliveira, Olivera, Olivero
Ames, EameS
Engel, Engle, Ingle
O'Neal, O'Neil, O'Neill
Andrews, Andrus
Andersen, Anderson, Andreasen
Ennis,-Eaos
Erichsen, Erickson, Ericson, Ericsson, Eriksen
Earley, Early
Erwin, Irwin
Aarons, Ahrends, Ahrens, Arens, Arentz, Arons
Ayers, Ayres
Ervin, Ervine, Irvin, Irvine
Erving, Irving
Osborn, Osborne, Osbourne, Osburn

Note:

Names whose compressed codes do not match'the one given'in the first column (and hence represent weaknesses in
the algorithm and/or the directory groupings) are underlined.

Note:

A small number of directory entries that do not bear on
the immediate problem have been deleted from the list:
Bell's see also Bells; Co-op see also Co-operative;
Palo Alto Clinic see also Palo Alto Medical Clinic; St.
see also Saint;

etc.

-127-

B*D
B*DS
B*KMN
B*L
B*L
B*L
B*L
B*L
B*LN
B*M
B*MN
B*N
B*ND
B*R
B*R
B*R
B*R
B*RBR
B*RG
B*RGR
B*RK
B*RN
B*RNR
B*RNS
B*RNSN
B*RS
BL*KBRN
BL*M
BR*D
BR*N
BR*IT

D*DS
D*F
D*GN

D*K
D*KNSN
D*KSN
D*L
D*L
D*L
D*MN
D*N
D*N
D*N

D*N
D*N
D*NL
D*R
D*R
D*RM
D*VDSN

Beatie, Beattie, Beatty, Beaty, Beedie
Betts, Betz
Bachman, Bachmann, Backman
Bailey, Baillie, Bailly, Baily, Bayley
Beal, Beale, Beall, Biehl
Belew, Ballou, Bellew
Buhl, Buell
Belle, Bell
Bolton, Boulton
Batun, Bohm, Bohme

Bauman, Bownan
Bain, Bane, Bayne
Bennet, Bennett
Baer, Bahr, Baier, Bair, Bare, Bear, Beare, Behr, Beier,Bier, Bryer
Barry, Beare, Beery, Berry
Bauer, Baur, Bower
Bird, Burd, Byrd
Barbour, BarberBerg, Bergh, Burge
Berger, Burger
Boerke, Birk, Bourke, Burk, Burke
Burn, Byrne
Bernard, Bernhard, Bernhardt, Bernhart
Berns, Birns, Burns, Byrns, Byrnes
Bernstein, Bornstein
Bertsch, Birch, Burch
Blackburn, BlagburnBlom, Bloom, Bluhm, Blum, Blume
Brode, Brodie, Brody
Braun, Brown, Browne
Brand, Brandt, Brant
Dietz, Ditz
Duffie, Duffy
Dougan, Dugan, Duggan
Dickey, Dicke
Dickenson, Dickerson, Dickinson, Dickison
Dickson, Dixon, Dixson
Dailey, Daily, Da:ley, Daly
Dahl, Dahle, Dall, Doll
Deahl, Deal, Diehl
Diamond, Dimond, Dymond
Dean, Deane, Deen
Denney, Denny
Donahoo, Donahue, Donoho, Donohoe, Donohoo, Donohue,
Dunnahoo
Downey, Downie
Dunn, Dunne
Donley, Donnelley, Donnelly
Daugherty, Doherty, Dougherty
Dyar, Dyer
Derham, DurhamDavidsen, Davidson, Davison
-128-

D*VS
DR*SL
F*
F*FR
F*GN
F*L
F*L
F*LKNR
F*LPS
F*NGN
F*NL
F*RL
F*RR
F*RR
F*RS
F*RS
F*RS
F*SR
FL*N
FL*NGNFR*FR*DMN
FR*DRKSN
FR*K
FR*NS
FR*NS
FR*S
FR*SR
G*D
G*DS
G*F
G*L
G*LMR
G*LR
G*MS
G*NR
G*NSLSG*NSLVS
G*RD
G*RD
G*RN
G*HNR
G*RR
G*S
GR*
GR*FD
GR*N
GR*S
H*D
H*F
H*FMN
H*G

Davies, Davis
Driscoll, Driskell
Fay, Fahay, Fahey
Fifer, Pfeffer, Pfeiffer
Fagon, Feigan, Fegan
Feil, Pfeil
Feld, Feldt, Felt

Fu1kner, Falconer
Philips, Phillips
Finnegan, Finnigan
Finlay, Finley
Farrell, Ferrell
Ferrara, Ferreira, Ferriera
Foerster, Forester, Forrester, Forster
Forrest, Forest
Faris, Farriss, Ferris, Ferriss
First, Fuerst, Furst
Fischer, Fisher
Flinn, Flynn
Flanagan, Flanigan, Flannigan
Frei, Frey, Fry, Frye
Freedman, Friedman
Frederickson, Frederiksen, Fredrickson, Fredriksson
Franck, FrankFrance, Frantz, Franz
Frances, Francis
Freeze, Freese, Fries
Fraser, Frasier, Frazer, FrazierGood, GoodeGetz, Goetz, Goetze
Goff, GoughGold, Goold, Gould
Gilmer, Gilmore, Gilmour
Gallagher, Gallaher, Galleher
Gomes, Gomez
Guenther, Gunther
Gonzales, Gonzalez
Gonsalves, Gonzalves
Garratt, Garrett
Garrity, Geraghty, Geraty, Gerrity
Gorden, Gordohn, Gordon
Gardiner, Gardner, Gartner
Garrard, Gerard, Gerrard, Girard
Gauss, Goss
Gray, Grey
Griffeth, Griffith
Green, GreeneGros, Grose, Gross
Hyde, Heidt
Hoff, Hough, Huff
Hoffman, Hoffmann, Hofman, Hofmann, Huffman
Hoag, Hoge, Hogue
-129-

H*GN

H*KSN
H*L
H*L
H*L
H*L
H*LD
H*LG
H*LM
H*LMS
H*LN
H*M
H*MR
H*N
H*N
H*NN
H*NRKSH*NRKSN
H*NS
H*NS
H*NSN

H*R
H*R
H*R
H*R
H*RMN
H*RMN
H*RMN
H*RN
H*RN
H*RN
H*RNGDN
H*S
H*S
H*S
H*SN
H*VR
J*
J*FR
J*FRS
J*KB
J*KBSN
J*KS
J*L
J*MS
J*MSN
J*NSN
J*S

Hagan, Hagen
Hauch, Hauck, Hauk, Hauke
Hutcheson, Hutchison
Holley, Holly
Holl, Hall
Halley, Haley
Haile, Hale
Holiday, Halliday, Holladay, Holliday
Helwig, Hellwig
Holm, Home
Holmes, Homes
Highland, Hyland
Ham, Hamm
Hammar, Hammer
Hanna, Hannah
Hahn, Hahne, Hann, Haun
Hanan, Hannan, Hannon
Hendricks, Hendrix, Henriques
Hendrickson, Henriksen, Henrikson
Heintz, Heinz, Heinze, Hindes, Rinds,
Hines, Hinze
Haines, Haynes
Henson, Hansen, Hanson, Hanssen,
Hansson, Hanszen
Herd, Heard, Hird, HurdHart, Hardt, Harte, Heart
Hare, Hair
Hardey, Hardie, Hardy
Hartman, Hardmen, Hardmon, Hartmann
Herman, Hermann, Herrmann
Harman, Harmon
Heron, Herrin, Herron
Hardin, Harden
Horn, Horne
Herrington, Harrington
Haas, Haase, Hasse
Howes, House, Howse
Hays, Hayes
Houston, Huston
Hoover, Hover
Jew, Jue
Jeffery, Jeffrey
Jefferies, Jefferis, Jefferys, Jeffreys
Jacobi, Jacdby
Jacobsen, Jacobson, Jakobsen
Jacques, Jacks, Jaques
Jewell, Juhl
Jaimes, James
Jameson, Jamieson, Jamison
Jahnsen, Jansen, Jansohn, Janssen, Jansson,
Janzen, Jensen, Jenson
Joice, Joyce
-130-

Ic

K*

K*F
K*FMN
K*K
K*L-----

K*L
K*LMN
K*LR
K*MBRLN
K*MBS
K*MP
K*MPS
K*N
K*N
K*N
K*N
K*N
K*N
K*N
K*NL
K*NR
K*NS
K*P
K*PL
K*R
K*R
K*R
K*R
K*R
K*RD
K*RLN
K*RN
K*RSNR
K*S
K*S
K*S
K*SL
K*SLR
K*SR
KL*N
KL*RK
KL*SN
KR*
KR*GR
KR*MR
KR*N
KR*S
KR*S
KR*S
KR*S
KR*SNSN

Kay, Kaye
Coffee, Coffey
Coffman, Kauffman, Kaufman, Kaufmann
Cook, Cooke, Koch, Koche
Cole, Kohl, Koll
Kelley, Kelly
Coleman, Colman
Koehler, Koeller, Kohler,Koller
Chamberlain, Chamberlin
Combs, Coombes, Coombs
Camp, Kampe, Kampf
Campos, Campus
Cahn, Conn, Kahn
Cahen, Cain, Caine, Cane, Kain, Kane
Chin, Chinn
Chaney, Cheney
Coen, Cohan,-Cohen, Cohn, Cone, Koehn, Kohn
Coon, Kuhn, Kuhne
Kenney, Kenny, Kinney
Conley, Conly, Connelly, Connolly
Conner, Connor
Coons, Koontz, Kuhns, Kuns, Kuntz, Kunz
Coop, Co-op, Coope, Coupe, Koop,
Chapel, Chapell, Chappel, Chappell, Chappelle, Chapple
Carrie, Carey, Cary
Corey, Cory
Carr, Kar, Karr
Kurtz, Kurz
Kehr, Ker, Kerr
Cartwright, CortrightCarleton, Carlton
Carney, Cerney, Kearney
Kirschner, Kirchner
Chace, Chase
Cass, Kass
Kees, Keyes, Keys
Cassel, Cassell, Castle
Kesler, Kessler, Kestler
Kaiser, Kayser, Keizer, Keyser, Kieser, Kiser, Kizer
Cline, Klein, Kleine, Kline
Clark, Clarke
Claussen, Clausen, Clawson, Closson
Craw, Crowe
Krieger, Kroeger, Krueger, Kruger
Creamer, Cramer, Kraemer, Kramer, Kremer
Craine, Crane
Christie, Christy, Kristee
Crouss, Kraus, Krausch, Krause, Krouse
Cross, Krost
Crews, Cruz, Kruse
Christensen, Christiansen, Christianson

Li

-131-

F

L*
L*
L*D

L*DL
L*DRMN
L*K
L*KS
L*LN
L*LR
L*MB
L*MN
L*MN
L*N
L*N
L*N
L*N
L*NG
L*NN
L*NS
L*R
L*RNS
L*RNS
L*RSN
L*S
L*S
L*SR
L*V
L*VD
L*VL
L*VN
M*D
M*DN
M*DS
M*DSN
M*KL
M*KM
M*KS
M*KS
M*LN
M*LN
M*LR
M*LR
M*LR
M*LS
M*N
M*NR
M*NR
M*NSN
M*R
M*R
M*R
M*R
M*R

Loewe Low Lowe
Lea, Lee, Leigh
Lloyd, Loyd
Litle, Littell, Little, Lytle
Ledterman, Letterman
Leach, Leech, Leitch
Lucas, Lukas
Laughlin, Loughlin
Lawler, Lawlor
Lamb, Lamm
Loc.=

Lemen, Lemon, Lemon
Layman, Lehman, LehmannLind, Lynd, Lynde
Lion, Lyon
Lin, Linn, Lynn, Lynne
Lain, Laine, Laing, Lane, Layne
Lang, Lange
London, Lundin
Lindsay, Lindsey, Lindsley, Linsley
Lawry, Lavery, Lowrey, Lawry
Lawrence, Lowrance
Laurence, Lawrance, Lawrence, Lorence, Lorenz
Larsen, Larson
Lewis, Louis, Luis, Luiz
Lacey, Lacy
Leicester, Lester
Levey, Levi, Levy
Leavett, Leavitt, Levit
Lavell, Lavelle, Leavelle, Loveall, Lovell
Lavin, Levin, Levine
Mead, Meade
Morretton, Morton
Mathews, Matthews
Madison, Madsen, Matson, Matteson, Mattison, Mattson
Michael, Michel
Meacham, Mechem
Marques, Marquez, Marquis, Marquiss
Marcks, Marks, Marx
Maloney, Moloney, Molony
Mullan, Mullen, Mullin
Mallery, Mallory
Moeller, Moller, Mueller, Muller
Millar, Miller
Miles, Myles
Mahan, Mann
Miner, Minor
Monroe, Munro
Monson, Munson
Murray, Murrey
Maher, Maier, Mayer
Mohr, Moor, Moore
Meyers, Myers
Meier, Meyer, Mieir, Myhre-132-

AIRMINIMMIlt1

M*RF
M*RL
M*RN
M*Rs
M*RS
MK*
MK*
MK*
MK*
MK*L
MK*LF
MK*LM
MK*N
MK*NR
MK*NS
MK*NS
MK*R
MK*R
MKD*NLMKF*RLNMKF*RSNMKL*D
MKL*KLNMKL*LN
MKL*N
MKL*N
MKL*S
MKM*LN
MKN*L
MKR*D
N*KL
N*KLS
N*KLS
N*KLSN
N*KSN
N*L
N*LSN
N*MN
N*RS
N*SBD
P*D
P*DRSN
P*G
P*LK
P*LSN
P*N
P*R
P*R
P*RK
P*RKS

Murphey, Murphy
Merrell, Merrill
Marten, Martin, Martine, Martyn
Meyers, Myers
Maurice, Morris, Morse
McCoy, McCaughey
Magee, McGee, McGehee, McGhie
Mackey, MacKay, Mackie, McKay
McCue, McHugh
Magill, McGill
McCollough, McCullah, McCullough
McCallum, McCollum, McColm
McKenney, McKinney
MacIntyre, McEntire, McIntire, McIntyre
MacKenzie, McKenzie
Maginnis, McGinnis, McGuinness, McInnes, McInnis
Maguire, McGuire
McCarthy, McCarty
MacDonald, McDonald, McDonnell
MacFarland, MacFarlane, McFarland, McFarlane
MacPherson, McPherson
MacLeod, McCloud, McLeod
MacLachlan, Maclachlin, McLachlan, McLaughlin, McLoughlin
McClellan, McClelland, McLellan
McClain, McClaine, McLain, McLane
MacLean, McClean, McLean
McCloskey, McClosky, McCluskey
MacMillan, McMillan, McMillin
MacNeal, McNeal, McNeil, McNeill
Magrath, McGrath
Nichol, Nicholl, Nickel, Nickle, Nicol, Nicoll
Nicholls, Nichols, Nickels, Nickles, Nicols
Nicholas, Nicolas
Nicholsen, Nicholson, Nicolaisen, Nicolson
Nickson, Nixon
Neal, Neale, Neall, Neel, Neil, Neill
Neilsen, Neilson, Nelsen, Nelson, Nielsen, Nielson,
Nilson, Nilssen, Nilsson
Neumann, Newman
Norris, Nourse
Nesbit, Nesbitt, Nisbet
Pettee, PettyPeterson, Pederson, Pedersen, Petersen, Petterson
Page, Paige
Polak, Pollack, Pollak, Pollock
Polson, Paulsen, Paulson, Poulsen, Poulsson
Paine, Payn, Payne
Parry, Perry
Parr, Paar
Park, Parke
Parks, Parkes
-133-

P*RS
P*RS
P*RS
P*RSN
PR*KR
PR*NS
PR*R
R*
R*
R*BNSN
R*D
R*D
R*D
R*DR
R*DS
R*GN
R*GR
R*K
R*K
R*KR
R*L
R*MNGTN
R*MR
R*MS
R*N
R*NR
R*S
R*S
R*S
R*S
R*VS
S*BR
S*FL
S*FN
S*FNS
S*FNSN
S*FR

S*FR
S*GL
S*GLR
S*K
S*KS
S*L
S*L
S*LR
S*LS
S*LV
S*LVR
S*MKR

Pierce, Pearce, Peirce, Piers
Parish, Parrish
Paris, Parris
Pierson, Pearson, Pehrson, Peirson
Prichard, Pritchard
Prince, Prinz
Prior, Pryor
Roe, RoweRae, Ray, Raye, Rea, Rey, Wray
Robinson, Robison
Rothes RothRudd, Rood, Rude
Reed, Read, Reade, Reid
Rider, Ryder
Rhoades, Rhoads, Rhodes
Regan, Ragon, Reagan
Rodgers, Rogers
Richey, Ritchey, Ritchie
Reich, Reiche
Reichardt, Richert, Rickard
Reilley, Reilly, Reilli, Riley
Remington, Rimington
Reamer, Reimer, Riemer, Rimmer
Ramsay, Ramsey
Rhein, Rhine, Ryan
Reinhard, Reinhardt, Reinhart, Rhinehart, Rinehart
Reas, Reece, Rees, Reese, Reis, Reiss, Ries
Rauch, Rausch, Roach, Roche, Roush
Ruth, Rusch
Russ, Rus
Reaves, Reeves
Seibert, Siebert
Schofield, ScofieldStefan, Steffan, Steffen, Stephan, Stephen
Steffens, Stephens, Stevens
Steffensen, Steffenson, Stephenson, Stevenson
Schaefer, Schaeffer, Schafer, Schaffer, Shafer,
Shaffer, Sheaffer
Stauffer, Stouffer
Siegal, Sigal
Sigler, Ziegler
Schuck, Shuck
Sachs, Sacks, Saks, Sax, Saxe
Seeley, Seely, Seley
Schell, Shell
Schuler, Schuller
Schultz, Schultze, Schulz, Schulze,
Shults, Shultz
Silva, Sylva
Silveira, Silvera, Silveria
Schomaker, Schumacher, Schumaker, Shoemaker, Shumaker
-134-

S*MN
S*MN
S*MRS
S*MS
S*N
S*N
S*NR
S*NRS
S*PR
S*R
S*R
S*R
S*R
S*R
S*1iL

S*RLNG
S*RMN
S*RN
S*RR
S*S
SM*D
SM*D

SN*DR
SN*L
SP*LNG
SP*R
SP*R
SR*DR
SR*DR
T*D
T*MSN
T*RL
TR*S
V*L
V*L
V*R
W*D
W*DKR
W*DL
W*DMN
W*DR
W*DRS
W*GNR
W*L
W*L
W*L
W*LBR
W*LF

W*LKNS
W*LKS
W*LN
W*LR

Simon, Symon
Seaman, Seemann, Semon
Somers, Sommars, Sommers, Summers
Simms, Sims
Stein, Stine
Sweeney, Sweeny, Sweney
Senter, Center
Sanders, Saunders
Shepard, Shephard, Shepheard, Shepherd, Sheppard
Stahr, Star, Starr
Stewart, Stuart
Storey, Story
Saier, Sayre
Schwartz, Schwarz, Schwarze, Swartz
Schirle, Shirley
Sterling, Stirling
Scheuermann, Schurman, Sherman
Stearn, Stern
Scherer, Shearer, Sharer, Sherer, Sheerer
Sousa, Souza
Smith, Smyth, Smythe
Schmid, Schmidt, Schmit, Schmitt, Smit
Schneider, Schnieder, Snaider, Snider, Snyder
Schnell, Snell
Spalding, Spaulding
Spear, Speer, Speirer
Spears, Speers
Schroder, Schroeder, Schroeter
Schrader, Shrader
Tait, Tate
Thomason, Thompson, Thomsen, Thomson, Tomson
Terrel, Terrell, Terrill
Tracey, Tracy
Vail, Vaile, Vale
Valley, Valle
Vieira, Vierra
White, Wight
Whitacre, Whitaker, Whiteaker, Whittaker
Whiteley, Whitley
Whitman, Wittman
Woodard, Woodward
Waters, Watters
Wagener, Waggener, Wagoner, Wagner, Wegner, WaggonerWilley, Willi
Wiley, Wylie
Wahl, Wall
Wilber, Wilbur
Wolf, Wolfe, Wolff, Woolf, Woulfe,
Wulf, Wulff
Wilkens, Wilkins
Wilkes, Wilks
Whalen, Whelan
Walter, Walther, Wolter-135-

W*LRS
W*LS
W*LS
W*LS
W*LSN
W*N
W*R
W*R
W*RL
W*RNR
W*S
WtSMN

Walters, Walthers, Wolters
Wallace, Wallis
Welch, Welsh
Welles, Wells
Willson, Wilson
Winn, Wynn, Wynne
Worth, WirthWare, Wear, Weir, Wier
Wehrle, Wehrlie, Werle, Worley
Warner, Werner
Weis, Weiss, Wiese, Wise, Wyss
Weismann, Weissman, Weseman, Wiseman,
Wismonn, Wissman
REFERENCES

1

Cox, N.S.M. and J.L. Dolby. "Structured Linguistic Data andthe Automatic Detection of Errors." In: Advances in Computer Typesetting. London, Institute of Printing, 1966.
pp. 122-125.

2.

Cox, N.S M o, J.D. Dews, and J.L. Dolby. The Computer and theHamden, Conn., Archon Press, 1967.
Library..

3.

"Efficient Automatic Error Detection in BiblioDolby, J.L.
graphic Records." R & D Consultants Company Report,
April 1968.

4.

Information Storage andBecker, Joseph and Robert M. Hayes.
Retrieval. New York, Wiley, 1963. p. 143.

5.

"Retrieval of Misspelled Names in Airlines
Davidson, Leon.
Passenger Record System." Communications of the ACM,
v. 5 (1962), pp. 169-171.

6.

"A Program for Correcting Spelling Errors."
Blair, C.R.
Information & Control, v. 3 (1960), pp. 60-67.

7.

"An Adaptive Information Transmission System
Schwartz, E.S.
Employing Minimum Redundancy Word Codes." Armour Research
Foundation Report, April 1962. (AD 274-135).

8.

"A Study of Methods for SystematBourne, C.P. and D. Ford.
ically Abbreviating English Words and Names." Journal of
the ACM, v. 8 (1961), pp. 538-552.

9.

Tukey, X.161.

10.

Resnikoff, A. and J.L. Dolby. "A Proposal to Construct a
Linguistic and Statistical Programming System." R & D
Consultants Company, 1967.

o"A Tagging System for Journal Articles 91d Othera Status Report." Statistical Techniques
Citable Items:
Research Group, Princeton University, 1963.

-136-

APPENDIX II

USER'S GUIDE TO THE TERMINAL MONITOR SYSTEM (TMS)

By

William D. Schieber
Institute of Library Research
University of California
Berkeley, California

USER'S GUIDE TO THE TERMINAL MONITOR SYSTEM (TMS)
By

William D. Schieber
Institute of Library Research
University of California
Berkeley, California

A.

INTRODUCTION

The Terminal Monitor System (TMS) provides on-line terminal
access to the Computer Center's IBM 360/40. All files are maintained on ILR's private disk facilities and are not accessible
by other 360 users. The system performs five general functions:
Text entry: the establishment of new files in which1.
the records can later be processed randomly.
File search: retrieval and display of records within
2.
an existing file.
Text editing: addition, replacement, and deletion of3.
character strings and individual records, within anexisting file.
Compilation of source programs: conversion to execut4.
able instructions from source language entered in same
fashion as text.
Interface to special user-written routines: ability for5.
terminal user to load and execute special-purpose programs.
B.

GENERAL DESCRIPTION OF TMSTMS has a time sharing design which allows multiple terminals
to operate at the same time. Like any conversational system itallows the user to carry on a dialog with the computer, and will
wait for the user to enter his response before continuing the processing. Descriptions of the formats of monitor messages and user
responses to th,.se messages are described in the following sections.
One section is included for each of the first three of the processing modules. The last two functions will be described in a subsequent edition of this guide.
C.

2740 OPERATIONThe IBM 2740 terminal is, in most respects, similar to a
In order to use it as a computer terminal, thenormal typewriter.
switch on the right marked 'COM' and 'LCL' must be set to 'COM',and the ON-OFF switch set to 'ON'. At this point the standby
light (marked '5') should be on. Messages sent to the terminalby the monitor will cause the receive ('10) light to go on during
the transmission. When the terminal is requested to send text
(Transmit) the user must press the 'Bid' key (which causes thetransmit ('T') light to go on). At this point he may type themessage he wants to send. The message mast be ended with theThis signals the monitor that the terminal'EOT' character.
njhas completed transmission, so that it may process the incoming
text.

DETAILED DESCRIPTION OF UTILITY PROCESSORSD.

There are six different utility processors which the terminal user may summon. However, before calling any of these
for the first time he vill be asked by the monitor to log in.
The format is currently as follows:
The monitor sends:

TERMINAL READYPLEASE LOG IN
Terminal user responds with his name:doe, john

Following this, the monitor will respond:(THANK YOU--TERMINAL CLEAR!

At this point the terminal is ready to enter any processor
existing in the ILR processor library. Processors currently
operational are:
Text Processor. The text processor is used to create
a new file. Records so entered can be retrieved later by Key.
The keys generated by the processor, have numeric value. To
enter this processor, type in:1.

itextiTMS responds:'TEXT PROCESSORTYPE FILE NAME1.

At this point, type the name by which you wish the file tobe known. If you are typing a source program, the file namemust be the name of the entry point (in assembler: theCSECT name; in PL/1: the PROCEDURE name). The file namemay be up to 8 characters in length. The format of it is:(name of file)1

If this name has not been used before, you will receive
space for the new file on the disk; before beginning text
entry, you will be asked to indicate the treatment of lower
case characters in creating the file:
SPECIFY CASE
If you wish lower case letters to be translated into upper
case, (you must if you are typing a program), type:Wu((indicating lower to upper case translation)

If you wish lower case characters to maintain their values,
type:
-140-

ITT1

(for lower to luver case translation)

with:
If you had specified '1/u' case, TMS would respond

1L/U CASE--BEGIN TEXT ENTRIJ
and would go on to the next line to type the first key:oolo
You may now enter the first record. It must not
Do notexceed 72 characters and must end with an "EOT".
TMS willenter a carriage return at the end of the line:
do this before it types the next record number.
To close the file when you have finished entering
text, type I+++' in position 1.

TM'S responds:

EXIT TEXT PROCEEal

used toSearch Processor. The search routines are
To call
display one or more records from an existing file.
the processors type:
2.

srch

TMS will respond:

1SEARCH ROUTINES--SELECT Ffal

User responds by typing the file name:(file name)

TMS, after locating the file, will respond:
FFEEi SEARCH REQUEST!
One isTwo facilities for search are available.
display of one or more records, where the records are
identified by the exact full key. If one specific record is desired, type:dis 1

If you want to display a group of records type:'display(key 1)1 to(key 2)1

where the value of key1 is less than key2, and key1 is an
full length.
actual key in the file. Both values must be of
The second type of search enables the user to scanportions of
the file using abbreviated key prefixes, orIn this type of search all records of the given
the key.
given
key class will be presented. For example, if a0015,
file contains records having keys of 0004, 0010,
and 0020, and a scan on key class 001 is requested, only
therecords whose leftmost three key characters match
three-character scan value will be presented; here, reis:
cords 0010 and 0015. The format for this request
Iscan i(key class)-1141-

After a search request is entered, records are presented sequentially until the request has been satisfied,or until the end of file is reached. However, to enable
the user to stop printing of the file, a checkpoint isentered following printing of the tenth record. If you
want to continue display, type a carriage return. If you
want to stop processing the request, type three plus signs
When the request is satisfied, or when it has been interrupted at a checkpoint TMS will type:
REQUEST!TYPE

at which time the user may enter another request or leave
the search mode. To exit, type '+++'. TMS then responds:[EXIT SEARCH PROCESSORI

Edit Processor. This facility is used to edit existreing files. There are three conventional edit functions:placement, addition, and deletions. These functions may be performed on individual characters, on character strings, on records, or on groups of records within a file.
3.

The edit processor is called by typing:
LeditiTMS responds:
LuaIT MODE -- SELECT FILO

User types in:(file name)

TMS replies:
1SPECIFY 6-51.1

To which the user must respond:

if he wishes lower case characters to be mapped
1/u
into upper, or:1/1

if he wishes lowei* to retain their values.

TMS now responds with:

PLACE EDIT REQUEST
The general format for an edit request is:r(Tode)+(key) 1+ I (character string) I + I

where 'code' is a two-character code which identifies theedit function, '+' is the tab character, 'key' is the actual
key value of a record, and 'character string' represents
data to be added to, replaced in, or deleted from the record.
-142-

The edit codes are summarized in the table below:
Done On

Edit

Records

Characters

Function
Replacement

RC

Addition

AC

AR

Deletion

DC

DR

_

Specific edit formats are as follaws:
1) REPLACE CHARACTERS - RC.

where 'old string' is the group of characters which are to
be replaced by those presented in the 'new string'.
2) ADD CHARACTERS - AC.

AC

(key)

(old string)in(new string)+where 'old string' is a character sequence following which
the 'new string' will be inserted.
3) DELETE

CHARACTERS - DC.

[DC 1+1 (key) i + Listring)where 'string' is the character sequence to be deleted.
4) ADD RECORD - AR.
[AR 1+(key)(new recorld

where 'key' is the key of the new record.
5) DELETE RECORD - DR.
1DR 1+ L(k)

where key is the key of the record to be deleted.
Following execution of the edit request TMS will indicate completion and invite you to place your next edit
request. When you have no more editing to do on the current
file you may exit by typing three plus signs. TMS thenresponds withEDIT MODE EXIT[TERMINAL CLEAR-143-

APPENDIX III

A DESCRIPTION OF LYRIC, A LANGUAGE FOR REMOTE INSTRUCTION BY COMPUTER

By

Stephen S. Silver
Institute of Library Research
University of California
Los Angeles, Califotnia

A DESCRIPTION OF LYRIC, A LANGUAGE FOR REMOTE INSTRUCTION BY COMPUTER
By

Stephen S. Silver
Institute of Library Research
University of California
Los Angeles, California

A.

PURPOSE AND SCOPELYRIC (Language for Your Remote Instruction by Computer) was
originally intended by its authors, Gloria and Leonard Silvern, foruse in computer-assisted instruction (CAI).* The language is, however, so simple and general that it can be used with ease in on-line
processor simulation and tightly controlled interactive situations.
The instructional strategist (programmer) can selectively display
Branching andtext, and read and analyze user-supplied answers.
jumping based on user-supplied information may be performed. These
text display and answer checking functions have been implemented at
the Institute of Library Research (Los Angeles) using IBM 2260 display consoles and the System/360 model 75 computing system, controlled by OS/360.
In addition, the full language implementation will give theprogrammer the ability to set counters and constants, store text,
and display this stored information at any time. It will be possiand
ble to identify each student uniquely and to record his answers
progress in the instructional program.
The current LYRIC monitor has infinite loop protection. Wheneighty operations are performed without an intervening write command, a read operation is forced. Typing the word 'end' at the top
left of the screen will return control to the time-sharing monitor.
B.

LYRIC RELEASE II

General Features. Release II of LYRIC will, when completed,have enough data acquisition and manipulation commands to perform
most of the error checking functions normally required by batch
scanning programg. The goal of this release is to give an information retrieval specialist the ability to write a LYRIC program thatwill make requests, accept answers, test the answers for acceptability, and modify a file used by other information processing programs.
1.

The other use for LYRIC is for its original intended purposeof computer assisted instruction. Courses can be prepared to give

*Silvern, Gloria M. and Leonard C. Silvern. "Computer-assisted inand prostruction: specification of attributes for CAI programs
Proceedings, 21stgrammers. Association for Computing Machinery.
National Conference. Washington, D.C., Thompson, 1966. pp. 57-62.
!,/4:,/=147-

a student enough background to use an information system or for any
course of study that lends itself to this technique.
Specifically, LYRIC will have many of the answer manipulationand storage commands that were lacking in the first ILR implementation. A whole series of inter-program linkage commands will be
written to allow LYRIC to control the flow of an information search.
Limitation of the Current Release. The major limitation ofthis release will be its inability to format output files in any
precise manner. Output will be in a format similar to OS/360's
RECFM=VB. Other limitations will be the techniques for indexing
counters. Only addition and subtraction will be allowed at first.
2.

Card Format. Each LYRIC statement is one line long andconsists of a label, an operation code, and a text or operand field.
The label is at column 1, the operation code at 10, and the text
at 16.
3.

Blocking of Lyric Records. The LYRIC executer will beable to sense and deblock records up to 12 cards per physical reThis block size is controlled by the BLKSIZE sub-parametercord.
in the DCB parameter on the OUTPUT DD card used by the assembler.
4.

Implementation Restrictions.
restrictions are:
5.

1)
2)
3)

The current implementation

labels shall be = 8 characters in length;
operation codes shall be = 5 characters in length;
text or operand fields shall be = 40 characters inlength.

All constants shall be S360 halfwords, except string-defining
constants which shall be two 1-byte instructions packed into a halfword. One byte will be a length specification and the other will bea location for the start of the operand.
Basic Programming Conce ts. Executor and assembler optimization will take second place to logic and simplicity of structure.
Use of subroutines (BAL type) will be used whenever possible. Table
look-up will be used whenever feasible to facilitate debugging andmodification of the system. Optimization for a particular system
will NOT be performed in this release. Absolute references to a
LYRIC record will not be done. Only relative statement numbers will
be used. Input/Output will be well defined and in subroutine form
to allow use on smaller machines not having a universal character
6.

set.
C.

THE LYRIC ASSEMBLER

The LYRIC ass ab.ler processes LYRIC source input, convertingit to executable LYRIC object code. The assembler.is divided into
two passes. Pass 1 reads in all of the LYRIC source data, modifies
an interpretable LYRIC object data set, and generates numerous-148-label tables for use in pass 2. Pass 1 updates and completes theobject module produced in pass 2, based on label table informationand generates a complete listing of the LYRIC program including any
error messages which may have been generated.
Pass I. Pass 1 reads in each LYRIC source statement sequenBefore
tially, producing an object record for each input statement.
processing each statement, error flags which are analyzed anq printed
in pass 2, are cleared. The default TRUE and FALSE LYRIC branchlocations are set to the default value of the next relative LYRIC
location. A running "current LYRIC location" is kept and updated
each time a LYRIC source statement is read in. This location value
is the basis for most entries in the various label tables.
I.

Explicit labels on source statements are entered into a generallabel table which saves the literal value of the label and its location value relative to the beginning of the LYRIC data set (using
the previously mentioned location pointer). If the label table isfull, an error flag is generated within the object LYRIC statement
and no entry is made in the label table.
Pass 1 continues with a branch to an operation code analysis
section 1
routine. This routine is divided into two sections:searches a list of all known valid op codes. If the op code underanalysis is not entered in this table another error flag is set in
the object record and processing continues by moving the operation
code, text, and label information into the in-core object record,
writing it out in the object data set (error flags and all) and
If the operation
looping back to read more LYRIC source statements.
code is found inthis section, control is turned over to the appropriate routine in section 2.
Section 2 is a series of macro-defined subroutines containing
all the information necessary to set pointers and otherwise process
a LYRIC op code. These subroutines can define the existence of astatement, decide whether or not a LYRIC block starts at this point,
decide whether a LYRIC block can end at this statement, define thesetting of object constants within the object module, and decide thebranching to NEX or PRO. These functions are performed by the useof some standard utility routines which modify label tables andanalyze operands.
An indicated branch to NEX or the start of the next LYRIC blockis accomplished by setting a global flag indicating that a NEX branch
is being attempted, setting a flag in the LYRIC object record indicating that a NEX has been requested and a label resolution is to be
performed in pass 2. A system generated number is loaded into the
TRUE branch constant of the object record. This system number isthe same number that will be forced into the general label table when
the next statement that may legally start a LYRIC block is encountered and the NEX global flag is on.
A branch to PRO is resolved in pass 1 by loading the present
value of the current step (which is updated each time an op code-149-

that starts a step is encountered) into the TRUE location and setting
an indicator flag in the record showing that the TRUE pointer has
been set.
At present operands are analyzed as either numerical fields ortext-character strings. By specifying what you wish 'co be searched
for, subroutines are invoked which search the proper fields on thesource statement and which load corresponding constant areas in theobject statement. Any error encountered will be flagged in the object statement.
The assembler parses an operand in a number of separate phases
for simplicity of coding and debugging.

Phase I (STRUCTURE) scans the whole operand field from left toright looking for operand separator delimiters (usually '") or
Starting at the boundaries of the operand subbracketed by blanks.
field, the search starts from the left to the right until either anon-blank or a string delimiter is rcached. The scan then searches
from right to left for a non-blank or string delimiter. The routine
then returns the true length of the searchable part of the string(less the string delimiters, if any) and the starting point of thestring (also less string delimiters) relative to the first positionof the whole operand field.
A possible difference for the quoted string is for a verbatim
test of the answer, starting and ending at the correct boundaries.
The unquoted string should be used for logical answers where theoperand is a logical word or phrase which is logically but notphysically surrounded by blanks. The difference between the twoshould be interpreted by the executer. The real differences comeonly at the boundaries of the answer where a non-existent blankoverhang should be assumed for non-quoted strings.
Pass II. Pass 2 produces a final LYRIC listing and resolves
all label references, which results in a fully executable LYRIC object module.
2.

When the last source statement encountered in pass 1 is processed, the object data set is closed, rewound, and opened for update so that the labels can be resolved in place.
Pass 2 now reads the object record as input. As each record
is read a subroutine is called which tests the object flags to see
if a label resolution must be done. If so, a match is attempted
comparing either the label table entries with the explicit label
in the object record or the label table's implicit system labelswith the system label value saved in the TRUE pointer. Any unresolved labels will print out an error message.

If a branch to the next phrase is required, a subroutine will
extract the relative length of the branch from a table generated inpass 1 and add it to the cureent location, thus generating the-150-

relative location of the next phrase.
into the FALSE pointer.

This value is then loaded

After these operations are finished the completed object record is printed in an expanded format, follawed by any error
messages that may have been generated. The object record is
updated to its final value and the program continues with the next
object record.
When the object deck has been exhausted, assembly is terminated.
D.

THE LYRIC EXECUTOR

The LYRIC executor is designed to interpret a LYRIC object data
set produced by the LYRIC assembler. While the LYRIC assembler is for most purposes - device independent, the executor is quite machine
dependent. The executor must interface between the intent of the
LYRIC programmer, the computer operating system, and the particulargraphical output terminal being used. The first function of theThisexecutor is to try to find and open a LYRIC student record.
student
andcanberecord saves status information on a particulardisplayed only by the instructor or, under certain conditions, thestudent. If this data set is not found, it is assumed that any instructions in the LYRIC program requiring the use of the student
data set will be treated as a "no operation."
Next, the LYRIC object deck is located and opened for use.
Execution will start wither at the beginning or at a location indicated by status information in the student record.
The general construction of the executor is very similar to theassembler since there is a driving section and many utility andoperand analysis subroutines.
Execution starts by branching to the statement extraction routine which reads in one object record as pointed to by the last
branch address (set by TRUE or FALSE pointers depending on the execution of the last instruction). If this turns out to be an invalid
record (not within the scope of the LYRIC data set) execution is
terminated and control is passed back to the operating system. If
the instruction defines the start of a new LYRIC step, an appropriate
pointer is set. A default value for the next retrievable record istaken from the TRUE pointer in the object record. Control is passed
back to the driving routine. The rest of the executor deals withthe execution of each of the individual commands. If the command
agrees with a list of legal commands, control is passed to that particular subroutine.
For example, GTO's would simply cause the record extraction
routine to get a record from the location pointed to, in the instructions operand field. A test continuation card would load a buffer
which, when filled (device dependent), would be dumped onto a display screen.
-151-

E.

CONCLUSION

The remarks made dbove should suggest some of the major aspectsof LYRIC in its present form.
A sample LYRIC program, a list of statements now implemented,
and three key operands are presented in the sections which follow.
F.

UCLA LYRIC SYNTAX AND DEFINITIONSCapital letters must be used exactly as shown.

Refers to any string of written information (including
blanks) not exceeding 40 characters.
"text".

Any integer not exceeding 255 (360 one byte constant).

"number".

"character string".

An optional label not to exceed 5 characters in"(label)".
length.
"label".

Any series of 40 or less characters.

A required label not exceeding 5 characters in length.

"executable statement". This is, at present, considered any
statement except additional text or GTO (see section on Operations
and Their Formats.)

All labels must be left justified in their fields. No morethan 40 labels per program including system labels generated by eachstep using a PRO operand.
NOTE:

G.

A SAMPLE LYRIC PROGRAM

PROBLEM: Ask a user if he would like to know how to use LYRIC.
If 'no' tell him he is 'negaIf 'yes' give him a positive response.
tive' and he should type in 'yes'. If he does not type in either'yes' or 'no' give him an error message and ask him tho question
If he is again wrong, terminate the program.
again.

START PRE THIS IS A TEST OF LYRIC.
PRO WOULD YOU LIKE TO LEARN MORE ABOUT
LYRIC? ANSWER YES OR NO.
ANSKEY YESYOU INDEED REALIZE THAT LYRIC IS WELL
WORTH LOOKING INTO.
GTO NEXKEY NOYOU ARE DEFINITELY NEGATIVE.
TRY A "17,S.

GTO START-152-

UNX 1YOU DIDN'T TYPE YES OR NO.
YOU HAVE ONE MDRE CHANCEGTO PROUNX 1
YOU HAVE AGAIN FAILED TO ENTER A'YES' OR A 'NO'.
GTO NEXPRE GOOD-BY
PRE
END
.

END

H.

OPERATIONS AND THEIR FORMATS
1.

PRESTATEMENT:ABSTRACT:Defines the start of a screen of text and a

LYRIC Wock.
SYNTAX:label PRE text
DESCRIPTION:

A new LYRIC block is defined and the text inthe statement field becomes the first text in theblock and on the screen.
EXAMPLE:PRE

This text does not require
user intervention but you
can ask for an ANS anyway.

ANS
COMMENTS:(label) PRE <text>
Present new step, no response required.
Used for display of information and can be theobject of a NEX statement. Resets the 'current step'
location.
2.

STATEMENT:PRO

ABSTRACT:LYRIC problem statement.
-153-

SYNTAX:label PRO text
DESCRIPTION:

Defines the start of a LYRIC step requiring a
It is exactly the same as a PRE since
response.
the ANS must always initiate an answer.
EXAMPLE:

PRO

sam

ANS
KEY

Hello there.
I will help you.
What did you say?
Ok.

COMMENTS:

PRO <text>
Problem statement with required response.
Similar to the PRE but implies the use of a
trailing ANS.
Resets the 'current step' location.
(label)3.

<blank)STATEMENT:ABSTRACT:

Text continuation.
SYNTAX:label <text

DESCRIPTION:

Defines text as it will appear on a console
If branched to without any preceeding PRE
screen.
or PRO's, the text is appended to the text on thescreen with no break in flaw.
EXAMPLE:

PRE
GTO

SAM

The rain in
Spain
SAM

falls mainlyon the plain.
-154-

COMMENTS:(label)<> <text>
Additional text.
May be used anywhere additional text must be produced.

4.

ST4TEMENT:ANSABSTRACT:

Request an answer from a student beginning at
a predefined location.
SYNTAX:label ANS time out number,LOC=(column,row)
ARROW=(NO/column,raw),CURSOR=(column,row)
DESCRIPTION:

The console will be placed in such a state thatwhen the ENTER button is pressed the text typed inby the student will be read into an answer save area
If no options are specified an arrow will be(ANS).
placed after the previous PRE/PRO text with the cursor following it. The answer will be expected after
this point. Answers can be read in from any pointon the screen as defined by LOC=(column,raw). The
default arrow can be eliminated or repositioned, andthe cursor can be positioned.
EXAMPLE:

PRO
ANS
PRO
ANS
ANS

The rain in spain falls where?
LOC=(1,3),ARROW=(1,3),CURSOR=(1,3)
what now?

GTO

MORE

10
You have failed to enter anything.

COMMENTS:This may require special programming for different consoles.
ANS
Request an answer at this point.
A small arrow is displayed indicating the startof the user's answer. The console will then wait
until information is typed in and the ENTER button
is pressed. The screen is read, the answer field is
loaded up to the cursor and execution continues. An
answer may not be longer than 255 characters. Excess
will be truncated.
-155(label)

5.

GUD

STATEMENT:ABSTRACT:

Test the first characters of the input answer
for an exact comparison against a correct answer.
SYNTAX:label GUD text
DESCRIPTION:

If the answer is exactly the same as the operand
up to the cursor, then give a positive result as in
KEY.

EXAMPLE:

PRO
ANS
GUD
GTO
UNX

What color is the sky?

BLUE
BLUE is correct
NEX
1

The sky is BLUE.

You are incorrect.

COMMENTS:(label) GUD <texi>
Test for exact correct answer.
Using the text field, up to but not including
the last blank, the operand is compared with theuser's reply. Only an exact comparison, starting
directly after the arraw, will be considered corOperations proceed as in a KEY.
rect.
6.

STATEMENT:KEY

ABSTRACT:

Search for a keyphrase in an answer.
SYNTAX:label KEY 'keyphrase', 'keyphrase', 'keyphrase'
DESCRIPTION:

Searches for a series of logically ORed keyphrases. As soon as one is found the search is
terminated and the next sequential instruction is
executed. and the required following GTO is executed.
If no match is found, the instruction
immediately follawing the next sequential GTO is
executed.
The search for each keyphrase starts-156-

at the first position of the answer field and ends
at the N+L-1 position where N is the length of theanswer field and L is the length of the keyphrase
and the first position of the answer is at position
Each keyphrase is treated as a separate search1.
(at the same level).
EXAMPLE:

KEY

same

GTO
GTO
KEY
GTO
BAD
GTO

lab

'go','to','there'
Excellent job.
ON3 4
NOGOOD'ain't','good' "9999"
GLITCHEnglishBAD9COMMENTS:

Each keyphrase in the statement will besearched for at the very start of the answer field.
The translator could supply the location for eachkeyphrase in the statement as a constant. Lengths
would also be helpful but a pre-TRT for the delimeter would do as well.
(label) KEY (text>
Key word search.
Using the text field, up to but not including
the last blank, searches the user answer for a
If a match occurs execute the next sequenmatch.
If not a match, go to the next
tial instruction.
non-continue, non-GTO statement. The scope of a
KEY usually ends with a GTO display of informationand can be the object of a NEX statement. Resets
the current step location.

7.

BAD

STATEMENT:ABSTRACT:

Test the answer for an exact comparison withan incorrect answer.
SYNTAX:label BAD text
DESCRIPTION:
Exactly the same as a GUD.
-157-

EXAWLE:HOW
You said HOW.
THIS IS NOT A QUESTION.
NEX
GTO
1
UNX
You haven't done anything wrong yet
BAD

COMMENTS:This statement will be exactly equivalent to
the GUD statement. It exists only for the mnemonicvalue.
(label) BAD <text>
Test for exact incorrect answer.
Used for its mnemonic
Acts exactly like GUD.
value.

8.

UNX

STATEMENT:ABSTRACT:Define the action in the event of an undefined
answer.
SYNTAX:label UNX number
DESCRIPTION:

The text of a UNX will be displayed each time
the user passes through the UNX code while in thesame LYRIC step. UNX's are executed when logicallyencountered in the same way as a KEY. They failonly when the statement has been exhausted. A UNX
with a 0 statement is a no-operation.
EXAMPLE:
KEY

YES
Good.

GTO
UNX

NEX
3

GTO

No no not that.
PRO

UNX

I

GTO
KEY
GTO

Very bad.
PRO
Help.
MORE
Well here we are.

PRE

-158-

COMMENTS:

UNX's initialize a special save area wheneverthey are encountered by the executor. They may
occur at any location in a LYRIC step. Leaving a
step initializes the UNX save areas. This technique is compatible with almost any kind of implementation of the UNX function.
(label) UNX <humbei>
Unexpected answer: In the event of an unexpected answer the follawing additional text records
will be displayed. This display will occur eachtime the UNX is executed but not more than the number of times shown in the statement field. When
the UNX is exhausted the next executable instruction is executed as in KEY.
9.

GTO

STATEMENT:ABSTRACT:Transfer control to the program location
indicated.
SYNTAX:label GTO/PHR/PRO/NEX/ANS/label/*
DESCRIPTION:

Causes control to be transferred to an explicitly or implicitly defined statement. The first
GTO after the start of a LYRIC phrase terminates
the phrase.
EXAMPLE:
GTO

PHR

ANS
GTO
GTO
FRO
GTO

ANS
NEX
hi der
PRO

MAN

gto

joe

COMMENTS:A GTO should not force the dumping of thebuffer or you can not concatenate text lines.
GTO <NEX/PROglabel>>
Go to the indicated statement.
NEX = go to the next step: either the next
sequential PRE or PRO encountered.
PRO = Branch to the first preceding PRO
encountered.
<label> = To any legal five character label.
-159-

10.

END

STATEMENT:ABSTRACT:Defines the end of a section of LYRIC code.
SYNTAX:label END blankPname of LYRIC processor module'
DESCRIPTION:

Defines the end of a section of LYRIC code. On
the execution of an END, control is passed to the
LYRIC program in the STATEMENT field, or, if the
STATEMENT field is blank the student is signed outof the LYRIC processor.
EXAMPLE:finish
end

END
END

'CSW015.SSS.LYR2'COMMENTS:

If you branch to END, display the buffer first,
then transfer control.
(label) END
End of program.

11.

STATEMENT:PAC

ABSTRACT:

Remove specified characters from the answer
field.

SYNTAX:label PAC 'character string'
DESCRIPTION:

The individual characters in the statement areremoved whenever they are found in the answer field.
EXAMPLE:
PRO
ANS
PAC
KEY
GTO
UNX

Type in 'encyclopedia'.
'aeiou'NCCLPDvery good.
MORE
1

Not correct at all.

COMMENTS:

The data in the answer field is permanently
changed by this instruction.
(label) PAC <bharacter string>Compact the answer by eliminating the indicated
If the string starts with a blank orcharacters.
if the field is completely blank, the instruction
will eliminate blanks. After the execution of thiscommand, text manipulation commands must take into
consideration the loss of compacted information.

12.

PHR

STATEMENT:

statement

ABSTRACT:A LYRIC phrase.
SYNTAX:label opcode PHR
DESCRIPTION:A LYRIC phrase is defined as the sphere of influence of a standard LYRIC text and display command
It is the location of the statement(i.e. KEY).
immediately following the next executable GTO ending
the LYRIC phrase.
EXAMPLE:KEY

samhello sam
phr
jim

GTO
GTO
joe
The GTO is equivalent to saying:jim
GTO
COMMENTS:

The main use of this will be to define LYRIC
command extents.

13.

STATEMENT:MOD

ABSTRACT:Define device de-endent considerations.
SYNTAX:

nolabel MOD device information-161-

DESCRIPTION:

Allows optimization for different display
devices.
EXAMPLE:

MOD
MOD

2260,LINE=80
CCI,LINE=40-162-

APPENDIX IVILR PROCESSING RECORD SPECIFICATIONby

Jay L. Cunningham
Institute of Library Research
University of California
Berkeley, California

ILR PROCESSING RECORD SPECIFICATION
BY
Jay L. Cunningham,
Institute of Library Research
University of California
Berkeley, California

prepared to document the
NOTICE: The following specification was
preliminary version of an experimental bibliographic storage record
It presents the record(internal processing format) for computer use.
The reader should beware
as of one point in time of an ongoing design.
and codes are taking
that extensive changes in data element definition
where the record isplace both at the Institute of Library Research
being formulated, and at the Library of Congress where the MARC II
Communications Format is being devised in preparation for a nationwide
will bemachine record distribution service. The ILR storage record
compatible with the MARC II record format. For this reason, thetemporary working paper.
present specification is to be considered aA.

INTRODUCTION

internal record
This report provides the specification of the logicaldesign for the ILR File Organization Project. The record is designed
At this writing theto hold records for all types of library materials.
monographs.
field definitions and codes comprise the elements related topertinent
In later phases, the design will be expanded to include elements
The record is not identical toto journal articles and serial titles.
to be convertible to andthe MARC II record. It is, however, intended
quite similar infrom MARC II. In that sense the ILR record appears
structure to the LC MARC II communications format.
and is not written as aThis is a specification for programmer use
field
detailed explanation of the record content. For details on thedefinitions and codes, see the following two documents:

Avram, Henriette D., John F. Knapp, and Lucia J. Rather.
a hic Data.
MARC II Format. a Communications Format for Biblio
6 p
Washington, D.C., U.S. Library of dongre6s, 1968.

The

Cunningham, Jay L. Instruction Manual for Editorial Preparation
Berkeley, Institute ofof Catalog Source Data. Preliminary Edition.
172 p.
Library Research, University of California, 1968.

B.

PROGRAM NAME

computer program codeThis specification has been implemented in aof chit." The
named "INFOCAL"--"Riput for file Organization at U.
The program was written
program exists in Version 1 as of May 6, 1968.
in PL/I, Version 3, Release 14 of F-level, 05/360.

/4W-165-

C.

NATURE OF THE RECORD

The data base in the File Organization Project is record oriented.
That is, there will be one master record for each bibliographic entity.
The initial entity to be recorded is a single book, or "monograph."
The record design has two primary components: data elements and codes.
The data are organized into groups of one or more data elements. Sucha group is called a field.
The IA% is the principal code used in the record. It both identifies the field by function and describes its type of content at themost general level. Example: Tag 100 identifies the field by function
as MAIN ENTRY for a given record, and describes the field as containing
the name of an author of the type PERSONAL NAME.
If a field contains more than one kind of data element, or if it
contains two or more values representing a particalar kind of element,
the elements are separated by codes called sub-field delimiters.
A set of 7 auxiliary codes called indicators accompanies eachfield. At present, only two of these indicators are implemented. The
indicators serve to provide additional information about the field.
An example is TAG 100, which will contain one of the following
values in Indicator 1:
FIGURE 1:

INDICATOR FOR MAIN ENTRY - PERSONAL NAME
Main ELLIty

is not Subject

Form of Name

Main Entry
is Subject

Forename

014Single Surname

1c,Multiple Surname

2

6Name of Family

37

The relationships among data elements and codes used in the record
are summarized in the following tdble:
FIGURE 2:

Record Component

field

sub-field

STORAGE RECORD ORGANIZATION

Content of Componen%---data element (group of
one or more)Type of Code Identifying
the Component,INE,

indicators

delimiter

data element

....

-166-

The ILR storage record contains all data input from a pre-codedcatalog record, with the exception of supplementary data fields
containing what are called "dashed-on entries," in cataloging terminology. Such groups of fields describe a supplement to a book, an
index, appendices, or other related material which is usually boundseparately and often is received after the main work for which thecatalog record was created. The supplementary fields will probably
be stored in a separate record linked to the main record by a number
which is an extension of the master record number. A final decision
has not been made regarding this problem at the time of writing.

D.

RECORD COMPOSITION

The storage record is composed of a variable number of charactersplaced in contiguous byte locations.
A complete storage record is composed of four segments, as
shown in Fig. 3. Each segment is composed of one or more fields.
The fields are of either the fixed or variable-length type, dependingon their function. The fields are organized into one or more data
elements along with certain of their associated codes.* The bibliographic data elements are variable or fixed in presence. The codes
are correspondingly fixed or variable in presence. The variable
length data elements are in principle repeatable, although in practice not all are defined as repeatable. The codes are likewise
repeatable in principle.
The four segments of the record are: the Leader (56 bytes); theRecord Directory (variable length, 12 bytes per directory entry); theFixed Length Data Elements (currently 26 bytes); and the Variable Fields(variable in occurrence, variable in length).

All byte positions in the fixed length segments of the record
are expressed in terms relative to the first byte regarded as one.
This was done only for the purpose of conforming to an early draftof the MARC II format, and will be changed in a future version of theconversion program (INFOCAL).
All fields which always appear in the storage record and which
express the negative condition of a set of values, e.g., in the Fixed
Length Data Elements, will be set to either zeros or blanks. The zeroand blank carry meaning in these fields, usually as the default value.
The zero will normally be set as the default in a binary-valued fixed
length data element and blank will normally be set as the default ina multiple-choice fixed data elements. The latter includes elements
which can have a range (e.g., one to three values).

*The tags and Indicators 1-5 are in the Directory.
7 are at the head of the variable field.
-167-

Indicators 6 andFIGURE 3:

SCHEMATIC OF ILR STORAGE RECORD, INFOCAL VERSION 1(as of May 6, 1968)

Segment 1

Segment 2

DIRECTORYLEADER

Segment 3

Segment 4

FIXED LENGTHVARIABLE FIELDSDATA ELEMENTS
51

N x 12

1

2

NOTE: As currently defined in the record design, Variable Field
access is obtained by a scan of the Directory to find the Tag, then add
contents of "Base Address of Data" to Starting Character Position of(S.C.P. is relative to the first character of the Fixedfield desired.
Length Data Elements.)

Access to the first character of the Fixed Length Data Elements
can be obtained by finding the Base Address of Data at character
positions 19-20 of the Leader, then use the value therein as a displacement from the beginning of the Leader.
The length of the Directory may be determined by a data element
contained in character positions 48-50 in the Leader. It is called"Number of Entries in Directory." Its contents multiplied by 12 gives
the length of the total Directory.

In the record currently implemented, there are no field terminatorsand no record terminator. Control is maintained by a total record
length field in the Leader, and field lengths in the Directory Entryfor each variable field (and for the Fixed Length Data Elements field).

E.

LEADERThe Leader occupies the first 56 characters of every record. It
contains elements describing and identifying the record, in contrast to
the variable data fields, which describe a bibliographic entity (i.e., amonograph). Thus the leader tells the type of content included in therecord, in terms of the form of library material represented; the components of the record structure in terms of the meanings of the tags
and codes for the particular form of library material indicated; andthe hierarchical level at which the record is pitched (e.g., for a monograph which is a member of a series, the record is for the monograph,
not the series).
The program syMbol name for the entire Leader is LIPREFX.
-168-

FIGURE 4:
ILR PROCESSING RECORD - SEGMENT 1, LEADERPROGRAMSYMBOLDATA
ELEMENTCHAR.

POSITION

CONTENTS, REMARKS, ETC.

LILENGTH

RecordLength1-5Total number of characters inrecord,stored as EBCDIC codes;
right justified, with blank fill.

LISDATE

Status Date

6-11

Six-character date referring to
LISTATUS. Currently all blanks.

LISTATUS

Record Status

12

One character.

Contents:

0 - uncertified
1 - certified
2 - changed record
3 - deleted record
LILEGCNT

LegendExtension

13

Provides facility for extending
record type (characters 14-17).
Not currently used. Set tozero.

LITYPE

Record Type14

One Character.

Contents:

a - bookb - manuscript
c - music (sheet)d - music (manuscript)
e - maps and atlases
f - maps (manuscript)

g - motion pictures andfilmstrips- microfilm (original edition)i - phonorecords (spoken)
j - phonorecords (music)
k - pictures, etc.

1 - computer media

rn-other_-169-(Continued on next page)FIGURE 4 (Cont.):
ILR PROCESSING RECORD - SEGMENT 1, LEADERPROGRAMSYMBOLDATA
ELEMENT

LIBLEVEL

Bibliographic
Level

CHAR.

POSITION
15-17

CONTENTS, REMARKS, ETC.
Contents: One to three characters, left justified, with blankfill.

a - analyticc - collectivem - monographs - serial
LIINDCNT

Indicator
count

18

Number of indicator bytes in a
directory entry. Now set to 5.

LIBASE

Base Address of
Variable Fields

19-20

The displacement to the firstcharacter of the fixed fields.
A binary number equal to '56 +12*n', where n is the number of
entries in the directory.

LIORIGIN

Origin ofRecord21-23

Three EBCDIC digits identifying
the agency which keyboarded the'003' = ILR; '790' =record.
U C Santa Cruz.

LIPDATE

Processor
Date

24-29

Six character date referring to
LIPROSOR.
Currently all blanks.

LIPROSOR

Processor orRecord30-32

Three EBCDIC digits indicating
the agency modifying/processing
the machine record. If LIORIGIN
is 790, LIPROSOR is 003.

LISOURCE

Source type of
Catalog card

33

A code identifying the generalsource of the original catalog
card.

a - central (e.g., LC card or
proofslip)
local origin (originalcataloging at LIAGENCY).
c - NUC

other library or source(e.g., Alanar)-170-(Continued on next page)FIGURE 4 (Cont.):
ILR PROCESSING RECORD - SEGMENT 1, LEADERCHAR.

PROGRAMSYMBOLDATA
ELEMENT

LIAGENCY

Agency ofSource Type

34-36

Code for specific agency of
LISOURCE, when it is known.
IntendedThree EBCDIC digits.
for uC network use.

LIADAPTR

Adapter ofCatalog Card

37-39

Three EBCDIC digits identifying
the adapter of the catalog card
when LISOURCE code is other thanIf not known, set to"b".
blanks.

LINUMBER

Master Record
Number (ILRassigned)

40-46

EBCDIC master record number.
Taken from cols. 1-6 of decklet.
Character 40 is zero, currently.

LICHECK

Checksum on
Record Number

47

A checksum on characters 40-46.

LIDIRLEN

Number Entries
in Directory

48-50

EBCDIC digits with leading
blanks.

LIDEOF

Date Enteredon Master File

51-56

EBCDIC in the order 'mmddyy'.
Now set to date of program
execution.

POSITION

CONTENTS, RENARKS, ETC.

[END OF LEADER]F.

DIRECTORY

The record directory is an index to the kind and location of thevariable fields within the record. It contains a series of fields(called directory entries) which contain the tag numbers, the lengthsof the variable fields, and the starting character positions of the
fields. The directory entry is fixed in length, but the number of
entries in a given record cannot be predetermined, so the directory as
a whole is variable in length. The directory is automatically generated by the INFOCAL program.
The program symbol name for the entire directory is LIARTRY.
-171-

FIGURE 5:
ILR PROCESSING RECORD - SEGMENT 2, RECORD DIRECTORY
Length Char. Pos.
Content

Directory Entry 1

3

0

Tag

3

Indicator 1* (EBCDIC character.
If not used, blank.)

1

Indicator 2* (Repeatable tag
number, applicable to thosetags which can appear morethan once in a given record.
If tag is not currently repeatable, indicator will be set tobinary zero. An 8-bit binary
digit.)

5

Indicators 3,4,5 (Character
positions provided for future
expansion. Currently set toblanks.)2

8

Field Length (A 16-bit number
giving the character length ofthe variable field, including
Indicators 6 and 7.)2

10Starting Character Position (A16-bit number giving the position of the first character ofthe variable field. Currently
the first character will always
contain Indicator 6.
This position is relative to the firstcharacter of the Fixed Length
Data Elements Field (Segment 3).)1

1
1

Directory Entry n(3 EBCDIC digits.)+[End of Directory]

The total length of the Directory Entry is 12 Characters. The
total length dt the Directory is 12 x the number of directory entries.
+See Fig. 7 for a list of tags and field names.

*For the most part, Indicator 1 has contents identical to that definedin the LC MARC II specification. Indicator 2 is a feature defined bY
ILR, but suggested by the Library of Congress for the purpose of insuring unambiguous access to data fields having duplicate tags in a
-172record.

The Tags are sorted by the program on the first digit of the tag
number only. The variable fields are stored in the order in which,theywere input (which generally corresponds to the order of appearance on
the source catalog card).
The justification for storing some of the Indicators in the Directory and the rest in the Variable Field was as follows: Indicators
which by their nature apply to the variable field as a whole (e.g.,code for sub-type of name) were stored in the Directory. This renders
them easily accessible when searching the record at the level of the
Directory only, e.g., to test for the presence of certain conditions.
A search for the variable field data itself thus will not have to bemade in each case.
Indicators which by their nature apply only to parts of fields (oreven one character), such as codes for diacritical marks, were placed
within the variable field data itself. Such indicators will most likely
be addessed only when there is actual need to process the variable fieldcontent.
G.

FIXED LENGTH DATA ELEMENTS

Elements in the Fixed Length Data Elements Field are assigned fixedlocations and lengths. The entire field may, in various types ofrecords, assume a variable length, and therefore it has been given a
tag number and a Directory Entry. For the initial version of the
INFOCAL program, only monograph catalog records are being processed.
The Fixed Length Data Elements for monographs have a field length of26, currently.
The program symbol name for the entire Fixed Length Data Elementv
Field is LIFIXED. Its Tag No. is 000.
FIGURE 6:ILR PROCESSING RECORD - SEGMENT 3, FIXED LENGTH DATA ELEMENTS

PROGRAMSYMBOLDATA
ELEMENTLIDTYPE

Date TypeCHAR.
1

POSITION

CONTENTS, REMARKS, ETC.

1c - two dates, second is copyright1

m - two dates, second is
terminal

n - date not known
q - digits missing in originaldater - reprints - single publication date
-173-

(Continued on next page)FIGURE.6 (Cont.):
ILR PROCESSING RECORD - SEGMENT 3, FIXED LENGTH DATA ELEMENTS

PROGRAMSYMBOLDATA
ELEMENT

LIDATE1First or Only
Date

2-5

LIDATE2

Second Date

6-9

CHAR.

CONTENTS, REMARKS, ETC.

POSITION

Four digits, or blank if not known

1
I

LIMICROR

10

Form of
Reproduction

Four digits, or blank if notpresent.

a - microfilm

V - noneb - microfichec - micro-opaque

LIFORM

11-14

Content

Form

One to four, or no characters.
Left-justified, with unused characters filled with blanks.

a - bibliographiesb - catalogs
c - indexes

d - abstracts
e - dictionaries
f - encyclopedias
g - directoriesh - yearbooks
i - statistics
j - handbooks

k - otherm - medical atlases
LIGOVPUB

Government
Publication
Indicator

15

a - U.S. federalb - California statec - California county/municipal

d - internationale - other governments

V - nonei-174-(Continued on next page)FIGURE 6 (Cont.):
ILR PROCESSING RECORD - SEGMENT 3, FIXED LENGTH DATA ELEMENTS

PROGRAMSYMBOLDATA
ELEMENTLICONPUB

Conference
Publication
Indicator

16

Main Entry
In Body

17

LiteraryGroup

18

LIMEBODY

LILITGRPCHAR.

POSITION

CONTENTS, REMARKS, ETC.

'0' = no'1' = yes.

'0' = no'1

= yes

a - complete/collected works
b - selected works

c - prolific writernone

Cancel Title
Added Entry
in Dict. Cat.

19

LICNTRYCountry of
Publication

20-22

All blanks. Not currently
implemented.

L2ILLUS

Illustration
Codes

23-26

One to four characters, leftjustified with blank fill. Set
by scan of Collation Input Field.

LICNCELT'0' = no, don't cancel'1' = yes, cancel in dictionary
catalog; make for div.

a - illus.

b - mapc - portraitsd - chartse - plansf - plates
g - musici - coats of arms

j - genealogical tables
k - forms
1 - diagrams[End of Fixed Length Data Elements]
-175-

H.

VARIABLE FIELDS

The variable fields include all the standard bibliographic data
elements defined for monograph catalog records, plus various control
numbers pertaining to catalog records, such as the LC Card No. (The
Fixed Length Data Elements Field, Tag 000, although not strictly avariable field, may vary in length from one kind of record to another.
Because of its complexity, it was defined in detail above, in Part G.)

The variable fields contain data which by its nature is variableboth in presence and in length. The fields are packed into the lastportion of the storage record with no intervening gaps.

Currently, the individual variable field begins with two specialpurpose character positions, Indicators 6 and 7, which are set to zero.
These are provided for future use. It is intended to use them tocontrol diacritical marks and similar special characters occurring in
the field, and to control the applicability of the field to a given
library on the basis of set-inclusion or exclusion codes.*The principal identifying codes (tags, and indicators 1-5) fora given variable field are, as indicated previously, stored in the
Directory Entry for the field, along with the field length and addressof its starting character position.
A list of tagged fields and the data elements contained in eachis presented in Fig. 7, below.
A list of the meanings of Indicator 1 which are currently
implemented, is presented in Fig. 8.
This does not exhaust the coding supplied in the storage record.
There is a third and last component, sub-field identification. Thiscode, called a delimiter, serves to identify and describe particular
data elements which may be contained in the tagged fields.

Delimiters are currently stored interspersed with the data inposition ahead of the sub-field they identify. It is not certain
whether this method will be retained in future versions of the storage
Currently a "%" is stored as the delimiter symbol in storage
format.
records produced by INFOCAL. A chart listing the present contents ofthe delimited sub-fields in each tagged field is presented in Fig. 9.

*See Part J.
-176-

FIGURE 7:
VARIABLE FIELD TAGS AND DATA ELEMENTS(as of May 6, 1968)

afi Variable Field Data Element

Iss.

Variable Field Data Element

110

CORPORATE NAME

111

CONFERENCE OR MEETING

112

FIRM NAME***

118

TITLE SUBHEADING120

CORPORATE NAME WITH FORM
SUBHEADING128

TITLE SUBHEADING

130

UNIFORM TITLE HEADING131

ANONYMOUS CLASSIC HEADING***

138

TITLE SUBHEADINGCONTROL FIELDS000

FIXED LENGTH DATA ELEMENTS
FIELD

001

CONTROL NUMBER (LC CARD NO.)

002+
LEGEND EXTENSION

003

LANGUAGESCONTROL NUMBERS
010

LC CARD NUMBER*

011

NATIONAL BIBLIOGRAPHYNUMBER

012

STANDARD BOOK NUMBER**013

PL 480 NUMBER

014

SEARCH CODE+

SUPPLIED TITLES
200

UNIFORM TITLE

LOCAL SYSTEM NUMBER+

210

ROMANIZED TITLEKNOWLEDGE NUMBERS

220

TRANSLATED TITLE**

+
019

020

BNB CLASSIFICATION NUMBER**

030

DEWEY DECIMAL CLASSIFICATIONNUMBER

050

LC CALL NUMBER

051

COPY STATEMENT (LC CARD)

052

CATALOGING SOURCE***

260

PLACE

060

NLM CALL NUMBER

261

PUBLISHER

070

NAL CALL NUMBER**

262

DATE(S)

071

NAL SUBJECT CATEGORY NUMBER300

COLLATION

080

UDC NUMBER**

350

BIBLIOGRAPHIC PRICE**

090

LOCAL CALL NUMBER (HOLDINGS)360

CONVERTED PRICE+

091

COPY STATEMENT(LOCAL CARD)***

TITLE PARAGRAPH
240

TITLE STATEMENT250

EDITION STATEMENTINPRINTSERIES NOTES400

PERSONAL NAME (TRACED THE
SANE )

MAIN ENTRY
100

PERSONAL NAME

408

TITLE SUBHEADING108

TITLE SUBHEADING410

CORPORATE NAME (TRACED THE
SAME)

-177-

(Continued on next page)FIGURE 7 (Cont:):
VARIABLE FIELD TAGS AND DATA ELEMENTS(as of May 6, 1968)

Ns.

Variable Field Data Element

Tag

Variable Field Data Element

411

CONFERENCE (TRACED THE SAME)

650

TOPICAL

412

FIRM NAME***

651

GEOGRAPHIC NAMES

418

TITLE SUBHEADING

652

440

TITLE (TRACED THE SAME)POLITICAL JURISDICTION ALONE
OR WITH SUBJECT SUBDIVISIONSSERIES UNTRACED OR TRACED
DIFFERENTLY

653

490

PROPER NAMES NOT CAPABLE OF
AUTHORSHIP

655

GENERAL SUBDIVISIONS (OTHER
THAN PERIOD AND PLACE)

BIBLIOGRAPHIC NOTES
500

BIBLIOGRAPHY NOTE

656

PERIOD SUBDIVISION

510

DISSERTATION NOTE

657

PLACE SUBDIVISION520

CONTENTS NOTE (FORMATTED)660

NLM SUBJECT HEADINGS (MESH)

530"BOUND WITH" NOTE

661

TOPICAL MESH SUBHEADINGS540"LIMITED USE" NOTE

662

GEOGRAPHIC MESH SUBHEADINGS

550

GENERAL NOTES (ALL OTHERS)

663

TIME PERIOD MESH SUBHEADINGS

560

ABSTRACT**664

FORM MESH SUBHEADINGS

570"IN ANALYTIC" NOTE***

670

580

"FULL NAME" NOTES***NAL AGRICULTURAL/BIOLOGICALVOCABULARY**

690+
LOCAL SUBJECT HEADING SYSTEMSSUBJECT ADDED ENTRY
600

PERSONAL NAME

608

TITLE SUBHEADING610

CORPORATE NAME

611

CONFERENCE OR MEETING

612

FIRM NAME***

618

TITLE SUBHEADING

620

CORPORATE NAME WITH FORM
SUBHEADINGOTHER ADDED ENTRIES

628

TITLE SUBHEADING

630

UNIFORM TITLE HEADING

631

ANONYMOUS CLASSIC HEADING***

638

TITLE SUBHEADING640

BOOK TITLE AS SUBJECT***700

PERSONAL NAME

708

TITLE SUBHEADING

710

CORPORATE NAME

711

CONFERENCE OR MEETING

712

FIRM NAME***

718

TITLE SUBHEADING

720

CORPORATE NAME WITH FORM
SUBHEADING

728

TITLE SUBHEADING

730

UNIFORM TITLE HEADING731

ANONYMOUS CLASSIC HEADING***

738

TITLE SUBHEADING(Continued on next page)-178-

FIGURE 7 (Cont.):
VARIABLE FIELD TAGS AND DATA ELEMENTS(as of May 6, 1968)

as. Variable Field Data Element
740

TITLE TRACED DIFFERENTLY

753

PROPER NAMES NOT CAPABLE OF
AUTHORSHIP

INz.

Variable Field Data Element

SERIES ADDED ENTRIES800PERSONAL NAME

808

TITLE SUBHEADING

810

CORPORATE NAME

811

CONFERENCE OR MEETING

812

FIRM NAME***

818

TITLE SUBHEADING

840

TITLE-ONLY SERIES HEADING

900

BLOCK OF 100 NUMBERS FOR
LOCAL USE+NOTES:

optional local use.
* An alternate code defined in LC MARC II for
ILR will use Tag 001 for LC Card Nos. available to it, thus
maintaining compatibility with the anticipated LC tape distribution
service.

ILR will supply this data** Provided in LC MARC II for future use.
available on thein its records only if the information is alreadyLC cards used as input records at the time of original conversion.
*** A local code or data element specified for original
ILR, but not defined in LC MARC II.

conversions by

Code and data elements not currently implemented by ILR, but
defined in LC MARC II.

+ An alternate code defined in LC MARC II for optior.-1 local uoe.
number in
ILR will place its own locally-generated master record
positions 40-46 of the storage record Leader.
-179-

FIGURE 8:
VALUES FOR INDICATOR 1 IN APPLICABLE FIELDS

Indicator

and

Value

Tag

Field

003

LANGUAGES

Single or Multilanguage
Translation

0
1

050

LC CALL NUMBER

Book in LC
Book Not in LC

0
1

100

MAIN ENTRY(Personal Name)

Main Entry Not
Subject

Main Entry is
Subject

Forename 0
Single Surname 1
Multiple Surname 2Name of Family 3

4Surname (Inverted) 0
Place + Name 1
Name (direct Order) 2

4

5

6
7

110
111

(Corporate Name)(Conference)

112
120

(Firm Name)(Corporate Name with
Form Subheading)

0

1

130

(Uniform Title
Heading)

0

1

5

6

.....____

200

UNIFORM TITLE

Not Printed on LC Cards
Printed on LC cards

1

210

ROMANIZED TITLE

No Title Added Entry
Title Added Entry

0
1

240

TITLE STATEMENT

No Title Added EntrY
Title Added Entry

0
1

261

PUBLISHER

Publisher Not Main Entry
Publisher is Main Entry

0
1

400

SERIES NOTE

Author Not Marti

Entry(Personal ,Author/
Title)

Forename 0
Single Surname 1
Multiple Surname 2Name of Family 3+This value not presently implemented.
-180-

Author is Main
Entry
4

5

6
7

(Continued on next page)FIGURE 8 (Cont.):
VALUES FOR INDICATOR 1 IN APPLICABLE FIELDS

Tag

Field

410
411

(Corporate Author/
Title)(Conference/Title)

412(Firm Name)

490(Series Untraced orTraced Differently)

600

610
611

Indicator

Author Not Main
Entry

14

0

1

Series Not Traced
Series Traced Differently

0(Corporate)(Conference)

Surname (inverted)
Place + Name
Name (direct order)

6

1
2
3

0

1
2

(Personal)

Corporate)

1

Alternative Secondary Analytical
Forename
Single Surname
Multiple Surname
Name of Family

710
711

5

0

(Personal)

OTHER ADDED ENTRIES
700

Author is Main
Entry

Surname (inverted) 0
Place + Name 1
Name (direct order) 2

Forename
Single Surname
Multiple Surname
Name of Family

SUBJECT ADDED ENTRIES

and Value

Surname (inverted)
Place + Name
Name (directorder)

H

B

D
E
F

C

G

@

H

A

D
E

B

F

J

@

A

I

J
K

I

712
720

(Firm Name)(Corporate with Form
Subheading)

0

1

2

730
731

(Uniform Title
Heading)

0

1

2

740

(Title Traced
Differently)

0

1

2

-181-

FIGURE 8 (Cont.):
VALUES FOR INDICATOR 1 IN APPLICABLE FIELDS

R.1

800

810
83.1

SERIES ADDED ENTRIES(Personal Author/
Title)(Corporate Author/
Title)

Forename
Single Surname
Multiple Surname
Name of Family0Surname (inverted)
Place + Name
Name (direct order)

0
1
2

1
2
3

I.

SUB-FIELD DELIMITER CODESIn the current version of the INFOCAL program, most sub-fieldeach sub-field beyond
delimiters have a form of coding as follows:
the first is signified by a combination of one or more per cent (%)
symbols in the storage record. That is, the sub-field coding is
positional rather than explicit. Two %'s followed by a data string,for example, means the first and second sub-fields in the given field
are vacant.
DATA PRESENT:

1st Sub-field data%2d Sub-field data%3d Sub-field data

DATA ABSENT:%%3d Sub-field data

Because this area is the most volatile one in terms of changes
in the form of the coding in the MARC II record designs both thepresently implemented INFOCAL coding and the latest known MARC II
coding are presented, for comparative purposes. This will assist
retrofit of INFOCAL in the future.
It should be noted that changes in the coding imply no change
in the kind or amount of bibliographic data included in the record.
The changes relate to
Full data will be included in all cases.
the method of coding and the amount of detail in data element
definition. Since the changes in sub-field definition affect fieldtags, the programmer should consult the latest MARC II record specifications issued by the Library of Congress before proceeding with any(Certain of the datachanges to the coding produced by INFOCAL.
elements currently defined in INFOCAL as tagged fields are redefinedin the new MARC II record as delimited sub-fields. Also, certain
data elements have been defined as sub-fields by MARC II which have
not yet been implemented in INFOCAL).

The revised system of sub-field coding announced by the LibrarYof Congress, as currently understood, is as follows:
In a field in which only one data element has been defined(e.g., Standard Book Number), the field will begin with the delimiter
The same data element may be repeated in a field as manycode "$a".
times as necessary by preceding it with its identifying delimiter.
For example, there may arise a need to assign more than one StandardBook Number to a single catalog record.
Only fields which can contain more than one kind of defined
data element are listed below.

-183-

FIGURE 9:
SUB-FIELD DELIMITER CODES
NOTE:The word "None" in this table means that the data element
was not defined by ILR for sub-field coding purposes at
the time the INFOCAL program was implemented.

Sub-field Code in
INFOCAL Storage
RecordRevised MARC II
CodeTAG, FIELD NAME, AND SUB-FIELDS

003
None

$a

%

$b

Languages

The group of 3-character language
codes needed to describe thelanguages of the text or its
translation
Summaries
Example:$aengfre$bgerrus

050

060
070
None
None

None
None
%

Class NumberBook Number

$a
$b

051

LC Copy Statement

091

Copy Statement (LOCAL CARD)Class NumberBook Number
Copy Information

$a
tb
$c

(Not defined in
MARC II)

052 1Cataloging Source
Name of Library Contributing Cat.
CopyClass No./Call No. (when present)

None
%

090
None
%
%
%

LC, NLM, and NAL Call
Numbers

Local Liinely Holdings

Call Number
Copy Number$a(Not def. in
MARC)
$b

Library Code
No. of Copies$c(Continued on next page)-184-

FIGURE 9 (Cont.):
SUB-FIELD DELIMITER CODES

Sub-Field Code in
INFOCAL Storage
RecordRevised MARC II
CodeI

TAG, FIELD NAME, AND SUB-FIELDS

Wo

11
600

Personal Name

700
800

None
None

$a
$b

%
%
%

$c

None
Tag*
None
%

$d
$e
$k
$t
$u
$v

Name
Numeration
Titles and other words
Dates
RelatorForm Sub-heading
Title (of book or title series)
Filing Information (in 700 only)
Volume or No. (in 400/800 only)
110

410
610

710
810

%

$a
$b

#

$k

Tag
None

$t

None

%

$u
$v

Corporate Name(also for 112/412/612/
712/812 which are ILR
tags for firm names)(also for 120/620/720,
Corp. Name with Form
Subheading)Name
Each subordinate unit inhierarchy
Form Sub-Inading
Title (of book or title of series)
Filing Information (in 710 only)
Vol. or No. (in 410/810 only)
11:1

411

611

Conferen,e or Meeting

711
811J

None

$a

Name

*The word "Tag" here means that the data element is identified in thecurrent version of INFOCAL as a tagged field. See Fig. 7.
-185-

FIGURE 9 (Cont.):
SUB-FIELD DELIMITER CODES

Code in INFOCAL
Storage Record
%
%
%

None
None
#

Tag*
None
%

Revised MARC II
Code
$b
$c

$d
$e
$g
$k
$t

$u
$v

TAG, FIELD NAME, AND SUB-FIELDS
Number
Place
Date
Subordinate unit in Name
Other information in the heading
Form Sub-heading
Title(of book or title of series)
Filing Information (in 711 only)
Vol. or No. (in 411/811 only)
130

630

730
None
Tag
None

$a
$t

$u

$e

%
%

$b
$c

%

$a
$b

$a
$b
$c

Edition Statement

Edition
Additional information after
edition
260

Tag
Tag.
Tag

Title Page Title

Short title from which added
entry is madeRemainder of title
Remainder of title page transcription
250

None

(also for 131/631/731,
Anon. Classic heading an ILR tag)

Uniform title (Main portion)
Title (of part)
Filing Information (73O only)
2451

None

Uniform Title Heading

Imprint

Place
Publisher
Date
300

Collation Statement

None
%

$a
$b

Pagination
Illustrative Matter

%

$c

Size

None

$a

Thickness*The word "Tag" here means that the data element is identified in thecurrent version of INFOCAL as a tagged field. See Fig. 7.
-186-

FIGURE 9 (Cont.):
SUB-FIELD DELIMITER CODES

Sub-Field Code in
INFOCAL Storage
RecordRevised MARC II
CodeTAG, FIELD NAME, AND SUB-FIELDS

Series Note Traced Same

44o

(Title)

None
%

$a
$v

Title Portion
Vol. or No.
650

651

Subject Added Entries

652

653
654

None
Tag*

$a
$x

Tag

br

Tag

$z

Tag

$t

Main Subject Heading (650-654 only)
General Subject
Subdivisions
1
Period Subject
kalso 600/
Subdivisions
610/611/612
Place Subject
and 630)
Subdivisions
Title portion of a
subject heading
Series Traced Differently

840(Title)
1

None
%

$a
$v

Title of seriesVol. or No.

*The word "Tag" here means that the data element is identified in the
See Fig. 7.
current version of INFOCAL as a tagged field.

-187-

J.

ADDITIONAL VARIABLE FIELD FEATURES

Two character positions (Indicators 6 and 7) have been placed at
the head of each variable field occurring in the record, beginning with
Tag 002, Legend Extension.
These indicators serve as the nucleus for expansion of the leading
portion of the variable field to allow encoding of three types of
information which can be applied to the field when it is prepared for
output display: the alphabet of the field, location-specific information,
and a list of diacritical marks. A proposed encoding sequence is shownin Fig. 10. (These features have not been implemented in INFOCAL).
FIGURE 10:
PROPOSED VARIABLE FIELD HEADER

START OF FIELD
alphabet
code

IND. 6

IND. 7DATA

lucationspecific codes

diacritics
list

variable fieldcontent

Alphabet Code. A flag to shift the encoding of the field fromthe English alphabet to some other character set such as Cyrillic. It
would specify both the fact of the shift and the particular encodedalphabet. An unresolved problem is that a deshift back to English or
to a third alphabet can occur within the field. The flag must be kept
distinct from the first byte of the location-specific information. The
alphabet code could be omitted in the default case of "all-English."1.

Location-S ecific Codes. Indicator 6 has been reserved as aflag to show whether a given field content is desired by a given
For example, the record might
library to which the record applies.
contain both LC subject headings and Medical Subject Headings. Library
A might want only the LC headings; library B only the MeSH headings.
This Indicator will facilitate the composition and display of the record
according to each library's specifications. Four configurations are
possible:
If the field is wanted by all locations, the indicator is to bea.
set to F016 (currently, perhaps it should be 00). This would be thedefault case. The next character in sequence is the diacritics indicator(Ind. 7).
If all but certain locations are to be provided with the field,b.
the indicator will be set to a stacked code of 4016 plus the number oflocation codes which follow. Location codes are set as three-digit
EBCDIC numbers, following Indicator 6 and preceding Indicator 7.
If only certain locations are to be provided with the field,c.
the indicator will be set to a stacked code of 8016 plus the number oflocation codes which follow.
2.

-188-

If none of the libraries are to be supplied with the field, a
code of 0016 is set. This will be rarely used, and will perhaps apply
to information used only by the processor of the records--e.g., the ILR
master record number, if stored in a variable field, would not conceivably
by desired as a "print" field by any libraries to which the record applies.
d.

The location-specific code will always be present.

Overprnted Character Coding. Indicator 7 is the nucleus for a
proposed method of storing diacritical marks and any other characterswhich are overprinted and thus require special processing, or whichmight well be omitted from certain kinds of displays, e.g., CRT's. The
major objective of the code will be to preclude a character-by-characterscan of the field content to determine if overprinted characters exist.
Rather a simple test on the indicator will yield the desired information.
The text string in the field will thus not be interrupted by special
codes, and no special processing will be necessary should it be desired
to bring the field out for display without the overprinted characters.
3.

The proposed technique is as follows:
Each variable field will have a one-byte signal in Indicator 7,
which will always be present. It will be followed by 0-n sub-fields,
each containing a value corresponding to a diacritical present in thefield, and a pointer (a relative character position).
a.

If the signal is set to zero, there are no special characters,
and the variable field data is free of special handling. Data starts in
Field + 2.
b.

IND. 6

IND. 70

Data

If the signal is set to an even-numbered multiple n (n=2-4254)
there are n/2 diacriticals in the field's data. The signal will be set
to the relative character position in the field of the first datacharacter. The signal will be followed by one or more sub-fields which
will show the value of the diacritic together with a pointer to the datacharacter to which the diacritic is to be applied. (The value comes
first so that the program can be set up to determine whether or not an
actual operation on the data should be subsequently performed.) The
second pointer is counted from the position of the text character
to which the preceding diacritic was applied. Etc.
c.

n!Value 1

Pointer I

Value 2

Pointer 2

Data ...

E.g., 41'3'2 Therese ...

If the field length exceeds 255, and one or more diacritics
occurs at a position beyond the 255th relative position, the signal
will be set as above, but the first (or subsequent) sub-field will be
set to blank as a Value, and 255 as a Pointer to serve as a displacemev.
d.

-189-

The next sequential subfield will show the relative position
beyond 255 of the character to which the next diacritic applies.
value.

Value 1
4

K.

255

)6

_Data ...

Value 2 15

EXAMPLE OF STORAGE RECORDLEADER065n)45771 146$$$

03 00)50

3$ 0100011dS 1

50468

LILENGTHDISDATELISTATUS-.

LILEGCNT

LITYPE--LIBLEVEL----LIINDCNTLIBASELIORIGIN------LIPDATE.

LIPROSORLISOURCE-LIAGENCY
LIAD
LINUMBER
LICHECKLIDIRLLIDEOF

DIRECTORY
Entry

Entry 2

Entry 3

,-

00

SS

n 0031

...

090

1001

2400

TAG--]

IND 1--IND 2

IND 3,4,5-FIELD LENGTH (16-BIT
START. CHAR. POS. (16-BIT

260

2610

262

300

4901

550

65001

65002

7411

812

END OF DIRECTORY (CHART CONT/D NEXT PAGE)-190-

FIXED LENGTH DATA ELEMENTS FIELDISTART OF VARIABLE FIELDS

----,

1

si96

$$

$4

4

Sitlo

sengP

am 64 -301%435x01

LIDT
LIDATE1
LIDATE2
LIMICROR
LIFORM
LIGOVPUB
LICONPU
LIMEBODY
LILITGRP
LICNCELT
LICNTRY

LULUS

Indicators 5 and 6 for Tag 003 - Languages
Data for Tag 003 - Languages code
Data for Tag 090 - Local Library Call No. & holdings info.

FGarmash , V . A . 00Quantizationof signals with non-unifo

Data for Tag 100.1°.--"1----1.--Ilthos=.
Data for Tag 240 - Title0
rm steps. 0 °Redondo Beach, Calif.,

W Space Technolog4

Data for Tag 260 - Place of Pub...I
Data for Tag 261 - Publisher

Library.$T
Laboratories 0011964. 006 1.02800STL Technical
Data for Tag 262 - Dat
Data for Tag 300 - Collation
Data for Tag 490 - Series Note

Data for Tag 550 - Notc

ineravnomernym shagom(CHART CONT'D NEXT PAGE)-191-

from Elektrosvyaz, no. 10, p. 10-12, 1957 00Information

Data for Tag 650 - Subject Heading (1st)illeasurement.00Signals and signaling.00ElektrosvyazAv. 101

Data for Tag 650 - Subject Heading
Data for Tag 74 - Added Entry for Periodical

ip. 10-12, 1957.00Space Technology Laboratories, Inc, Lol

Data for Tag 812 - Series-Traced-Differently Added Entry

isAngeles.%Technical LibrarY 0

ranslation,%no. 80

END OF RECORD
Data for Tag 818 - Series title

The catalog card shown below was used as the source for thepreceding example of a storage record.

i*Eng.-N4th.

Sciences
A
Garmash, V
Pam
Quantization of signals vith non-uniform
64-3
steps. Redondo Beach, Calif., TRW Space Technology Laboratories, 1964.
61. 28 cm. (STL Technical Library. Translation, no. 80)

Translation of nantovanie signalov s neravnomernym shagom from Elektrosvyaz, no. 10, p.
10-12, 1957.

UCLA

1. Information measurement. 2. Signals andsignalling. I. Elektromaz, v.10, p, 104-12,
1957. /I. Title. (Series: Space Technology
Laboratories, Inc.°, Los Angeles. Technicalion, no. 80)
Library. Translat-192-

APPENDIX V

SUMMARY OF RECORD FORMATS FOR DATA BASES TO BE CONVERTEDTO ILR PROCESSING RECORD FORMAT

1.

Santa Cruz Record Format

2.

ILR Input Record Format

3.

Experimental On-line Mathematics
Citation Data BaseSANTA CRUZ RECORD FORMATFIGURE 1:
SAMPLE CATALOG RECORD IN ORIGINAL SANTA CRUZ FORMAT
V - 1.

1961

1Q_AE111._H68_HOCKING, _JOHN _GILBERT, 1920
YOUNG, _GAIL _S_TOPOLOGY, BY _JOHN _G. _HOCKING A_READING, _MASS. _ADDISONJLSL1Y
ILLUS< ADDISON WESLEY SERIES IN MATHEMTOPOLOGY
3141

I
Col.

Card Type No.4

ND _GAIL _S. _YOUNG
_CO."

(

_INCLUDES i3IBLIOGRAPEI
6o16910I 801

135

70-72

punched in the
The above is an example of a catalog record
local input format devised at the University of Santa Cruz
card-image file, the converLibrary in 1965. From this kind ofcard images resion program being written by ILR will produce
formatted into the ILR standard input format for subsequent
processing by the program entitled "INFOCAL." The INFOCAL output will be the same record in ILR Storage Format (see Appendix
The latter record is the one that is convertible to andIV).
by the Libraryfrom the MARC II communications format devisedof Congress.

the card forThe following pages constitute a summary of
The information
mats for each card type in the Santa Cruz record.
Specification
has been excerpted from: Computer Usage Co., Inc.
Prepared for University of
Library
California at Santa Cruz. Palo Alto, Calif., April, 1965.
pp. 51-60.

,°-195-SHELF KEY CARD - CARD TYPE NO. 000

A.

FROM
FIELD ID

COL

TO
COL

Call number

1

24

Series25

Volume26

28

Blank or 1 - 999, Left justifyPart number

29

30

Blank or 1 - 99, Left justify

Year

31

34

Year of publication'Donor number

35

38

Blank or Donor number from Gift list

Date rec'd.

3942Month and year received

Location

43

44

Alpha code designating Campus or Library

Type Code45

C OMMENTS

Blank, Alpha, or Numeric

0 = book 2 = reference 4 = see author
1 = serial 3 = govt. pub. 5 = see subj.
6 = see also subj. o'pch'-' = So. Pacific

Language
Codes

46

50

Up to 5 Alpha Language codes, or 4 codes
followed by 1.0

Pages

51

55

Left justify

Correct/
DeleteCard No.

Delete old records if 'C' or
SUbstitute new records if 'C'

56

57

69

- unused -

70

72

'000'

73

75

Currently unused, but available as partof Accession number field

76

80

Accession Number
-196-

B.

PERSONAL AUTHOR - CARD TYPE NO. 100-104

Limit:

COL

TO
COL

1

60

name of author - left justified

68- unused -FROM
FIELD ID

Author0 - 5 Cards

C OMMENTS- see page following corporate author
card -

SpecialCode

69

Card No.

70

72

100 tfiru 104

Accession No.

73

80

same as Shelf Key card

Note:

The 1st Author to be processed by thecomputer is considered the main author.
The Main Author appears on all catalogs
and is represented in the title by "**".

C.

CORPORATE AUTHOR - CARD TYPE NO. 110-119

Limit:FIELD ID
Corporate
Author1 or 2 cards per author/ 0 - 5 Authors

FROM
COL

TO
COL

COMMENTS

1

60

May be continued on second card67

- unused -is - if author is continued on

Continuation
Indicator

68

SpecialCode

69

Card No.

70

12110 - 119

Accession No.

73

80

same as Shelf Key Card-'second card

- see next page -See note concerning main author on
Personal Author card.

-198-

SPECIAL CODES FOR AUTHORSType 1:

Type 2:

To create added notation on Author Catalog

NOTATIONCODEMEANINGJ

Joint Author

joint auth.

C

Compilercamp.

E

Editored.

G

Joint Editor

joint ed.

I

Illustrator

illus.

P

Publisher

pUbl.

T

Translatortrans.

To specify a substitute sort key.

X

Use this author as a substitute sort key
for previous author. Previous author
will appear on appropriate catalog but
this author will not.

D.

TITLE CARD - CARD TYPE NO. 200-224

Limits:1 to 5 cards per title/ 0 - 5 Titles

1

FIELD ID
Title

FROM
COL

TO
COL

1

60

C OMMENTS
may be continued on up to 4 additional
cards

67

- unused -Continuation
Indicator

68

Special Code69

Card No.

7072

200-224

Accession No.

73

80

same as Shelf Key cardis '' if continued on next cardsee next page -SPECIAL CODES FOR TITLES

CODE(blank)MEANING
Suppress listing this title in TITLE catalog
Title is a transliterated title

Title is a series title
Partial title,

Standard title

This title is to be listed in TITLE catalog

Note:

In all cases, the 1st title encountered when processing
a given entry will be the only title which appears in
the Shelf, Author, and Subject Catalog.

-201-

E.

PUBLISHER/SOURCE CARD - CARD TYPE NO. 300-305

Limit:

1 - 2 cards per publisher/source/3 pUbl./sources total

FIELD ID

FROM
COL

TO
COL

1

60

may be continued on a second card67

- unused -C OMMENTS

rublishersiourceis '-' if publ./source is continued
on next carC

Continuation
Indicator

68

SpecialCode

69

Card No.

7072

300-305

Accession No.

73

80

same as Shelf Key card

= publisher

S = source

F.

COLLATION CARD - CARD TYPE NO. 400

FIELD ID
Collation

FROM
COL

TO
COL

1

60

as desired

69- unused -C OMMENTS

Card No.

70

72

400 (1 card only)

Accession No.

73

80

same as Shelf Key card

-203-

G.

COMMENTARY CARD - CARD TYPE NO. 500-509

Limit:

1 - 5 cards per comment/2 commentaries (1 of each type)

FIELD ID

FROM
COL

TO
COL

Commentary1

6o

may be continued on up to 4 more cards

61

67- unused -C OMMENTS

is '-' if commentary continued on
ncxt card

Continuation
Indicator

68*pecial Code69

Card No.

70

72

500-509,ccession No.

73

80

same as Shelf Key card

Note:

'S' for commentary to appear on shelflist only

If 2 commentary entries are used, atleast one must have an 8S1 code.

-2014-H.

SUBJECT CARD - CARD TYPE NO. 600-6o4

Limit:FIELD ID
Subject

1 card per subject/5 subjects

HCOOM
L

TO
COL

1

6o

as desired

61

68- unused -C OMMENTS

may be used to indicate level of subject*

Special Code69

Card No.

70

72

600-604

Accession No.

73

80

same as Shelf Key card*the special code is ignored by thesystem at present time.

-205-

V-2.

ILR INPUT RECORD FORMAT
By

Jay L. Cunningham
Institute of Library Research
University of California
Berkeley, California
A.

INTRODUCTION

In this report, a specification for a bibliographic data
input record is presented. It was developed for use in converting
catalog cards for monographs on an original basis - i.e., when no
machine record is found to exist or expected to be obtainableelsewhere.* Because the format is experimental, the specification
1) the preliminary version, which has
is presented in two parts:
been programmed and is operational in a prototype production fortest and evaluation purposes, and 2) suggestions for a revised
and improved Irersion which has not been programmed but which isbased on the experience gained from the initial model.

The principal conclusions drawn from the evaluation of the
prototype input format for this project are that a record of thecomplexity and variability of the MARC II record (to which the
It isinput record is convertible) is very costly to prepare.
difficult to recruit and retain personnel of the quality appropriate to Ithis work yet suited to its semi-clerical nature. Preliminary estimates are that average editing time per record will
range from about 2-1/2 minutes to 4 minutes per man-record. The
coding required by the MARC II format is the principal reason
for these figures. That is quite aside from the unavailability
at this time of an editing manual from the Library of Congress,
and the fact that the MARC II design is itself still developmental(with the result that the instructions for editors are continuallysubject to revision). The reasons lie elsewhere: in the illstructured nature of cataloging itself, in the ambiguities of
the rules and the consequent ambiguities built into MARC (which
is based on the rules), and the variance over the years of theserules and the reflection of this in catalog cards. Retrospective
cards in particular have caused considerable delay in the project.
Adjustments in the coding prescribed by MARC (which was aimed atcurrent and future records) have repeatedly been necessary whenolder records are to be converted.
Nevertheless, these problems comprise a small percentage ofthe records. Therefore, the source of improvement does not seem

*The writer is indebted for the great deal of groundwork on thisrecord design done by Kelley Cartwright, formerly of the Institutestaff at Berkeley, and Miss Martha Bovee, of the Technical Services
Staff, University Library, University of California, San Diego.

ozof

20T-

to lie in a reduction of the MARC design to a less complex level.
Rather the approach has been to utilize the MARC data definition
to the fullest degree, in tha expectation that the residue ofcoding interpretation problems will disappear as the format is"debugged." Only the form of coding has been altered in our input
format.

We are employing a more concise notation in the coding,
exploiting the concept of "default" settings, and building computer
algorithms for the recognition and identification of the data
elements in catalog records. The improved version of the input
format extends this approach, and reflects a number of recommendations that we now feel confident can be implemented by program, asa replacement for a certain amount of manual editing effort.
These program routines would operate either by simple key word
matching or by more complex data recognition algorithns which
process clues imbedded in the fields. The new format takes advantage of a minimum of human-applied codes that serve as bcundary
markers for the operation of the algorithms. Where possible
these markers would be set by actions such as paragraph indention
symbols keyed by the device operator without detriment to a normal
typing rhythm.
B.

NATURE OF THE MACHINE RECORD

AB implied above, the input record format isGeneral.
compatible with the MARC II Communications format. Strietly speaking, the input record is converted in our system to the ILR Processing format. The latter is convertible to and from MARC II.
1.

For all of these record formats, however, the machine record
organization can be summarized as follows: the two primary building blocks are data elements and codes. The data elements andcodes are organized into the components shown in Fig. 1. The
field is the basic component for processing purposes.

FIGURE 1:

STORAGE RECORD COMPONENTS AND ORGANIZATION

Record Component

Content of Component

Type of Code Identifying
the Component

field

data element (groupof one or more)

tag+indicators

sub-field

data element

delimiter-208-

It is important to understand the reasons for the organization of
the record into these components, in order to see why the input
format while identical in content, is different in structure andcoding from the other formats.
The MARC record coding is intended to serve four basic functions:
1.
2.
3.
4s

printing
sorting
information retrieval
catalog division

In the simplest design to serve these functions each data elementin the computer record could have been defined as occupying its
own field. Each field would have an identifying code or "tag."
Due to the desire to use a record directory technique in theProcessing and Communications Formats, the use of a field tag foreach value of an element occurring in a given record becomesinefficient, since each tag is stored in a directory entry of 12(This constraint does not apply to thecharacters in length.
input record, since it does not have a directory of its own
contents.)

Moreover, the data elements in many cases naturally group inclusters that will respond to the functions mentioned above. Many
elements are both derived and manipulated in combination; forinstance, the Imprint field comprises place, publisher and dateof publication. It is thus logical to group more than one element
in a field. There is still the requirement to identify individual
elements, however, so a form of sub-field identification, lessclumsy than the tag and its directory information was created.
These are called delimiters and are embedded in the variable data.
(The tag is relegated to a separate directory section of thelogical record).
The ILR input format takes no intentional cognizance of theIts only requirement is to uniquely but concisely
four functions.
identify each element so that it may be transformed into anothercode which itself is more efficient for either internal processing
or communications. It acts only as a one-directional data conversion vehicle from source to machine edit program. As a consequence the codes used in the input format must be adjusted tothe structural patterns - options, alternations, repetitions - incatalog data in different ways than the processing format codes.
These field patterns arise from the ways the values forbibliographic data elements are organized in a given catalog
record:Single occurrence of a value for a single element field,e.g., LC Card Number;
Repeated valv.es for a single element, e.g. 5 several Dewey
numbers on one card;-209-

Sequences of values for two or more different data
elements, e.g., places publisher, date;
Variability in presence of values in these sequences,e.g place, date;
Combinations of the above, e.g., place, place, publisher,date, date; place, publisher, date, place, publisher,date.

In the input folmat, convenience in editing demands thatcomplex patterns of elements in dense catalog text be coded as
Input tags for a pattern such as "place,
concisely as possible.
place, publisher, date, date" may be coded as/place,$placedpubli3heradate,date
The slashes identify the data by ordinal position, i.e., theIn the processing
5th slash in an input string is always'place'.
format (MARC II) each of the above symbols might be coded by athree digit tag and generate a directory entry. The MARC design,
which is still evolving, tries to balance the assignment of the
tag level of format definition to a data element by considerationof such factors as:
Is the data element processed frequently?

If so, is it searched for or manipulated independently
or in combination with other elements?
What is the average length of thz values of the data
element?
How often do sequences of repeated values of the element
occur?
Does it occur in every record?

What is its format function (does it convey information
about the document, about the record, abcut the file oreven about other fields or codes in the record)?
Fig. 2 shows the structural patterns in the revised coding
From this, it is evident that our input recordfor MARC,,records.
represents a more primitive coding. This is acceptable in thisformat because the heavy volume of input symbo1-s4:anning andsyntactical checking is done only once7 when the input compilation is performed. Once the internal record with its apparatus
of field starting position pointers, field length counters, endof-field/record/file signals etc. has been constructed, the user
may process the data in strings and with table-driven routines.

-210-

FIGURE 2:

STRUCTURAL PATTERNS IN MARC RECORD DATA DEFINITION

Single-Element Multiple-Element
Field
Field
SingleValueSingle Mult.
Value Value

Repeatable
Field CodeNon-Repeatable
Field Code

Repeatable
Sub-Field Code

Mult.
Value
1*

Non-Repeatable
Sub-Field Code
Repeatable
Sub-Field Code

2

9

7

5

__,

8

....

3

6

4

.

Non-Repeatable
Sub-Field Code10-

*Entries correspond to.explanations and examples immediately
following the diagram.

STRUCTURAL PATTERNSMultiple Element Field Multiple Values.
1.

2.

3.

Field Repeatable, Sub-field Repeatable.
Example:6XX--Subject Tracings, with Subject Subdivisions.

Form:$aMain Subject Heading$xSubject Subdivision$xSubject Subdivision$xSubject Subdivision.

Field Re eatable

Sub-field Non-Re eatable.

Example:440Tit1e-Only Series Note (Traced Same).

Form:$aSeries Title, $vVolume,Volume,Volume,...
or inclusive numbers, e.g., $vv. 11-15.

Field Non-Repeatable, Sub-field Repeatable.
Example:260--Imprint.

Form:$aPlace$bPublisher$cDate;$aPlace$bPublisher$cDate.
(Dates non-contiguous)-211-

4 Field Non-Repeatable, Sub-field Non-Repeatable.
Example:260--Imprint.

Form:$aPlace$aPlace$bPublisher$cDate,Date.
(Dates contiguous; delimiter not required)

Multi le Element Field
5.

Sin le Value.

Field Repeatable, Sub-field Non-Repeatable.
Example:051--LC Copy Statement.

Form:$aClass No.$bBook No.$cCopy Information.

6. Field Non Re eatable

Sub-Field Non-Re eatable.

Example:Tag 041--Languages.

Form:$aLanguage Codes of Text$bLanguage Codes ofSummaries

Single Element Field, Multiple Values.
7.

8.

Field Repeatab122 Sub-field Non-Re eatable.
Example:490--Series Untraced or Traced Differently.

Form:

490$aXXXX; XXXX.

Field Non-Repeatable, Sub-field Repeatable.
Example:082--Dewey No.

Form:$aXXX$aXXX$a...

Single Element Field
9.

10.

Single Value.

Field Repeatable, Sub-field Non-Repeatable.
Example:500--General Notes.

Form:$aXXXXXXXX.

Field Non-Repeatable, Sub-field Non-Repeatable.
Example:241 - -Romanized Title.

Form:$aXXXXXXXX.

Mapping of Input Codes. To illustrate more precl-ely how
the input format works, it must be remembered that the input codes
do not necessarily bear a one-to-one correspondence to the fieldcodes in the Processing Record. The external fields generally
speaking map into logically identical internal fields, sometimes
on a many-to-one basis (e.g., the place, publisher, and date element's
are separate fields in the input record but map into one internal
field with three sub-fields in the revised MARC II format).
-2122.

Notationally their codes are different, usually in a one-to-many
relation, e.g., the second A-Field slash, with no "ua" code,translates to Tag 100, Indicator value of 1=Main Entry, PersonalName, Single Surname sub-type, Condition: main entry is not
subject. The mapping is complex, that is, input fields and codes
are translated (expanded), re-arranged, concatenated, or split
up to form the internal fields. Fig. 3 is an illustration ofsuch a mapping.
There are four levels of coding in the tag and indicatorof a MARC record. These encompass the aspects of each data
entity needed to serve the four functions mentioned above. In
some cases, the coding has been extended by the device of "stacked"
coding, to serve auxiliary functions such as recording the factthat the main entry heading is identical with the subject of theThis allows the user the option of printing out thebook.
record under the name as a subject heading in a divided catalog.
FIGURE 3:

EXAMPLE OF INPUT FORMAT MAPPING INTO PROCESSING FORMATPROCESSING RECORDINPUT RACORD

External
Field Code

Data Element Valueor Code Value(DIRECTORY ENTRY).

.

11 0 6 0 17S 1/416dddd.

(UACODH - MAIN ENT. TYPE)

(ua

c2)
.

(UBCODE - M.E. IS SUBJ.)(VARIABLE FIELD)

ub
0 0

.

AUTHOR NAME/0/0DATE
.11

.

.

(2ND A-FIELD)(LTHOR NANEMATE
'...
.

.

.

.

-213-

In Fig. 3, the four levels are exemplified as follows:
MARC II
Code Type
Level I - Function of the Field identified
by the code

TAG.

(digit 1)e.g., 2nd A-Field slash = Main
Entry Heading

El XX

Level II - Type of Entity in a Heading (ua)TAG(digits 2-3)

e .g., c = Corporate Body

Level III - Sub-Type of Entity
e .g., 2 = Corporate Boay in formof Direct Order and Role
Relation is Main Entry is
Not Subject ("stacked code")

Level IV - Role Relation of Functional Fatity
e .g., "ub" code in input stream
resets Role Relation instacixd code above to 6 =Main Entry is Subject

l 1E3(Indicator 1)
110(Indicator 1)
110

It should be noted that there is no necessary dependence
between coding in the input format and the processing format. Theinternal code could, for example, be stored in two separate indicators, e.g., Indicators 1 and 2, with no effect on the input
format except to change a table in the input edit program.

As part of this translation process, the edit program must
perform all the bookkeeping actions necessary to assemble therecord in Processing Format (e.g., placing tags in directory
entries, counting field lengths, etc.). .Redundant data or purelyintermediary codes are purged. Default settings and initializations are completed. Computed values are set, either from the
input data (e.g., illustration codes in the Fixed Fields) or bylogical combinations (e.g., a blank 2nd slash with the absence ofa "ua" code means a title main entry iE being input). Charactertranslation (dependent upon the particular keyboard device used)
is made into the representation of the computer.
Some other differences between the input format and theprocessing format are:Repeatability. Certain codes in the input record,e.g., the 10 A-Field slashes, are not repeatable, whereas theircorresponding internal codes are repeatable.
-214-

Presence. Certain Fixed Length Data Elements are input
only when present, whereas the internal record always contains
each of these elements, even if the value is "off", "none", orsuch.

Certain codes in the input format may translate
to a tag in one case and a sub-field in another case, e.g., the
3rd A-Field slash translates to a tag (Title); the fourth slashtranslates to a sub-field delimiter (remainder of title).
Level.

C.

RECORD COMPOSITION

Logical Structure. A complete input record is composedof a stream of data and codes, in-three segments: The Indicatorsegment (I-Fields), the Body of the Catalog Card (A-Fields), and.
the Remaining Data (B-Fields). Each segment is composed of several
The fields are of either fixed or variable-length type,fields.
depending on content. The data elerents are variable or fixed
The codes ar2 correspondingly fixed or variablein presence.
in presence. The codes to the left and below the catalog cardbox on the coding sheet are the I-Fields, in pre-printed form.
1.

The coding sheet* designed.for experimental use in theFile Organization project provides space to record certain infornation which is not explicitly recorded on the catalog card,or for combining in one field the information (e.g., on holdings)
gathered from several cards representing the same title, or torecord information in coded rather than natural form. The I-Fields
either checklist or fill-in. The
are structured in two ways:
codes are pre-printed on the sheet in checklist format as a memory
aid to the editor. The boxes are checked to indicate that theyshould be keyboarded. Codes which are merely checked are of theself-transmitting type - the information they carry is embodied
in a first letter defining the general group of the code; theIn some cases
second letter is a specific value for the code.
there are multiple values assumed by a particular internal data
element (e.g., Content Form may be checked from 0-4 boxes). The
second letter values will be inserted into the proper element in
the Fixed Length Data Elements Field in the Processing Format.

Where the list of options for a code is too lengthy to bepre-printed on the coding sheet, the editor selects a code froma reference list in the coding manual and writes in the proper
values in the box. In Type of Main Entry, "ua" merely identifies
the value which it transmits. An entry could be "uacr meaning"Type of Main Entry = Corporate Name, Direct Order, Main Entryin this record is not subject of the work".

*See Fig. 11, Part III, Final Report of File Organization Project,
Phase I.
-215-

Certain data elements are not explicitly coded: e.g., theMaster Record Number is always placed first in the input record
and is recognized by position.

The 1st section of the coding sheet is that outside of theIt contains boxes:
catalog card image.
to provide values to certain elements in the Record
Leader.
to transmit codes to elements in the Fixed Length
Data Elements Field in the Processing Record.
to provide values for parts of tags and indicatorsthat are not supplied by coding in the other segments of the input record.
to supply data that cannot be recorded on the compact
face of the catalog card image, e.g., call numbers
from other campuses possessing copies of the work.
to record certain control infcrmation, such as
editing and keyboarding statistics.
The codes are alwayz 2 or 4 character combinations, both tofacilitate expansion and for validity checking,In the card image portion

there are two types of coding:A-fields. The body of the source record, the ten fields ordata elements most frequently occurring on the card.

Collateral data which occurs with less preB-fields.
dictability on the cards, such as supplementary notes.
The tables in Section D list the data element names and theircodes in the ILR input format, together with the corresponding
Processing Format codes into which they map.
physical Record. The input record is composed of avariable number of characters placed in contiguous locations on
an input medium, e.g., punched cards. A physical input record
is composed of one unit of the medium used by the keyboarding
device:e.g., a tab card. There might be several tab cards in(See Fig. 4.)
a complete logical record.
2.

Beginning of record.
A master record number isplaced at the beginning of the input record.
For the purposesof the present conversion, it is 6 characters in length.
a.

Continuation segments. If a medium such as tab
cards is used, the record number is repeated in character positions 1-6 in each succeeding card. A serial card number isplaced in c.p. 7-8, right justified, beginning with "01". A
maximum of 15 cards per decklet is currently permitted in the
INFOCAL program.
-216b.

t_

11111,--.7.110(10001110:: _ANCIELES:X_TECHNICAL _LTP.RARY:$_TRAMLATUINIXNO.. 80:+//10001107Z,XV. 10, P. 10-12, 1957.*R_SPACE _TECHNOLOGY -LABORATORIES, _INC., -LOS;ND SIMNALING:1;0_ELiKTOSVYA/ 10001106 19577AM_INARMATION MEASTREMENT7KM_SMAI_S
4
Nog

le

U.

mweggigl 10, P. 10-10

OM

10001105IE SIGNALOV S NERAVNOMERNYM SHAGOM FROM -ELEKTRUSVYAZ, tin.
100011 04-71: -17CITN I CAL -L7I;RARY.P$-TRANVT I OP7, ;:NO.
100011 03%_CALW.

n 0 1102V.

A.

"T_R_1112PAr7F _Th7cHrotriny

alANT I ZAT I rIN
g

c

7

r

7

(7$1(- TRANSIAT Iruil OF

LABflRc.TORIE,/142..1; L.

GrT4L's Uri% NOtl2uNiFtwtri 'STEPS. //11-REDITNOCI _ikAch
7

77

1

7.

17

1

*

1

1000110119641957BRDADSEBFA435jA/=43USAENGRUSUAW0000S1C2/_PAM

1 111 1111

_KANTOVAN

1

11 1

1

1

1

11 1
64-3/_GARMASH5

11

0

1I

L-3

II
II I
II
I
I
ollloolomoommoolommoo1110001100000111101111110001100000000110m010101

G

1111111111111111111111111111111111111111111111111111111111111111111111111111111
22 2222 2 2222 2 2 22212 22212122222222 22222 212 22 2221222 2 2 22212 21122 222222 2 2 272 2 22212 22?

tzi

4:

123456789miluom516171819minnumannaminnummtnnocnumsnenowsluuNsmussmomucmmumeumnnnunnunlm

IPi

3 3 3333 3 3333 3 3 3333 3 333 33 3 3313 33 3 333133 3 3333 33 33133 3 33333133 333 3333 3133 33333 3 3133
1444444444442444444141444441444441111441444444141444444444444444144144444441444444

I

55r.555555555551555555515555515555555/5551155555555555555555155555555515555555551
66666 6 6 66616 6 6666 6 66666 616666666 6666 66 6 6 666 66 666616 6666 66 666 6 66616 6 6 666 6666 6 6 666

I7177777777777717177111777777777711777777717777777777777777717777777771777777177
88888888888888888888888888888888118881888888881888988888888188888888818888881101

999999 9999 99999999999 999999/999 999 9
999999 9 9919 9 9199919999 9 9 q9999999 99 999999999/9
401142131415 1611110 50 51 525351 5556 51 51596061"63646566 6161 69 10 11 721111 157671 117951
.

7

1

A

5

6

1 8 9 10 11 12131115 1611 11 19 2121 222324 152621 21293131 32 33 31 3536313139
ism 132D

I

I

H
ti
w

9
8
R

0
2

End of record. An explicit symbol is used to signify
Currently, a "+" is used for thisthe end of the logical record.
purpose.
c.

Logical Record. The logical record content and encoding
is organized to be independent of the physical medium used for
It is not dependent on the(It is "free-floating".)
input.
symbols provided on any particular device, although a particular
implementation of the input format implies selection of a particular set of symbols for a given device. The set chosen in theremainder of this specification is based on the keyboard of aStandard 029 keypunch.
2.

D.

DATA ELEMENTS AND CODES

The following tables list the complete data elements andcodes currently implemented in the ILR Input Record Format.

FIGURE 5:ILR INPUT RECORD FORMATPROCESSING FORMAT
nonEs

I-FIELDS: DATA ELEMENTS AND CODES

..

1 -I

FIELD
CODE

ELEMENT NAMECODE
VALUE(S)

z
ri

-1

None

2. Record Status:

Uncertified new record
Certified new record

0

151

El

IT!

c9,

A cal 5-,

6B8

L

40-46

L

12
(0)

It

(1)

a

Deleted record

None

4. Publication Date 2

r..,

c...)

None

Changed record

3. Publication Date 1i 1 I 2 CH'

%1

°

8
1. Master Record Number (ILR)

m

(2)

a

(3)

None

F

2-5

11

F

6-9

F

1

5. Pub. Date Type:2 dates, 2nd is copyrt

b

c

(c)

2 dates, 2nd is terminal

b

m

(m)

Date not known

b

n

(n)

Digits missing

b

q

(q)

Prey, public., all digits

b

r

(r)

Single date, all digits

b

s

(s)

L

6. Type of Record:

14

Language materials (books)

c

a

(a)

Language materials (mss.)

c

b

(b)

Music, printed

c

c

(c)

Music, manuscript

c

d

(d)

Maps/atlases, printed

c

e

(e)

KEY to components:..----

L = Leader, D = Directory, F = Fixed Length Data
Elements Field, V = Variable Fields-219-

FIGURE 5 (Cont.):
ILR INPUT RECORD FORMATPROCES SING FORMAT
CODES

I-FIELDS: DATA ELEMENTS AND CODESELEMENT NA ME

FIELD
CODE

CODE
VALUE(S)

8

N
N

o

' 2w

144

w

o

14

L

6. Type of Record (Cont.):

H

0

t-14

o

CO

Maps, manuscript

c

f

(f)

Motion pic. & filmstrips

c

g

(g)

Microform publications

c

h

(h)

Phonorecords, spoken

c

5_

(1)

Phonorecords, music

c

j

(j)

Pictures, designs, etc.

c

k

(k)

Machine-readable data

c

1

(1)

15-17

L

7. Bibliographic Level:

Analytical

d

a

(a)

Collective

d

c

(c)

Monograph

d

m

(m)

Serial

d

s

(s)

33

L

8. Cataloging Source Type:Central (LC Card)

e

a

(a)

Local original

e

b

(b)

NUC

e

c

(c)

Other

e

d

(a)

e

z

D

10. Agency Code for Cataloging
Source

f

a

L

34-36

11. Agency Code for Adaptorof Catalog Card

f

b

L

37-39

9. Book Not in LC (LC Call
No. Bracketed)

-220-

050

1

FIGURE 5 (Cont.):

ILR INPUT RECORD FORMATPROCESSING FORMAT
CODES

I-FIELDS: DATA ELEMENTS AND CODES

H

z
E-1

ELEMENT NAME

FIELD
CODE

CODE
VALUE(S)

8

o

o

Al CD

'El

riM

.4

'2ff-i
r...,

I 41=

Ocqr3c.
10Microfilm

g

a

(a)

Microfiche

g

b

(b)

Micro-opaque

g

c

(c)

11-14

F

13. Content Forms:

.

ti4

F

12. Form of Micro-reproduction

1

tO3

64

°

,,

Bibliographies

h

a

(a)

Catalogs

h

b

(b)

Indexeshc(c)Abstracts

h

d

(d)

Dictionaries

h

e

(e)

Encyclopedias

h

f

(f)

Directories

h

g

(g)

Yearbooks

h

h

(h)

Statistics

h

i

(1)

Handbooks

h

j

(j)

Other

h

k

(k)

Holdings:

j

a-j

D

090

Call Number

None

V

None

Copy Number

%

V

(%)

Library Code/Location

%

V

(%)

Total copies in location

%

V

(%)

F

15

15. Government Publication:

U.S, Federal

k

a

(a)

California State

k

b

(b)

-221-

FIGURE 5 (Cont.):

ILR INPUT RECORD FORMATPROCES SING FORMAT
CODES

I-FIELDS: DATA ELEMENTS AND CODESHELEMENT NAME

FIELD
CODE

CODE
VALUE(S)

0
C.)

N

o

o

P-ci

.
cci

ei

El

A-----

0 Ca 0 =H

F

15. Government Publication

41=

15

(Cont.):

California County/Muni.

k

c

(c)

International Orgn.

k

d

(d)

Other Countries

k

e

(e)

16. Conference Publication

m

a

F

16
(1)

17. Main Entry Heading Is
Repeated In Body of Card

n

a

F

17
(1)

18

F

18. Literary Group Filing
Indicator:
Complete/collected works

p

a

(a)

Selected works

P

b

(b)

Prolific author

P

c

(c)

19. Cancel Title Added Entry
in Dictionary Cat. Only
(See AA 33P3 and 6)qa

20. Cancel Title Added Entry
in Both Dict. & Divided
Catalogs

r

a

D

240

0

21. Language codes:

s

a

D

003

X

Lang. of text sub-field
Summaries

22. Translation Indicator

F

19
(1)

+

None

V

None

%

V

(%)

a

t

D

003

1

An "X" means this portion of the code is supplied via the input codesin another element of this table, or another table (q.v.).
-222-

FIGURE 5 (Cont.):

ILR INPUT RECORD FORMATPROCESSING FORMAT
CODES

I-FIELDS: DATA ELEMENTS AND CODES
H
FIELD
CODE

ELEMENT NAME

.0

(Z W

cr3

H 1-4

44 A F.T.1

o

PI

<4.4

o

0

14 H

2...,
0 0=
1

0
0

o
.4
H

M
H

ir-SI

co
czt

u

a

D

1XX

X

24. Main Entry is Subject
Indicator (See Fig. 8
for table of values for
Indicator 1, for which
this is a stacked code)

u

b

D

1XX

X

25. Main Entry is Publisher

u

c

D

261

1

26. Type of Added Entry Codes(See Fig.9-12 for tableof values for each category of added entry):w

2-char.
repeatable
codes

D

4XX

X+

Subject Added Entries

D

6xx

x

Other Added Entries

D

7XX

X

Series Traced Diff.

D

8XX

X

I

M

o

r=4

23. Type of Main Entry Code(See Fig. 8 for tableof values which complete
the tag and indicator)

Series Traced Same

ii

CODE
VALUE(S)

H

rA =

cil

An "X" means the 2-character codes in the *Value column translate into
the 2nd and 3rd character of the tag and the value for Indicator 1.

-223-

FIGURE 6:ILR INPUT RECORD FORMAT

A-FIELDS: DATA ELEMENTS AND CODESPROCESSING FORMATCODESAELEMENT NAME

0
0

A

CODE
VALUE(S)

rA

H
r...

Local Call Number

/

88c

0
H
2M
.:4

0
0

0
g

D

090

1-4

.

IA
. gl
=

H

rA

cr)

1-4

PT-4 HI
r...

lXX

2. Main Entry Heading:

Personal Name
None

V

None

Titles of honor, etc.

%

V

(%)

Identifier

%

V

(%)

Relator

%

V

(%)

None

V

None

V

(%)

None

V

None

Number

%

V

(%)

Place

%

V

(%)

V

(%)

Name sub-field

Corporate Name (&-Firm Name)

Name sub-field
Each subheading unit
Conference Name

Name sub-field

Date of conf,

Corporate with Form Subhd.

Name sub-field

None

V

None

Form subheading

%

V

(%)

None

V

None

Uniform Title M.E. Heading
Title sub-field
Other information

11

11+An "X" means this portion of the code is supplied via the input codes
described in another table in this specification (q.v.). Each variable
field contains 2 positions for Indicators 6 and 7 at the beginning of
The data starts in character position 3.
the field.

-224-

FIGURE 6 (Cont.):
ILR INPUT RECORD FORMAT

A-FIELDS: DATA ELEMENTS AND CODESPROCESSING FORMAT
CODES
VT1

ELEMENT NAME

FIELD
CODE

CODE
VALUE(S)

80

r:

C13'

Ec-1

2 gi:
a

<4
(..)

H VI

rii

I

H
H

CD

.

Supplied Title (Uniform
Title in Interposed
Position):
Not printed on LC cards$$+

Printed on LC cards

$

D

200

/

D

240

Remainder of Title

/

V

(%)

Remainder of Title Page
Transcription

%

V

(%)

Edition#D

Remainder of Statement%

V

Title Statement:
Short Title

.

X

None

Edition Statement:
250

None
(%)

6. Place of Publication/

R($)++

D260

7. Publisher

/

R($)

D

261

8. Date of Publication

/

D

262

pagination/Volumes

/

D

300

Illustrative Matter

/

V

(%)

Size

/

V

(%)

$

D

9. Collation Statement:

0. Bibliographic Price

None

350

1

+This code does not exist in the present version of INFOCAL.
If tag is non-repeatable,-4.+An "R" means the data element is repeatable.
element
occurringin a record are
the 2nd and succeeding values of theassigned a preceding delimiter ("%"). If tag is repeatable, "$" translates to tag.
-225-

FIGURE 7:ILR INPUT RECORD FORMATPROCESSING FORMAT
CODES
H
'AZ
ru
H
C41;% 8 C
IZ

B-FIELDS: DATA ELEMENTS AND CODESELEMENT NAME
(Primary Series of Codes)

1. Series Note, Traced Same
(Auth. +Title form):

IELD
CODE

CODE
VALUE(S

*

a

E9

rzl

0

Personal Name
Sub-fields are the same as In Main Entry Healing

rT.4

H CD

Bd

D

2 pe

C.)

4xx

H

I

x

with addi ion of:Series title sub-field

$

D

4o8

None

Volume/Number

%

V

"

(%)

Corporate Name (& Firm Name
Sub-fields are the same as in Main Entry Heading

Series title sub-field$

Volume/Number

with addi7;ion of:

D

418

v

m

None
(%)

Conference Name
Sub-fields are the same asn Main Entry Headingwith addition of:
None

Series title sub-field$D

Volume/Number%

V

See Fig. 9 for codes which
Indicator.

dentify remainder of tag dEgits +
*

418

(%)

b

D

440

c

D

490

0

*

d

D

490

1

*

e

D

011

6. Bibliography Note

*

f

D

500

7. "IN" Analytic Note

*

g

D

570

8. Dissertation Note

*

h

D

510

2. Series Note, Traced Same
(Title only)
3. Series Note, Not Traced
4. Series Note, Traced
Differently.

National Bibliography Number

-226-

,

FIGURE 7 (Cont.):
ILR INPUT RECORD FORMATPROCESSING FORMAT
CODES
CJ
H
p:

B-FIELDS: DATA ELEMENTS AND CODESELEMENT NAME

FIELD
CODE

CODE
VALUE(S)

0H cf)
tI4 0

H
*

i

D

520

10. "Bound With" Note

*

j

D

530

11. General Notes

*

k

D

550

12. Dash Supplements*ft+

13. LC Subject Headings:

*

m

D

6xx

9. Contents Note (Formatted)

I-1

PA 2 m1.7-1

HI0 cf) H

Personal Name
Sub-fields are the same as in Mai
addition of:
Title of Book

Entry HeadingD$with the608

Corporate Name (& Firm Name)

Sub-fields are the same as in Mai
addition of:
Title of Book

Entry HeadingD$with the618

Conference Name

Sub-fields are the same as in Mai
addition of:
Title of Book

Entry HeasingD$with the618

Cor.orate with Form Subhd.

Sub-fields are the same as in Maid Entry Hea ing
addition of:
Title of Book

D$with the628

Uniform Title Heading
Sub-fields are the same as in Mai. Entry H a ing
addition of:
Title of BookD$

See Fig.10 for codes which identi

other twith the638

es or LC lubj c+This code does not exist in the present version of INFOCAL.
-227-

Hdgs.

FIGURE 7 (Cont.):
ILR INPUT RECORD FORMATPROCESSING FORMAT
CODES

B-FIELDS: DATA ELEMENTS AND CODESz

E-I

ELEMENT NAME

FIELD
CODE

CODE
VALUE(S )

14

88c

°
H
6xx

--

hyphens

5. Bibliographic History Note

*

p+

D

503

16. Added Entries, Non-Subject& Non-Series:

*

q

D

7XX

4, LC Subject Subdivisions

Personal Name
Sub-fields are the same as in Main Entry He:din:. , wit
addition of:
Title of Book

$

D

the

708

Corporate Name (& Firm Nam.-)
Sub-fields are the same as in Main Entry He:din:. , wit
addition of:

the

718

Title of Book
Conference Name

Sub-fields are the same as in Main Entry Heading, with theaddition of:718

Title of Book
Cor.orate with Form Subhd.

Sub-fields are the same as in Main Entry Heading
addition of:wit

Title of Look

728

the

Uniform Title Heading
Sub-fields are the same as in Main Entry Headin:
addition of:

Title of Book

$

D

wit

the

738

See Fig.11 for codes whicl identify other type. ofof this category.

ded -ntries+This code does not exist in the present version of INFOCAL.
processing Format tags referred to are MARC II definitions.
-228-

The

FIGURE 7 (Cont.):
ILR INPUT RECORD FORMATPROCESSING FORMAT
CODES

B-FIELDS: DATA ELEMENTS AND CODES

%

0

E-1

W

ELEMENT NAME

1

Series Added Entries(Traced Differently):

.

FIELD
CODE

CODE

Z
o

r

*

D

i-.1

Cl;13 C

0

.=

q H
114

rzi

c.)

0

VALUE(S )

Iri

H
fr-1

8XX

Personal Name
Sub-fields are the same as in Main Entry Heading
addition of:D$
Series title sub-fieldwith the
None

808
II

(%)

VOlume/Number
Corporate Name (& Firm Name)

Sub-fields are the same as in Main Entry Heading
addition of:Series title sub-fieldD$with the
None

818

(%)

Volume/Number
Conference Name

Sub-fields are the same as in Main Entry Heading
addition of:with theSeries title sub-field

$

D

818

None

Volume/Number

%

V

"

(%)

See also Fig. 12 for Series Added Entry in Title only Form.
18. Title Romanized Note
(No Added Entry)

*

n

D

210

0

LC Call Number:

*

s

D

050

0

1

.

Class Number sub-field

Book Number

None

V

II

V

-229-

None
II

FIGURE 7 (Cont.):
ILR INPUT RECORD FORMATPROCESSING FORMAT
CODES

B -FIELDS: DATA ELEMENTS AND CODES

w

H

Pb
Eizi4

ELEMENT NAME

20. LC Copy Number Statement:

Class Number sub-field

FIELD
CODE

CODE
VALUE(S)*t

II

Copy Information%uClass Number sub-field

None

Copy Information%

22. Dewey Decimal Class No.:*

Segment sub-field/

23. LC Card Number
4. Overseas Acquisition No.

w

D

124

RM

M

I

Ffl

11

(%)

091

V

None

V

11

V

(%)

D

030

R(%)
R(/)

*

x

D

001

*

Y

D

013

-230-

H U)

None

V

[End of primary series]

14

.

051

V

*

11

D

V

21. Local Library Copy Statement:

Book Number

10 .-:-

2 pc

.c4

V

None

Book Number

0

.9=
csn

FIGURE 7 (Cont.):
ILR INPUT RECORD FORMATPROCESSING FORMAT
CODES

B-FIELDS: DATA ELEMENTS AND CODES

41

A ::

88c

f--1

I

0
A

ELEMENT NAME

N

izi

1-4

CODE
VALUE(S)41(Secondary Series of Codes)

r24

H Ca

o
0

H

w

2M
H

a+

D

012

.

b

D

540

3. Abstract or Annotation

!

c

D

560

4. [Reserved for later use]!

d+

5. MeSH Main Headings

!

e

D

660

6. NAL Agric./Biol. Vocab.

!

f+

D

670

7. Local Subject Heading System

!

g+

D

690

h

D

060

i

D

070

j+

D

071

k

D

052

n

V

052

P

D

580

q

D

210

1. Standard Book Number

12. "Limited Use" Note

1

.

1

8. NLM Call Number.

9. NAM Call Number

1

10. NAL Subj. Category No.

t.

11. Cooperative Cataloging
Library Call Number
12. Special Classification
System
13. Supt. of Documents Catalog
Number

.

1

.

r

m+

!

!

16. Title Romanized Note
(Make Added Entry)'.

17. MeSH Topical Subheadings

.

t

D

661

18. MeSH Geographic Subheadings

!

u

D

662

19. MeSH Time Period Subheadings

!

w

D

663

x

D

664

20. MeSH Form Subheadings
21. NUC Card Number

COI-144/4-.

14. Agency Name of Cooperative
Cataloging Source
15. "Full Name" Notes

I

1

1

.

1

.

(%)

I

y+

[End of Secondary Series]+This code does not exist in the present version of INFOCAL.
Processing Format tags referred to are MARC II definitions.
-231-

The

FIGURE 8:
INPUT CODE VALUES TABLE FOR TYPE OF MAIN ENTRY(UA CODE)TYPE OF ENTITY

SUB-TYPE(Function = Main
Entry Heading)

N

I

E-1

co

PROCESSING FORMAT
CODES

N
Z
0 fli
0 P4
P7-1

Combined with:
A-Field 2nd "/"

Pi

TAG

INDICATOR 1

A
0
0
0 0

A

Main Ent.' ain Ent.
Not Sub. Is Sub.

PERSONAL NAME

Forename

4Single Surname

5Multiple Surname

6Name of Family

7

Surname (Inverted )CORPORATE NAME

c

0/)4

100

0

)4.

Place or Pl.+ Name

5

Name (Direct Order)

6,

CONFERENCE NAME

Surname (Inverted)

e

0/4

111

0

4

Place or Pl.+ Name

e

1/5

111

1

5

Name (Direct Order)

e

2/6

111

2

6

f

0/1

112

0

1

0/1

120

0

1

FIRM NAMECORPORATE NAMEWITH FORM SUBHD.

UNIFORM TITLE

TITLE MAIN ENTRY

General

u

0/1

130

0

1

Anonymous Classic

a

0/1

131

0

1

t

0

240

0

1

+One-digit codes to the right of the slash are set by input of the "ub"code in the I-Fields.
-232-

FIGURE 9:
INPUT CODE VALUES TABLE FOR TYPE OF ADDED ENTRIES(W CODE)

TYPE OF ENTITY

SUB-TYPE

PROCESSING FORMAT
CODES

I

A
rn

(Function = Series
Notes, Traced Same,
Author+ Title form)

z
o

o ca4

rzi

rzi

N
A

N

o
o

Combine ylth:B-field *a

g441

TAG

INDICATOR 1Aoo,uth. Not Auth.In
ain Ent. Main Ent

PERSONAL NAMECORPORATE NAMECONFERENCE NAME

Forename

p

0/4

400

o

4

Single Surname

p

1/5

400

1

5

Multiple Surname

p

2/6

400

2

6

Name of Family

p

3/7

400

3

7

Surname (Inverted)

c

0/4

410

o

br

Place or Pl. + Name

c

1/5

410

1

5

Name (Direct Order)

c

2/6

410

2

6

Surname (Inverted)

e

0/4

411

o

4

Place or Pl. + Name

e

1/5

411

1

5

Name (Direct Order)

e

2/6

411

2

6

f

0/1

412

0

1

FIRM NAME+One-digit codes to the right of the slash are set by input of the"uc" code in the I-Fields.

-233-

FIGURE 10:
INPUT CODE VALUES TABLE FOR TYPE OF ADDED ENTRIES(W CODE)

TYPE OF ENTITYw
P4(Function = Subject
Added Entries &Subj. Subdivisions)
Combine with:
B-FiPld "*m"PERSONAL NAMECORPORATE NAMECONFERENCE NAME

1

gq

PROCESSING FORMAT
CODES

E

SUB-TYPEM

CZ PT1

41

ri-i

PT1

14

o

o

A

c..)

A

TAG

INDICATOR 1

c..)Forename

p

0

600

0

Single Surname

p

1

600

1

Multiple Surname

p

2

600

2

Name of Family

p

3

600

3

Surname (Inverted)

c

0

610

0

Place or Pl. + Name

c

1

610

1

Name (Direct Order)

c

2

610

2

Surname (Inverted)

e

0

611

0

Place or Pl. + Name

e

1

611

1

Name (Direct Order)

e

2

611

2

f

0

612

None

620

None

FIRM NAMECORPORATE NAMEWITH FORM SUBHD.
General

u

0

630

None

Anonymous Classic

a

0

631

None

TITLE OF WORK

t

0

640

None

TOPICAL

0

0

650

None

1

0

651

None

UNIFORM TITLEGEOGRAPHIC NAME,POLIT. JURISDICT.

2

-234-

0

652

None

1

[7

FIGURE 10 (Cont.):
INPUT CODE VALUES TABLE FOR TYPE OF ADDED ENTRIES(w CODE)

TYPE OF ENTITYPROCESSING FORMAT
CODES

SUB-TYPEi(Function = Subject
Added Entries)

R4 r4

w

w

TAG

INDICATOR 1

653

None

655

None

PERIOD SUBJ.+SUBDIVISIONS

66

None

PLACE SUBJ.+SUBDIVISIONS

657

None

Cont'd.

I.

R4

A A
0
0 00

NAMES NOT CAPABLE
OF AUTH.

GENERAL SUBJ.
SUBDIVISIONS

5

0+See also B-Fields, Primary Codes, item No. 14.

-235-

FIGURE 11:
INPUT CODE VALUES TABLE FOR TYPE OF ADDED ENTRIES(W CODE),

TYPE OF ENTITY

SUB-TYPE

PROCESSING FORMAT
CODES

ill

m
(Function = Nonsubject, Non-series
Added entries)

o

wki

r4

r4 '

c.)

r...)

P
o

Combine with:

o PA

r5.4

Po

INDICATOR 1

TAG

B-Field "q"Role
Alt. Conn.Analyt.

PERSONAL NAMECORPORATE NAMECONFERENCE NAME

Forenamep @/d/h 700

@

d

h

Single Surname

p a/e/i 700

a

e

i

Multiple Surname

p b/f/j

700

b

f

j

Name of Family

p c/g/k 700

c

g

k

Surname (Inverted)c@/(141

710

@

d

h

Place or Pl. + Namec ale/i

710

a

e

i

Name (Direct Order)

c b/f/j

710

b

f

j

Surname (Inverted)e @/d/h

711

@

d

h

Place or Pl. + Namee eVe/i

711

a

e

i

Name (Direct Order)

e

Vf/j

711

b

f

j

f 0Y1/2

712

0

1

2

r 0/1/2

720

0

1

2

Generalu 0/1/2 730

0

1

2

Anonymous Classic

u 0/1/2

731

0

1

2

Partial Title Work in Hand

t

0

740

0

Title Added Ent.Another Work

t

1

740

Analytic Title

t

2

740

FIRM NAME,CORPORATE NAMEWITH FORM SUBHD.

,

1

UNIFORM TITLE
TRACING

TITLE ADDEDENTRIES (TRACED
DIFFERENTLY FROMSHORT TITLE)

NAME NOT CAPABLE
OF AUTHORSHIP

n 0/1/2

-236-

753

1
2

0

1

2

FIGURE 12:
INPUT CODE VALUES TABLE FOR TYPE OF ADDED ENTRIES(W CODE)

TYPE OF ENTITY

SUB-TYPE

Z
0
ri .1

41

A

Combine with:
B-Field "*r"CORPORATE NAMECONFERENCE NAMEPROCESSING FORMAT
CODES

1

E

(Function = Series
Added Entry, Traced
Differently)

PERSONAL NAME

rxi

1414
OP-1
PA

I.

tl
41

TAG

A

INDICATOR
1

o C.)
o o
Forename

p

0

800

0

Single Surname

p

1

800

1

Multiple Surname

P

2

800

2

Name of Family

P

3

800

3

Surname (Inverted)

c

0

810

0

Place or Pi + Name

c

1

810

1

Name (Direct Order)

c

2

810

2

Surname (Inverted)

e

0

811

0

Place or Pl. + Name

e

1

811

1

Name (Direct Order)

e

2

811

2

812

None

840

None

FIRM NAMETITLE-ONLY FORM

t

-237-

0

E.

PRESENCE AND DEFAULT CONDITIONS

For purposes of the logic of the computer edit program, theI-Field elements have special "presence" conditions. Fig. 13
lists the status Of each input field code according to its
presence. Note that the absence of an input field does notnecessarily imply that the data element will be absent from theprocessing record. For those I-fields always present, it was
decided to define default settings that could be made by program
without any action by human editor or keyboard operator. Theseexperimental default coding conditions are listed in Figs. 14-15.

For purposes of setting Indicator 2, the value of which
records the sequence numbers of fields with multiple occurrences
in a record, Fig. 16 shows the fields which are currently repeatable in the INFOCAL program.

FIGURE 13:
PRESENCE OF FIELDS IN AN INPUT RECORD (MANUAL EDIT)PROGRAMSYMBOLALWAYS PRESENTDATA ELEMENT

RECORD COMPONENTEXTERNAL CODEI

LISTATUS

I-Fields "a"Record Status

Only present to override default setting

LIDTYPE

I-Fields "b"Date of Publication Type

Only present to override default setting

LITYPE

I-Fields "c"Record Type

Only present to override default setting

LIDATE1

I-Fields - noexternal code

Date of Pub. 1

No

LIDATE2

I-Fields - noexternal code

Date of Pub. 2

No

LIBLEVEL

I-Fields "d"Bibliographic
Level

Only present to override default setting

LISOURCE

I-Fields "e"Source Type of
Catalog Card

Only present to override default setting

LIORIGIN

I-Fields "f"Agency Code ofOrigin of Mach.

Only present to override default setting

Rec.

LIPROSOR

I-Fields "f"Agency Code ofProcessor of
Mach. Rec.

Only present to override default setting

LIMICROR

I-Fields "g"Form of Microreproduction

No

LIFORM

I-Fields "h"Content Form(s)

No

JCODE

I-Fields "j"Holdings Field(

LIGOVPUB

I-Fields "k"Govt. Pub. Indic. No

LICONPUB

I-Fields "leConference Pub.
IndicatorNo

LIMEBODY

I-Fields "n"Main Entry in
Body

No

-239-

)

Yes

part set by default)

FIGURE 13 (Cont.):
PRESENCE OF FIELDS IN AN INPUT RECORD (MANUAL EDIT)PROGRAMSYMBOLRECORD COMPONENgi

LILITGRP

I-Fields "p"

ALWAYS PRESENTDATA ELEMENT

EXTERNAL CODE
Literary GroupNo

Indic.

LICNCELT

I-Fields "q"

Cancel-Title Added
Entry - Diet.

No

RCODE

I-Fields "r"

Cancel Title Added
Ent.-Dict. & Div.

No

SCODE

I-Fields "s"Languages Codes

Only present to override default setting

TCODE

I-Fields "t"Translation
IndicatorNo

UACODE

I-Fields "u"Type of Main
Entry

Only present to override default setting

UBCODE

I-Fields " "Main Entry Is
Subj. Indic.

No

UCCODE

I-Fields "u"Main Entry is
Publ'r. Indic.

No

WCODE

I-Fields "w"

Type of Added
Entries CodesNoFIGURE 14:
INFOCAL DEFAULT INITIALIZATIONS REFLECTED IN PROCESSING
RECORD FORMAT WHEN NO INPUT CODES ARE RECEIVED

PROGRAMSYMBOLVALUE(S)DATA ELEMENT

LIPREFXLEADERLISDATE

Record Status Date

6 blanks

LISTATUS

Record Status

EBCDIC zero = uncertified

LILEGCNT

Legend Extension

EBCDIC zero = no extension

LITYPE

Record Typea- = language material,

printedLIBLEVEL

Bibliographic Level

ma = monograph

LIINDCNT

Indicator CountEBCDIC 5 = no. of indicatorbytes in a directory entry

LIORIGIN

Agency Code for Origin of
Machine Record

003 = original input at ILR
790 = original input at UCSC

LIPDATE

Date of Processing of
Machine Record

6 blanks (Refers to date oftranslation of UCSC originalformat to ILR processing
format)

LIPROSOR

Agency Code for Processorof Machine Record

003 = format translation
done by ILR from UCSC orignal format

LIPSOURCE

Source Type of Catalog
Carda = central (LC Card)

LIAGENCY

Agency Code of Source
Card000 = LCLIADAPTR

Agdipy Code of Adaptor of
Source Card

3 blanks

LIFIXEDFIXED LENGTH DATA ELEMENTS
FIELDAll elements are set to blanks except:
LIDTYPE

Date of Publication Types = single date

LICONPUB

Conference Publication
Indicator

0 = noLIMEBODY

Main Entry in Body Indicator

0 = noLICNCLT

Cancel Title Added Entry

0 = no (don't cancel title
added entry--make one fromshort title for dict. cat.)-2)41-FIGURE 14 (Cont.):

INFOCAL DEFAULT INITIALIZATIONS REFLECTED IN PROCESSING
RECORD FORMAT WHEN NO INPUT CODES ARE RECEIVED

PROGRAMSYMBOLVALUE(S)DATA ELEMENTELEMENTS FROM I-FIELDS WHICH ARE TRANSMITTED TO VARIABLE
DATA FIELDS
JCODE

SCODE

Copy No. at This
Holdings:
Location (2d sub-field)

01 = Copy No. at the shelflocation indicated is
Copy 1.

Shelf Location
Holdings:(3rd sub-field)

x = Stacks

Holdings: Total Copies at
This Location (4th subfield)

01 = There is one copy
assigned to thisshelf location, withthe copy no. shown inthe second sub-field.

Language Codes

eng = English is the languageof the text.

-242-

FIGURE 15:

DEFAULT SETTINGS FOR INDICATOR 1 IN VARIABLE FIELD DIRECTORYENTRIES TO WHICH INDICATOR IS APPLICABLE

INPUT COMPONENTAND CODEATTRIBUTE OF
INDICATORDATA ELEMENT

CODE
VALUE

I-Fieldsa

Languages

Single or Multilanguage

0

B-Field*s

LC Call NumberBook in LC

0

A-Field
I-Field
I-Field/2

Main Entry

1

ua
ub

Personal Name,
Single Surname,Not Subject

A-Field$

Uniform Title

Printed on LC Cards

1

A-Field
I-Field/3

Title Statement

Make title added
entry

1

A-Field
I-Field/6

Publisher

Publisher Not
Main Entry

0

ra

uc

These codes are set programmatically in the absence of an external
input code.

-2143-FIGURE 16:

LIST OF TAG NUMBERS WHICH ARE CURRENTLY REPEATABLEIN PROCESSING FORMATFIELD NAMETAG NUMBER,

051

LC Copy statement090

Holdings

091

Local Library Copy statement

260

Place of Publication

261

Publisher

400

Series Notes

550

General Notes

Oa

Subject Added Entries

7XX

Other Added Entries

8XX

Series Added Entries

-244-

F.

SUGGESTIONS FOR REVISION OF THE INPUT RECORD

General. The revised input format is predicated on anumber of factors:
1.

Unavailability of human editing talent in sufficientnumbers to accomplish large scale conversion of bibliographic files.
a.

Catalog cards follow a number of long-standing
numerals prenotational and stylistic conventions, e.g., Arabic
in parentheses.
ceding subject tracings, series tracings enclosed
ina bibliographic
These conventions will provide the textual clues
string that will support computer algorithms to accomplish areasonably high degree of reliable field editing.
b.

Ability of keyboard device operators to perform aforminimum of obvious coding, e.g., keying a synbol which stands
inspection of the catalog
a paragraph indention revealed by simple
source card.
c.

Sufficient regularities in catalog data, by form
(e.g., English)of material (e.g., monographs) and by language
Large
that computer-assisted formatting will prove feasible.
volumes of non-standard cards and idiosyncratic cataloging practices will tend to reduce the effectiveness of computer assigned
d.

codes.

The burden of proofreading and correction keyboarding must be less than in the straightforward system.
e.

The conversion may have to be "phased", i.e., theProcessing Format with full MARC II coding down to the level of
every data element and indicator may not be achievable on thefirst pass of the conversion effort. Several iterations preceded by development and re-design may be necessary to achievea fully-encoded machine file.
f.

The following table (Fig. 17) is illustrative of the kindsof clues and patterns found on standard catalog cards which might
be utilized in computer-assisted formatting. A total of six
slash marks are used to break the card up into a set of segments.
Various stylistic and notational clues are used to recognize byalgorithm the fields within each segment. This is a proposed
technique, more or less slashes may be required.

The recognition of sub-types of name and of the roles playedby names in added entries, form perhaps the most difficult problemof tagging in MARC II. However, it is not unreasonable to sapposethat a degree of accuracy could be attained that would be highenough so the total effort in proofing and correcting would beless than that now required to manually edit and proofread arecord.

-245-

FIGURE 17:

REVISED FIELD CODING:

A-FIELDS & B-FIELDSMANUAL EDITELEMENT NAME
Field
Code

Recognize by

COMPUTER EDIT
Examples of Conditions
Providing Clues to
Algorithms
I

None

Position in input
string; length indigitsLocal Call Number,None

Position following MRN

Main Entry Heading/

Master Record Number(MRN)

Supplied Title(Interposed)
Title Statement:
Short Title

Position;font

Punctuation; content
atterns

Brackets (left & right)
must be present

None

Paragraphindention/

Right bracket of unif.
title, if present

Remainder of Title

None

Punctuation; length
in characters fortitle added entry
purposes

Remainder of Title Pa e

None

jay word (e.q., "by")

Edition Statement

None

Key words; digits; etc.

Place of Publication

None

Publisher

None

Punctuation at end ofplace; authority tablelook-up; etc.

Date of Publication

None

Digits pattern

Punctuation

I.

Collation Statement:

Pagination/

Paragraphindention

Punctuation; character

patterns ("v%"p':)

Illustration

None

Key words; set Fixed
Field codes also

Size

None

Key word ("ce); digits

Price

None

Key symbols; digits;
punctuation

Parentheses; sequence
after collation or
price

Series Notes (1st group) None

.

-246-

FIGURE 17 (Cont.):
REVISED FIELD CODING: A-FIELDS & B -FIELDSMANUAL EDITELEMENT NAME

Field
Code

Paragraphindention

Series Notes (2ndgroup) and all othernotes
National Bibliography
NumberTracings: (code even ifnone present)

Recognize by

Paragraphindention;

Arabics &Roman Num.

Beginning of subject
tracings

Key words & patterns(this is the mostambiguous region on acatalog card)
Sequential position;
key symbols; digits

None

/

COMPUTER EDIT
Examples of Conditions
Providing Clues to
Algorithms

Arabic numerals; matchafter conversion to atagged authority file

Subject Subdivisions
Beginning of Other
Added Entries

Two hyphens (--)
Roman numerals

Series Added Entries

Parentheses; key word;position

Fields at bottom of card:/

Position

LC Call Number

Character patterns

LC Copy Statement

Multiple hyphens("dash".on format)

Dewey Number

Digit pattern

LC Card Number

Digit pattern

Overseas Acquisition
NumberKey symbols & abbrev.

An Illustrative Example. As an example of what can bedone to design computer-assisted formatting, the following isa preliminary algorithm for delimiting the Short Title sub-fieldof a MARC II record. This procedure is a special case of string
decomposition. This algorithm is phrased in such a form that it
can be used either for manual editing or in a computer program.
In essence, it is "machine-like" editing which is now performed
manually by card preparation sections in libraries. It is ofcourse subject to review by the cataloger on the basis of judgment as to what makes a title semantically informative and linguistically or grammatically acceptable. The algorithm is experimental and has been tested only in the manual coding effort
undertaken during Phase I of the File Organization Project.
2.

ALGORITHM FOR DELIMITING SHORT TITLE
the beginning of the titleComment. It is assumed that
explicitly identified in some way
page title statement will beof some title(visually, by a paragraph indention, or in the case
The same will be true formain entries, by hanging indention).
code such as "/" will
the Collation field. An explicit field
The problem is to determine
precede the first word in the title.
in MARC II. The operational
the end of the Short Title, as defined
title statement
definition of Short Title is: that portion of thea part ofwhich is to be used as a catalog entry heading or as
and which might be used separately
a heading, either main or added,
record, e.g., in index lists
from the remainder of the catalog
always berather than in card catalogs. The short title will
present.
delimited when more than one of the MARC II sub-fields are
setting for title added
This is independent of the Indicator
entry.
of the next
The algorithm does not identify the content
separate routine.
succeeding sub-field: that is assigned to atitlesThis set of rules is still "fuzzy" in that cases of long carried
the procedure to bein older catalog records which force
will not always produce
through the string length cutoff step
result.
a semantically or grammatically acceptabledependence upon the
A weakness in the procedure is its heavy
insertedpunctuation used on the title pages of books and upon thatpracthe. Since thisby catalogers according to conventional
adhered to because
practice is rarely documented or consistently
the need to rely on cataloger
of the variability of title pages andisjudgment, it is likely that the approach through punctuation
not susceptible of successive improvements.

Another weakness of the algorithm is that it cannot be
beginning of theconsistently applied for situations where thefrom the name of a real
short title contains one or more words
biographical study of Q.").
person (e.g., "Arthur Quiller-Couch; aif the subtitle were
A title added entry would be more suitable
included, and so the delimiter is not placed after the semicolon.

single intervening
The algorithm in draft form operates on
multiple patterns
marks of punctuation or other special symbols,Aof occurrence of a symbol, and combinations of symbols.
would be apotential source of improvement in the algorithm
catalog
statistical study of frequency of these symbol patterns inAtrecords, in order to optimize the sequence of the steps.
"intuitive" order.
this stage the steps are presented in purely-2)49-FIGURE 18:

TABLE OF VALID SYMBOLSNAME

GRAPHICEBCDIC HEX

Left Bracket

3E

Right Bracket

3F

Period4B

Less Than Sign

4c

Left Parenthesis)4D

Exclamation Point

5A

Right Parenthesis

5D)

Semi-colon

5E

;

Comma

6B

Greater Than

6E?

Question Mark

6F

Colon

7A

.

Quotation Mark

7F

[

]

.

(

1

.

2

ff

TABLE 1
Rule

UNCONDITIONAL (START)

1Scan left-right from start of

1

1.
t

'Title paragraph to start of1'Collation field (count)1'Flag end of each sentencei
I found (valid markers)*_J

For each intervening mark or symbol

1

3.
1

I

1

prior to first sentence marker, set
a position pointer (show type of

1

1

isymbol)*I*

Go to Table 2.

*A sentence is defined operationally as a phrase containing at least
one character the end of which is marked with one of the 4 marks:? and right (close) quotation mark " (the latter cannot be incharacter position 1).
!

**The first sentence is initially defined to be that string which is
See Fig.
marked by one of the 4 symbols, whichever comes first.
18 for list of valid marks and symbols.

-251-411,:aANN

TABLE 2Rule

SENTENCE END MARKER

End of sentence marker for first
sentence in Title para. is Period

1

1.
1

End marker is Exclamation Point

I

.

3.

,End marker is Question Mark

4!End marker is Right Quote MarkI

5. 'Valid punc. symbol intervenes
I

1 2 3

456

8

E *

Y Y
_

Y Y

Y Y
Y Y

YNYNYNYN(other than one of the 4 end
1'markers)i

6.

1Go to Table 3
I

7.

Go to Table 4

8.

1Go to Table 61

9.

IGo to Table 7

10.1Go to Exception Table**
1*Means "Else" - the rule to follow when none of the others apply.
**Not yet defined.

- 252 -TABLE 3
Rule

VALID SYMBOL SWITCHES*
Next intervening symbol is

1

1.

[

1 2 3 4 5 6 7 8 9 10 EYlY

2. 'Next intervening symbol is
3. 'Next intervening symbol i

.

Y

t

4. !Next intervening symbol i
5.

!Next intervening symbol is(6._]Next intervening symbol isI

7.

8. :Next intervening symbol is
1

9.
I

Y

Next intervening symbol is,

Next intervening symbol is >Next intervening symbol is

Y

:

.1

11.IGo to Table 8
12.IGo to Table 9
1

13.!Go to Table 10
x x

14.:Go to Table 11x

15. Go to Table 12

16.1Go to Table 13
17. Go to Table 14..--ii-x

18. I Reiterate Table 3 ***For symbols other than the 4 end of sentence markers, except a period.
Valid symbols (such as left ") not listed, and non-valid symbols
encountered in the string, follow "Else" rule.
**

Exit when no further intervening valid symbols exist in the string,$b +or when $b and $c have been set, or any combination of these:
no $c found; $c + no $b found. Length limit must always be tested.
(The logic for $c is dependent on a further algorithm). In any case,
if no further processing can be done for short title, procedure continues to further algorithms.

- 253 -TABLE 4
PERIOD (FULL STOP)

Rule

Period occurs in char. position

1

1

2

3

14.

5

Y

Y

N

N

N
.

1 ._.41

II , 2 3

Hit on author statement key

I

6

or 4
Y

Yi

Y

N

-

L

GT

LE

words list (test left & right)

1
1

End marker occurs in position
compared to string length limit*

1

Insert Interim Delimiter after

1

14.

N

1

j

I

lend mArker
1

.

6.

!Do Table 5

Insert a $c delimiter prior to authorix!statement key word (in "word - 2"
1

position if non-blank in Word -1I!position)i

Go to next end marker

8.

Go to Table 2

9.

Go to [tables for further algorithms,

x,

4

1

las needed**]1length of n where n is the*Length limit is a parameter for string
Shortmaximum number of characters desired in the output for which
overprinted in a
Title is being used, e.g., two lines of text
heading on an added entry card.
of the**Not included in this presentation. A positive determination
short title will often
explicit code for the sub-field following
parts of the title paradepend on algorithms for the subsequent
Delimiter may begraph, including imprint. Moreover, the Interimof Title nor Remainder ofdeleted altogether if neither Remainder
to exist in the
Title Page Transcription sub-fields are determined(A Place of Publication sub-field always exists).
record at hand.
end for the length
Finally, a check should always be made at theratherlimit, which may involve insertion of a $b delimiter on aarbitrary basis, in some older records.
must cope with are many: e.g., the period
that
this
test
The anomalies
appear to the left, as inmay be embedded, as in 'Ed. by ...'; it may
title proper, as in 'Philosoph'ctitle. By ...'; and it may be in thelmthose who studied...'ies men live Ia..' or 'Great teachers, portrayed- 254 -TABLE 5

LENGTH LIMIT (CLOSED)

Rule

1

2

Relation of author state- Hit left

1.

Hit right

3

4

5

Hit Rt. NoHit NoHit

ment key word to end
markerBlnk

Character to which length

2.

Non4

'S

limit signal points_

No short title delimiter3.

x

required1.

x

Scan right to left for 1st

non-blank character(hope-!

fully an interword 10
Insert $b delimiter

5.

I.

x

x

x

x

in blank position
Exit

6.

x

TABLE 6or ? END MARKER

1

1'after the
I

71^

ty

Rule

Insert Interim Delimiter immediately

1.

.

,

.

or ? markIGo to Table 3

-255-

1

x

Non-

TABLE 7
1

Rule

RIGHT QUOTE MARK END MARKER*

2

or

1

1. !Character in next right character

-

4

3

or, Blank Blank

position from Right Quote Mark

1

1

2. !Hit on author statement key words
I!list (test word to right of RightI!Quote)7--' Insert interim Delimiter to right

.

!

1

of punctuation mark in next

'succeeding character position to.
1'right of Right Quote MarkI

Insert Interim Delimiter to Right4.
1

1 of Right Quote Mark

x

1

5.

' Insert $c delimiter to left ofixi' author statement key word (or.
1

I left bracket if any)
1

6.

x

x

' Go to Table 3x-*A left Quote Mark is assumed to have been found in the initial scanof the Title Statementsentence.
TABLE 8RuleLEFT BRACKET
1

1.

!Digits in character positions

1

2

3

Y

N

N

11-4 in Title Statement**I

Hit on author statement key words

2.

Y
4

3. JBracket position: Length limit

LB

Gt LE

4. 1No $b delimiter needed
1

I

5.

!Insert Interim Delimiter1

tpreceding left bracket
I

6.

Do Table 5

x

I

7.

!Insert $c delimiter precedingI!left bracketI8.

!Go to Table 3

Lx,

x

x

**Also scan to right of Left Bracket for string: "i.e.,- 256 -xx

TABLE 9i

RuleNEXT SYMBOL = ]) or
1.

Interim delimiter already set

2.

Symbol is followed by ";"3.

Symbol is followed by a 16

then cap letter not in auth.
statement key word list

Hit on author statement key

1

Y

2

6

7

Y

-

E

GT

NNNNN
YYYNNYNNYN
NYNN
3

14.

word list (word to right)
r-

EQ

5.

Site counter

6.

Insert Interim Delimiter after";"

7.

Insert $c delimiter

EQ,

:

EQ

Ev.,

x

prior to key word
8.

Go to [tables for further
algorithms]

9.

Return to site of corresponding
(

10.

or <

Go to Table 3

x

-257-

TABLE 10
PERIOD (NON FULL STOP)

Rule

1 2 3 4 5 6 7 8 9YNNNNNNNNYNNNNNNN
Period is "inside" to right of string
Interim Delimiter already set

1.
.

starting with [ < ( or left "3. J Period foll, by right " then lb or

YYNNNNN

4. JHit on author statement key words

Y N,YNNNN

list (test left and right),Y N N N

Single preceding character

.

Y N N

Single preceding word

6.

1

Y N

Next word after period is a repeat of

.

I

1_

first word or root thereof in Title
Statement

8.

Y

Ellipsis found, or string of interspersed periods,

Insert Interim Delimiter after.

x.

right " orIX x

10. 1 Insert Interim Delimiter after period

Insert $c delimiter prior to key word

11.
12.

1

x

Do Table 15

13.

1

Go to Table 3

14.

I

Go to [tables for further algorithms]._ x,

x

x

.

ixxxxl
i

xl
,

;

xxxl

.1

TABLE 11
Rule

LESS THAN OR LEFT PAREN.

1 2 3

1.

Interim Delimiter already set

Y N N

2.

First< or ( in Title Statement

Y N

3.

Set site counter

4.

Insert Interim Delimiter prior t
< or

,

T.

x x x

(

5. L Go to Table 3

i

i

1

6.

Go to [tables for further algorithms ] x x x

1

1

TABLE 12Rule

SE14-COLON
1.

Left Bracket pointer "o "2.

Left Quote Mark pointer "

3.

< or

4.

Interim Delimiter already set

5.

Semi-colon is first intervening(

symbol

"

1 2 3

5 6

ii.

Y

.

site counter non-even

Y Nother than 1-3 abovet

6.

Insert Interim Delimiter aftersemi-colon

7

xix x

Go to Table 3,

8.

Go.to [tables for furtherxx

algorithms]
;

;

_

-259-

TABLE 13

COMMA

Rule

Left Bracket pointer "on"1

1.

6 7 8 9 10.E

1 2 3 4Y=

2.! Left Quote Mark pointer "on"3.1 < or ( site counter non-even

Y
_

1

Y

Y

4.1 Interim Delimiter already set
1

5.

YNNNN

Digit pattern following comma1

6.1 Comma is 1st interven. symb.
I
!

7.
1

1

Hit on author statement key words
list (test right)

,

.

,

8.1 Hit on multiple title conditions*19.

Insert Interim Delimiter after comma

10.1 Insert $c delimiter after comma1

land prior to key wordI

Go to Table 3

11.
i

12.

Go to [tables for further

x

x x x

x x

x

algorithms]*E.g., combination of ", and 0..." where 0= Capital letter.

_

TABLE 14
Rule

COLON

9 E

Y

1.

Left Bracket sointer "o "2.

Left Quote Mark pointer "on"

3.

< or

4.

Interim Delimiter already set

5.

Hit on author statement key(

1 2 3 4 5 6 7 8YIsite counter non-even

Y

YNNNN

words list
6.

Next is period or

7.

Next is,8.

Next is[9.

Next is another colon

;

Y
(

Y

or 4

10.

Set Interim Delimiter after .;11.

Insert Sc delimiter prior to word

12.

Set Interim Delimiter after13.

Set Int. Delim. prior to [ etc.

14.

Go to Table 3

15.

Go to [tables for further

xxxx

x x

x x

algorithms]
16.

x

Repeat starting at Step 5

TABLE 15ELLIPSIS
1. I

CLOSED

3rd period followed b

2

1

Rule

Valid s Hbol

Blank

letteror

non-valid simpol
2.

Insert Interim Delimiter.rior to first eriod in
ellipsisx.1 Exit

[END]

-261

x

V-3.

EXPERIMENTAL ON-LINE MATHEMATICSCITATION DATA BASE*

By
Mary L. Tompkins
Institute of Library Research
University of California
Los Angeles, California
A.

INTRODUCTION

A Mathematical Citation Index was started irL March 1965 under
the auspices of the UCLA Computing Facility with the encouragement
and cooperation of the University Research Library, the Engineeringand Mathematical Sciences branch library and the Numerical Analysis
Research Library. Starting in May 1966, the project was continued
by the Institute of Library Research as an example of mechanizedservices which would be provided by the Center for Information Services in the University Research Library.
It is of particular value because a real data base is accumulating which will be of direct value to mathematicians and otherscientists on the campus, as well as contributing to increased
expertise in the utilization of large files.

Publication of a series of indexes designed to identify scientific serials has been started. These are generally being identified
as MAST: Minimum Abbreviations of Serial Titles. The first volume
in this MAST series, on Mathematics,was to appear in 1968, publishedby Western Periodicals, Inc.
Not only do we foresee the need to publish for wide distribution some products of our work, but we will need to utilize products
produced by others. To recreate files already compiled by others
is not feasible in terms of time or money. Thus, our planning for
an operational citation index at UCLA assumes the purchase andutilization of existing files, or selections from them, such as the
I.S.I. Science Citation Index tapes and the Physics citation data
being developed by M. M. Kessler at M.I.T. Such acquisitions would
then be augmented to satisfy local specialized demands.
B.

MAST:

MINIMUM ABBREVIATIONS OF SERIAL TITLES

Before the compilation of citations was begun it was evident
that some way was needed to identify bibliographic references whichwere incomplete, ambiguous or otherwise unidentifiable. Hence, thepreliminary work on the project concentrated on a circularly shifted
index of abbreviations of mathematical journals.
*The following material is excerpted from "Experimental On-line
Mathematics Citation Data Base," by Mary L. Tomlkins. Part 8 of
tne Final Report on Mechanized Information Services of the University
Libral.annin. Los Angeles, Institute of Library
Research, Dec. 15, 1967. pp. 3-13. Work on this particular phaseof the project was supportedufF Grant GN-503.

What has been widely discussed as a K-W-I-C (Keyword-in-Context)
descripor permuted index, we prefer to call by the more accurately
tive name Ci/cularly Shifted Index. In this index of abbreviations
word in aof serial titles, the abbreviation of every significant
title appears alphabetically at a center gutter, with its preceding
and following words around it. Each indexed abbreviation is accomThepanied by an identification number and a short one-line title.
identification number refers to a Reference Section which lists(for the more than 2000 serials) full and complete titles, datesof publication, and histories of title changes.
For any journal with more than one title, corresponding additional abbreviations are entered. Likewise, if a journal is sometimes known by a name not appearing as part of its title, thisname would be added. An example of this is "BIT," which is used
for the "Nordisk Tidskrift for Informations-Behandling"--the resultof a fancy cover design.
Minimum abbreviations (Minabbs) of serial titles and the indexing of them instead of the full titles, is a scheme devised byJohn W. Tukey at Princeton University and the Bell Telephone Laboratories in connection with his Citation Index for Mathematical
Statistics and Probability, and in response to a wide-spread desire
for a means of identifying journals cited by unclear and unexpandable abbreviations. The work at UCLA has been a direct outgrowthof the author's work with Professor Tukey.*

Briefly, a minimum abbreviation is formed by using the initial(For this purpose,
letters of the word up to the second vowel.
contiguous vowels are considered one.) Minabbs are not necessarily
the most common abbreviation and they are frequently not unique,but they may be formed from longer dbbreviations and from words
in any language. As a example: either the title, "Verhandlungender Naturforschenden Gesellschaft tm Basel", or the acceptableabbreviation "Verh. Naturforsch. Gesellsch. Basel" can be shortened
to the Minabb VERH NAT GES BAS, which can be located in the Circular Shift Index in any one of four places.
Ambiguity in an abbreviation is useful in this index because
it immediately displays where confusion may lie. J MATH PHYS may
indicate either the "Journal of Mathematics and Physics (MIT)" or
the "Journal of Mathematical Physics (New York)". The reference
section indicates that the former dates from 1921; the latter from1960. A rough volume-year correlation will help determine which
serial was intended.

*Tompkins, Mary L. and John W. Tukey. "Permuted Circularly-Shifted
Indexes: A Mechanically Prepared Aid to Serial Identification". IN:
Progress in Information Science and Technology; Proceedings of theAmerican Documentation Institute, 1966 Annual Meeting. Adrianne
Press, 1966. pp. 347-355.
-264-

.

Another type of confusion rests in multiple titles. The"Annales Academiae Scientiarum Fennicae Series A" is usuallycatalogued under its Finnish title "Suomalaisen Tiedeakatemian
Toimituksia. Sarja A", but is usually cited under the Latin.
The entry under the abbreviation of the first title will lead tothe second through the identification number, and thence to thelibrary classification and shelving of the volume.
The results of this phase of the Citation Index project are:
1.

C.

A dictionary of acceptable abbreviations for editing
input to the Mathematical Citation Index files, forindicating new or omitted titles of mathematicalserials and for drawing together in the citation
files different versions of a single title.

2.

An Institute of Library Research-UCLA Computing
Facility report on the computer programs.*

30

Publication of Minimum Abbreviations of Serial Titles:
A Circularly Shifted Index to Mathematical Journals.

SELECTION OF SOURCE JOURNALSAt the time this project was started "Mathematical Reviews"
listed 872 journals which publish research in mathematics. In
order to determine how many of these journals constitute the "core"of the literature, several mathematicians at UCLA and Princeton
were asked to indicate journals in which most of the significant
research appears. Each listed about 100 titles, and a set of 53
journals appeared in all lists. Our "core" consists of these 53
and other multiple but not unanimous choices.

From this an ordering of journals to be processed was established. First, twenty-two current issues of nineteen different
journals were scanned and the nunber of times journals were citedwas counted. There were 215 papers which referenced 1294 itens
distributed among 325 different journals. Those journals mostcited were given priority in processing. A slightly changed priorbe
ity list reflected the rankings after 13,500 citations could
counted. A third count of 22,800 journal.citations substantiates
these rankings. They are also compatible by and large with thosemade by Charles H. Brown using 3,168 citations from ten journals
published between 1950 and 1952."

A Circular Shift Index*Johnson, G. D., and Tompkins, Mary L.
of Abbreviations of Journal Titles: Description of the 7094
Computer Program, ILR-UCLA Computing Facility Report, 1966.
**Brown, Charles Harvey. Scientific Serials. Chicago, Associationof College and Research Libraries, 1956.

-265-

D.

SCOPE OF CITED ITEMS

During the first two phases of this project, 4094 articles
were processed. They were drawn from 25 journals published from1965 to 1967. Five journals were covered from 1965 to mid-1967;
nine more were covered for 1965 and 1966; eleven more were coveredin 1966 only.
New titles were introduced starting with their current issue,
working backward as time allowed. Once undertaken, the journal
was completed for the entire volume except when all processing had
to stop temporarily in July 1967.

At present 31,424 citations exist on punched cards and on
magnetic tape. 22,823 (or 73%) cite journal literature; 5840 (18%)
cite books; 3% cite material unpublished at the time of citation;and the remaining 1829 items fall in the miscellaneous categorYof reports, theses, lecture notes, proceedings of meetings, etc.
(These precentages substantiate those compiled on 18,000 citations
six months earlier: journals, 71%; books, 18%; and, unpublished,
3-1/2%.)E.

PROCESSING CITATIONS

Material for the Citation Index is keypunched on cards which
are later transferred to magnetic tapes, disks or data cell forcomputer manipulation and storage.
Current issues of scientific journals have been the solesource of citations and 73% of the input citations have referredto scientific journals. Therefore, the card format and the experiments with on-line input have been geared to journal identification.
It is possible, however, that bibliographies from other sources,
particularly from proceedings of meetings or Festschrifts, may bedesirable for inclusion. The format for a citing journal is readily adaptable to a book by merely assigning an arbitrary sevencharacter designation to the item to correspond with the journal
serial and volume nurbers. This number with that of the first pageserves to bind related citations to the source item.*
The author and title of the citing paper and of each referenced
work (excepting only personal communications) are recorded. An

The data record formats used in the early stages of the
UCLA Citation Indexing Project were developed in advance of the
Library of Congress MARC format. It is expected that analysis of
the feasibility of adapting a version of the MARC II format to thejournal article file format will be undertaken during a later phaseof the File Organization Project so that a uniform record structure
concept can be tested in an on-line environment, and for purposesof making the data base convertible to MARC II for transmission
to other agencies.
-266*NOTE:

AIL

attempt is made to preserve all bibliographic information given
fit conveniently on the rigidly
by the author. That which will notformatted citation or C-card is expanded in full text on supplemultiple authors whichmentary or S-cards. Extra long names, orrecorded on nameexceed the 15 character cited-author field areor N-cards.
Fig. 1 illustrates the cards punched for citations froma single paper.
F.

USE OF DISPLAY STATIONS FOR ON-LINE EDITING

Programming the IBM 2075 for the Mathematical Citation Indexof cathode ray
has been designed to exploit the display potential
on-line visual access,tube terminals. These instruments provide
query, and input to computer stored data. Man-machine communicar.
tion with them requires a minimum of special operator skills.
They can be used geographically distant from the computer.
UCLA time
The program we call CITATION is an entry into the
It is designed to facilitate input of new data
sharing system.
into the Citation Index file or correction of existing records.
numerical data.
It edits input records, refusing to store illegal
An illegal entry may be one that is too long, or too small, orA message
one that contains a blank or an embedded character.
indicating the error will appear on the scre,en until correction
is made.
Separate formats accommodate information about cited andthe punched card format
citing items. These have been based on
predominate, attenfor easier conversion. Since journal articles
tion has been paid to their peculiar bibliographic conventions.
The program commences by clearing the screen (buffer) anddisplaying MAIN TITLE:
CITATION INDEXTYPE A FOR CITING AUTHOR FORMATT FOR TITLE OF CITING PAPER
C FOR CITATION FORMATR FOR CITED TITLE
I TO INITIALIZE
D TO DISPLAY
S TO STORE

IL

I

When the letter I is typed the screen displays:
JOURNAL FORMATJOURNAL NUMBER
VOLUME NUMBER
YEAR
-267-

FIGURE 1:

CARDS PUNCHED FOR ONE PAPER PUBLISHED IN VOL. 66
OF, THE COMMUNICATIONS IN PURE AND APPLIED MATHEMATICS'

For the Citing Paper
Field

1. Type Card (Al = Author; T1 = title)
2. Type Number
3. Year
4. Journal
5 -Volume
6. First Page
7. Last Page
8. Author name or title statement

For its Cited Items
Field

1. Type of Card
2. No. of Citation
3. Citing Journal
4. Citing Volume
5. Citing Page
6. Author
70 Designation of Multiple Author
8. Designation of Non-journal
9. Abbreviation of Location (title of source of cited
10. Volume
item)
11. Issue
12. Year
13. First Page
14. Last Page

-268.-

The volume, year and assigned identification number of thejournal containing the citations to be recorded are entered in the
INITIALIZE routine. These numbers provide the link between citedand citing papers and automatically become a part of every succeeding record mitil tPe program is reinitialized.
Typing letter A presents the AUTHOR display:
CITING AUTiOR INPUTFIRST PAGE NUMBERLAST PAGE NUMBERYEAREntered page numbers are checked to be sure the last is at least
as large as the first. Multiple authors may be listed.
The TITLE display is simply:

TITLE OF CITING PAPER
and allows for 44o characters of free text.
The citations are now entered on the following format.
CITATION INPUTVOLUMEYEARISSUE, EDITIONFIRST PAGE CITEDLAST PAGE CITED
JOURNAL ABBVAUTHORSThe CITATION FORMAT is displayed automatically with serially
increasing numbers for each entry until A is called, for the next
citing author input. Before the next series of citations can beentered, however, a subtitle asks the operator,
ARE YOU STILL IN THE SAMEVOLUME AND JOURNAL? IF YOU ARE
NOT, PLEASE RE-INITIALIZE.
The editing routines of this program are being constantly
expanded. The next powerful addition to the system will be a checkof journal abbreviations to insure that all citations to a given
paper are being lumped together regardless of the vagaries ofreferencing by the citing authors.
-269-

APPENDIX VI

SWLE SIZE DETERMINATION FOR DATA CONVERSION QUALITY CONTROL

By

Jorge Rodriquez
Institute of Library Research
University of California
Berkeley, California

1.*

SAMPLE SIZE DETERMINATION FOR DATA CONVERSION QUALITY CONTROL
By

Jorge Rodriguez
Institute of Library Research
University of California
Berkeley, California
A.

INTRODUCTION

This aaalysis aims to solve a specific problem of theInstitute of Library Research, summarized as follows: it isdesired to investigate the possibility of using e, statisticalcontrol method to determine the accuracy of the data conversion
process and to estimate the frequency of errors associated witha common source.
The constraining factor involved in the control process isthe cost of sampling as opposed to the cost fluctuation of theconversion process as a function of the level of accuracy.
B.

DISCUSSION

It has been suggested that sampling inspection, and more
specifically, sequential sampling inspection, be studied as apossible control method. It will later be explained that thenature of the problem suggests other means of control.
In sampling inspection, whether it is single, double, orsequential, we normally start with two fixed points. Assumingthat the parameters in question lie on the mean of a distribution(as in the present case) these two points are: a previously
estimated mean value, which is assumed to remain fairly constantand considered acceptable; and the worst value of the mean thatwill be considered acceptable. This introduces the notion of
acceptance and rejection of the lot in question.
The test consists of obtaining a sample from which themean of the population is estimated using a proper estimator.
Then, using the lot size and sample size, obtain some measureof the quality of the decision. For this purpose, two types oferrors are defined and their probabilities estimated: namely,
the probability of error of type I, which is the probability ofrejecting a good lot; and the probability of error of type II,
which represents the probability of accepting a bad lot.

The situation in the data conversion is somewhat different.
In the present problem it is not assumed that the mean of the
distribution remains fairly constant, nor that there is a regionof acceptance opposed to one of rejection. The problem here ismore related to point estimation, to the actual estimation of
the expe,..ted number of inaccurate records, and to obtaining some
e2 7,V-273-

^measure of the estimate in the form of a probability that deviates
by an mnount less than A .
C.

PROPOSAL

Consequently, it seens appropriate to use a simple point
estimation procedure. In this procedure, the size of the sample
will be determined by economic factors. Then the weight of theand evalestimate will 7be measured by fixing a deviation 41
uating the probability of the estimate being within a plus orminus A from the exact value.

The following is an analysis and detailed description of
the steps of the proposed method.
Selection of Lot Size IL The lot size will be determinedby the frequency at which the control is performed, and/or by
the total number of records to be processed. This decision is
not critical as long as it is over, a reasonably large number ofsampled records (not less than 1000). In general though, it isdesirable to use as large a lot size as possible.
1.

Selection of the Sample Sipe. The sample size will
mainly be determined by economic considerations, and for thispurpose a cost function is defined which includes all thevariable costs of interest:A(i)= f (m) - f (M) + f (n)
C
2.

t

1

2

3

where,
f (M) = cost function representing the cost of correcting
1

M wrong records.
M = exact number of wrong records due to the conversion
procegs.
M =Aestimate of M.
f (ND = cost function representing the discount when the2

4.

number of wrong records due to the conversion process is M.
n = sample size
f (n) = cost function representing the cost of obtaining a3

minimum sample of Size n.

It is reasonable to adopt a minimum sample size. In otherwords, the estimate will not be accepted if it is obtained usinga sample smaller than the lower limit. The first step, then, isto obtain a sample whose size is equal to the lower limit and to
estimate the number of wrong records by using a simple averageas the estimator, or(2)where m = number of defective records in the sample (records
containing at least one error).
-274-

,;

The next step is to investigate the possibility of increasing the sample size by comparing the cost of obtaining the previous
sample size with the expected saviag, which can be expressed by

S = f (M) - f
2

1

(3)

(1'41)

If this value is less than f(n), then a larger sample is not
economically justified, and Vie proceed to para. 3. If S islarger than f (n) then the sample size can be increased to some3value n' such that:(4)f (nt) = S
3

We can obtain a new estimate of the number of defective records(Mt) by using equation (2) again
M'

m'
) N
n'

(5)

where m' is the total number of defective records in the sampleof size n'.

By repeating the process of comparing sampling costs withexpected saving we can determinL whether it is justified toincrease the sample size.
It is also reasonable to set an .11.2222: limit on the sample

In other words, the sample will not be increased beyond
that limit even if it is economically feasible.
size.

A

Measure of ths Estimate M. Once we have obtained thesample size n' and the number of defective records in the sample
m, we can obtain a statistical measure of the estimate by using
a convenient sampling table.
3.

The table referenced (1) gives a relationship of fiveparameters: N, n', mt, confidence level, and the precision.
Any four of them will determine the other.
In our case we already have N, m' and n'. And from thestructure of the table, it is easier to decide on the confidence
level desired and then obtain from the table the precision.
To clarify the meaning of each of these terms, the following
equation is presented, lhowing the relationship among them:
A

P (IM

-

IM

s (Precision) 11111= --and the population is(6)N) = confidence level
-275-

D.

ILLUSTRATIVE EXAMPLE

200,000 records to be processed in 50 weeks.
Given:
Cost of correcting one wrong record = $0.10
Discount per wrong record = $0.25
Cost of sampling = $0.05/record.
It is desirable to obtain a control output every week.
1.

Solution.
Select the lot size N, which is given by:
2.

200,000 records= 4,000 records/week
50 weeks

Select lower and upper bounds for sample size, and set these
at 5% and 20% of the lot size, respectively.
Now, to perform a control for one week we proceed as follows:
By using a table of random numbers we select a sample size ofn = 200. Assume that we found 9 defective records in the sample,
then m = 9. We obtain the first estimate by using equation (2)where,a = (!) N = (A) 4000 = 180.
By using equation (3) we evaluate the expected savings:
S = 180 (25) - 180 (10) = $27.00
Evaluate f (n) = 200 (5) = $10.00
3

Comparing S and f(n) we see that it is justifiable toincrease the sample sie in order to get a better estinate.
using equation (4) we obtain a new sample size.

Byn'(5) = 27000, or n' = 540, which is only about 15% of the
lot size. We obtained 340 more records. Suppose that we obtain
15 more defective records. Then m' = 15 + 9 = 24, and our new
estimate from equation (5) will be MI= 178.
The new expected savings are then S' = $26.70, which is lessthan f (5)40) = 2700, and therefore we do not increase the sample
size.

3

Now, suppose we want to obtain the precision of the estimate
with a confidence level of 95%.
To use the table, first we calculate p, the proportion of
defective records in the sample
P =

m'

n

24
5 0

= --4

(100) = 4.4%-276-

Rounding this to the larger integer, to be on the conservative side, we have p = 5.
Using Table I, page 23 of the referenced source we findthat if n were 409 the precision would be 2%. Consequently,
we conclude that 100<M <260, with a confidence level better
than 95%.

M - .02M<M 4M.02M]

If it is desired to estimate the errors attributable toeach type of source we need only to use formula (2) with thecorresponding m's, and by using the tables obtain a measure of
the estimates.

REFERENCE
1. Brawn, R. Gene and Lawrence L. Vance. Sampling Tables for
Estimatin Error Rates or Other Pro ortions. Berkeley, Public
Accounting Research Project, Institute of Business and Economic
Research, University of California, c1961. p. 23.

- 277 -

APPENDIX VII

THE ORGANIZATION, MAINTENANCE AND SEARCH
OF MACHINE FILES

By

Ralph M. Shoffner
Institute of Library Research
University of California
Berkeley, California(Published in the Annual Review of Information
Science and Technology, v.3, edited by Carlos A. Cuadra.
137-167)
Chicago, Encyclopaedia Briticanna, Inc., 1968. pp.

UNIVERSITY OF CALIFORNIAINSTITUTE OF LIBRARY RESEARCH
Berkeley, California 94720

Reprinted from The Annual Review of Information Science and Technology, Vol. 3, 1968,
edited by Carlos A. Cuadra, by permission of
Encyclopaedia Britannica, Inc. Copyrighted 1968.

Permission to reproduce this copyrighted
material has been granted by Encyclopaedia
Britannica, Inc., to the Educational Resources
Information Center (ERIC) and to the organization
operating under contract with the Office of
Education to reproduce ERIC documents. Reproduction by users of any copyrighted material
cout'aned in documents disseminated throughthe ERIC system requires permission of the
copyright owner.

-srnlm.o..PIP.MPMPPPIPPIMPP.

i

,

i
li

II

S.

I

c.c.)eThe Organization,
Maintenance andSearch of
Machine Files'"4

sp <c\\OC2 \04r
CO

RALPH M. SHOF
41)
%OD
University of California
coBerkeley, California(<;C\

INTRODUCTION

File organization and search is concerned with the structure and operationof computer systems that store and retrieve large amounts of information.

The following are four key questions about file systans that, ideally, a
review article of the current literature should answer: How well do such
systems operate? What are the structures of file systems? What is thenature of the system operation? How does one design an appropriate
system?

Unfortunately, there are no simple answers to these questionsfirst,
because the literature contains only partial answers, and second, becauseof the great diversity among file systems and file system languages. These
two reasons give rise to a host of other reasons:
1. With many new people working in t17.c field, undefined
vocabulary increases more rapidly than strict definition reducesit.

2. File systems have to be designed to resolve contradictory goals,
such as rapid access, high capacity, low user training, and highreliability.
3. The machine system does more of the processing to organize
and group related information.
4. There is an increase in different kinds of information to beprocessed'This review was supported in part by the U.S. Office of Education Grant No. OEG-1-7-071083-5068.
21 am indebted to Luke T. Howe for his extensive belp in prepating tbis review.

S.

138

RALPH M. SHOFFNER

5. Desired modes of access to information are increasing.
6. Machineprocesses are being developed to reduce the need for,experienced specialists to act as broker between the users andthe syStems.

1. Even with a determined file structure, the performance oflarge-scale, multiple-access ffie systems can be significantly
affected by the search procedures used.
8. The expanding computer field means there are more computer
systems and programming languages with which to implement
file systems.

9. Many special-purpose systems are being developed and theirreports are being published.
W. In spite of all these trends, that Holy Grail the all-purpose data
management systemis being sought by an ever-increasing
number of people.

One evidence of the increasing interest in file structures is theestablishment by the Association for Computing Machinery of a one-day
professional development seminar in "File Structures for On-line Systems." As with the other ACM seminars, it was scheduled and presented inseveral cities throughout the United States. The material was wellorganized and the meetings were well attended. No doubt the specificmaterial will change, but seminars should certainly be continued. Anotherimportant event in 1967 was the publication of Meadow's excellent book(60) devoted to file organization. It will serve as a text around which filesystems courses can be organized. These courses will also be able to makegood use of several other recent books that devote sections to fileorganization and search (37, 72).

This review is divided into three sections: General Issues, Aspects ofFile Systems, and File Systems. The division between the last two areas issomewhat artificial, both in the sense that some of the literature must be
discussed in more than one place and in the sense that one aspect of areport is emphasized tn the exclusion of other aspects where the chosen
aspect seemed most important. However, rather than attempting to give abalanced review of individual papers, we have tried to report significant
changes occurring over the entire literature of file organization.
GENERAL ISSUES

Evaluation of Systems

Although a separate chapter of this volume (Chapter 3) is devoted toevaluation, a portion of that literature needs to be considered here becauseof its potential importance in system design. The growth of machine files

ORGANIZATION, MAINTENANCE AND SEARCH OF MACHINE FILES

139

has increased the capability for making partial searches, and it is important

to be able to determine the extent to which file structure and searchtechniques influence recall, precision, and other measures of system
performance.

Pollock (68) provides a brief but excellent review of some of themeasures which have been used for the evaluation of systems. He thendefines another, a normalized "sliding" ratio measure. Instead of having
two classes, his measure has multiple classes to which each retrieved

document is assigned according to its relevance to the query. Thisapproach should prove useful since it avoids the difficulty of having tojudge a document as completely relevant or irrelevant.
The primary measures for the evaluation of document retrieval systems
continue to be those of recall and precision. Swets (79) provides aninteresting representation of these measures, in which, by a change ofvariables, he plots an operating characteristic curve of the type used instatistics. His objective is to simplify the task of evaluation by replacing thetwo measures with a single measure of performance. This, he argues, canbe done by treating the slopes of the operating characteristic curves as
equal. If they are equal, they can of course be ignored. However, his data
do not appear to justify this treatment. That is, a different selection method
will provide the best performance, depending upon the ratio of recall to
precision one is willing to accept.
As an alternative to the use of recall and precision for the measurementof information transfer, Hayes (37) has suggested an abstract measure thattakes into account the amount of information in the search specification,
the total size of the file from which the result is selected, and the size of theresulting retrieval.
While such an approach may prove useful, the measure developed still
will not incorporate the issues of system and user time and cost. So long as
this is the case, evaluation will be largely irrelevant to the design of filesystems. Hayes recognizes this in his introduction:

The third aspect organization arises because as the file getslarge enough, it is impossible, at least uneconomical, to scan every
item in the file to judge its relevancy. It is therefore necessary to
structure the file to provide indexing tnechanisms, and to provide
intermediate measures of degree of match which are less sophisticatedthan the ultimate measure of relevancy. It is this third aspect whichreally constitutes the technical problem in information retrieval
system design, since it is here that the size of the file, the requisite
response time, the degree of selectivity, and the accuracy of the
response all interact. (p. 265)

140

RALPH M. SHOFFNER

When measures incorporating all of these aspects are developed, they
will provide a quantitative basis for file systems design. Until that time,
design will continue as a highly subjective process.
Measurement of Association

The measurement of association is central to the organization and searchof machine files. Such measure'snent is needed in order to associate filerecords with each other and to associate them with a search request.
Usually this measurement is of the match-mismatch type. That i, either agiven file record matches the search specification completely or it isconsidered a mismatch. In this case, no estimate of the relative mismatch of
the file records is made.

While most operating file systems have been based on exact match,considerable experimental work with statistical measures of association
has been performed. As a result of this work, associative measures havereached the stage where they will be used in operating file systems. Jones& Curtice (41) provide a comparison of term association measures. The

authors use a framework that encompasses most associative measures.
They define a weighting in which the measure of association of two index
terms is equal to the frequency of the joint occurrence of the terms divided
by the nth power of the frequency of one of the terms. By varying the valueof n, the behavior of the various associative measures can be approximated.
The authors provide an illustrative example, using a term taken from theNASA collection, which contains 100,000 documents and 18,000 index

terms. In addition to providing useful approximations, the authors also
show that a specific value of n in the approximation formula impiies aweighting of the relative importance of recall versus precision in a search
operation.

Salton's work (71) gives experitaental evidence of the useful aess of
associative techniques.
Specifically,

the procedures based on synonym recognition,weighted content identifiers, cosine correlation to match documents
and search requests, and document abstract processing are always
more effective than methods using simple word stem matches (without
synonym detection), nonweighted terms, correlation terms based only
on the number of matching terms, and analysis procedures which
consider only the titles of the documents being examined. (p. 1-4)
The importance of these procedures for operational systems appears to
be in the order in which they are mentioned. In general, the importance ofsynonyms is already recognized. To a limited extent, weighted index terms

.2

S.

ORGANIZATION, MMNTENANCE AND SEA RCH OF MACHINE FILES

141

are already in use. The next important step will be the incorporation ofterm pairs with associated frequency information so that requests tailoredfor recall and precision can be satisfied without significant increases insearch cost.

Research on associative measurement is continuing, with two major
areas of investigation. One is the machine analysis of text to obtain theappropriate index terms. (This work is covered in Chapter 6, on automatedlanguage processing.) The other is iterative search, in which the retrieval

results and the associative measure are used to modify either the searchrequest or the indexing provided in the file. In addition to Salton's group,groups reporting on associative adjustment systems include Jones, et al.
(42), Bryant, et al. (16, 73) and Lehigh University (2). The work reported
by Salton's group is of the greatest interest in that experimental results areprovided for techniques utilizing user feedback for the modification of
search.
ASPECTS OF FILE SYSTEMS

Logical Record Encoding

Record encoding has two separate components. The first component is the
specification of the logical content of the record. The literature concernedwith the determination of logical content is found in the preceding chapter
on document desc:iption and representation. However, a few documents
are discussed in this section because they have as a common characteristicemphasis upon the development of a hierarchical classification system forencoding the record infoimation. The second component of record
encoding is that of the representation of the record as it is held in computerstorage. This physical encoding is discussed in the next section.

Two papers report on the experimental application of the Universal
Decimal Classification (UDC) to machine search systems. Freeman &Atherton (32) report on its application to 250 documents in the field of
oceanography, and Caless & Kirk (17) report on its application to an
unknown number of seismological documents at the VELA Seismic
Information Analysis Center (VESIAC), University of Michigan. Bothpapers conclude that the UDC can be used for encoding and searching
documents. However, Freeman & Atherton point out that there is anunanswered question of whether it should be used in preference to anindexing language designed specifically for machine processing. Bothpapers give an indication of some of the difficulties in adapting UDC formachine search. Caless & Kirk indicate that extensive skills andpreparation were needed. The skills were in the subject area, in theapplication of UDC, and in library classification. Their system is highly
dependent upon a skilled staff performing the encoding both of the

142

RALPH M. SHOFFNER

documents and of the search requests.
This dependence upon skilled staff is not unique to the UDC system,
but, rather, is inherent in any highly structured representation of document
content. The proper conclusion from this is not necessarily that such

systems should be abandoned, but, rather, that when they are used,
specialized information centers such as VESIAC should be set up. In thisway, the record encoding can be performed, in all of its complexity, by
information specialists and then can be distributed and used wherever
there is need for that literature. This, of course, is the direction in whichmachine information systems have been proceeding during the last decade.

Just as many problems are encountered in the application of UDC,
many problems have been encountered in the use of classification
structures for chemical information systems. In general, the literature of
the current year represents a continuation of work reported by Tate (80).

Bowman (13) reports on the use of the Wiswesser line notation for 100,000
chemical compounds at Dow Chemical. He gives costs for their method of

encoding the chemical structure information. Starting with a legible
structure diagram and a molecular formula, only $i 76 was required tocode and check the notations and formulas for 1,000 compounds. He does
not indicate the length of training time for the encoders, nor does heindicate how familiar they were with the process prior to its operation.

Lefkovitz (48, 49, 30) compares the two main approaches to theencoding of compounds, the connection table and the line notation
methods, with respect to the requirements of a system handling approximately 3,000,000. In addition to the two basic approaches, he discusses aderivative system called the Mechanical-Chemical Code (MCC). Although
not as complicated as the others, this code could serve as a rough screen in
an on-line retrieval system.

Two reports of the continuing work at Canadian Industries, Ltd.,

appeared during the year (39, 59, 82). The objective of this work has been

the automatic generation of a connection table from the Wiswesser
notation. This table is to be used in place of the original notation forinternal storage, search, and display purposes.

A study at Chemical Abstracts Service is concerned with automatictranslation from one representation to another. This one is concerned withthe conversion of traditional systematic nomenclature. Vander Stouw,
et al. (86) provide a preliminary discussion of this study, which shonld
produce some generally useful results.
Bobka & Subramaniam (10) report on the development of a chemicalcoding system, called Medical Coding System (MCS). The system, which
was developed in the Comparative Systems Laboratory of Case Western
Reserve's Center for Documentation and Communication Research, is

ORGANIZATION, MAINTENANCE AND SEARCH OF MACHINE FILES

143

based upon a classification of chemicals by the type of units or groups
present. It does not record the sequence in which the functional units
appear; rather, each group is coded individually without reference to otherparts of the molecule. No comparison with other notation schemes is made,and it is not clear why a different system was felt to be necessary forencoding the compounds. Presumably some of the experimental andoperating systems that are capable of incorporating more than one type ofnotation system will provide some of the comparative informationnecessary to clarify the advantages and disadvantages of the various
notational schemes (38, 87).
Physical Record Encoding

Character Encoding. Physical encoding is concerned with the binary
representation use d to encode the data within the computer system. Most
systems have used encoding schemes in which characters are the units thatare independently represented. For alphanumeric data the most common
coding systems r..re the well-known Binary Coded Decimal (BCD), and theAmerican Staudard Code for Information Interchange (ASCII) six-bit andeight-bit codes. Morena & McLean (64) recommend an Information

Processing Code (IPC), an eight-bit code constructed so that subsets ofseven, six, five, and four bits can be derived from it. Wnile defined as aneight-bit code, the eighth bit is not used, and thus it is the seven-bit subsetthat is presented in the paper. The intention of the authors is not to replace
the function of ASCII as an information interchange standard; rather, it is
to use IPC for internal information storage, particularly in on-line systems.
As the authors indicate, the internal information processing and systemuser interface requirements of on-line systems are of sufficient importanceto warrant consideration of specially designed codes to satisfy them. As thenumber and size of on-line file systems increase, further attention to theinternal codes can be anticipated.
Word Encoding. Whereas a character-by-character representation of
nonnumeric information is likely to be the most desirable for manipulation

within the central processing unit and for output over the various
peripheral units, the increasing use of mass storage will foster thedevelopment of coding schemes that require less storage than character-by-

character encoding. As is generally known, the encoding of natural

language is not compact because our words use only a small proportion of

the very large number of possible combinations of the 26 letters of thealphabet. One approach to obtain a more compact representaiion has been
to develop systems that encode word-by-word rather than character-bycharacter. These are ofzen referred to as code-comi. ression systems.
Although information theory provides a sound basis for the development

144

RALPH M. SHOFFNER

of efficient coding structures, the file systems literature reveals little use ofthis theory. Bemer (6) devotes his paper to the use of compressed codes toreduce the cost of long-line communication of natural language. He showsthat if the codes could be generated properly, variable-length coding would
be the most efficient because it can take advantage of the frequency of useof words in the natural language. By this approach, the codes arecompressed to 35 % of the space required for a character-by-character
representation. Although he does not discuss the compression technique,he apparently assumes that all programs and tables fit into the computer's
main memory.

Bemer estimates total coding and decoding time at 250 microseconds
pee word, on an IBM 7090. Using a cost of $800 per hour, he computes thetotal conversion cost to be $0.0056 per word. While the estimates weremade for a different purpose, they can be used to indicate the potential ofcode compression for file system storage. In the interval since this work
was done, the cost of equivalent processing has been reduced by approximately 90%. Assuming that encoding and decoding times are roughly
equivalent, encoding siv..1.4k1 now cost approximately $0.0003 per word.

If the annual cost of the mass storage is $6 per thousand words, then acode-compression strategy that reduces a space to less than half of theoriginal wouk: provide a storage cost reduction of more than $3 perthousand words per year. Obviously, retranslation and other requirements

must be considered in order to determine the effectivimess of code
compression for file systems. Even so, it is clear that systems with mass
random-access storage could benefit from attention to code compression.
In sonvz. cases, non-unique codes may be useful in file systems. In thisarea, Bourne & Ford (12) report experimental work showing the degree of
non-uniqueness as a function of the length of code generated. The analyses
were performed on files of 2,082 index terms and of 8,207 student names.
The transformations used were variant methods of selecting letters to be
dropped from the source word. The letters were chosen either on the basisof position or on the basis of frequency of occurrence. They show thatsavings in spacein an index, for examplecan be achieved without agreat loss of uniqueness. Marron, et al. (58, 28) describe COPAK, a
three-component code-compression system. One component, NUPAK,
converts fixed-point data to a compact form. Another, ANPAK, compresses alphanumeric data, either the output from the first component or
any other fixed string. The third component, IOPAK, provides a final
compression of the strings before output to a storage medium. At present,
NUPAK and ANPAK have been implemented.
ANPAK is the component of greatest interest for file systems. ANPAK
operates repetitively on the string of data and removes repeating elements

ORGANIZATION, MMNTENANCE AND SEARCH OF MACHINE FILES

145

from the string. The generated code contains all information necessary fordecoding without loss of original information. The authors give an
example of the use of ANPAK on text material for which character-by-character encoding required 15,180 bits. This was encoded in 9,291 bits,
representing a compression to 61 % of the original size. No indication of
the processing requirements is given. However, the authors assert that suchcompressior is economically feasible, since read-in time for compressed
information plus time for decompression is significantly less than read-intime for the original information.
Field Encoding. Beyond the representation of the information on acharacter or word basis, the next problem is that of mapping the logicalfields of the records into inWrnal storage. Benner (7) provides a design
technique for mapping logical records into physical records of fixed size.
By this method, a common program can perform all direct-access filehandling regardless of the specific content or size of the logical records.
This approach encounters the problem of determining the size of a physicalrecord and the fields of a logicai record that it should contain. Benner's
technique deals quantitatively with this issue in terms of the lengths of thecontrol versus the data portion of the fields, the activity of the fields, andthe distribution of field lengths. These variables are used to balance storage
utilization against access. From this balancing, the design of the record isestablished. The author describes the results achieved using this techniquein the design of a bus:ness information system and estimates storage
utilization to be 65 %. This is the expectation that a character of storage
will be occupied by a character of useful information from the data base(excluding control information). The length chosen for the physical record
was 796 bytes. The ratio of file processing time to application program
execution time is 1:1.2. The system operates with a mean inquiry responsetime of 10 seconds.
Graphic Data Encoding. The rapid development of peripheral equipment providing input and output of data in graphical form has stimulated
interest in the development of data structures for the storage of graphicalinformation. As Van Dam & Evans (85) indicate, data structures forstoring line drawings have been at one of two extremes. They have been
either descriptions suitable for direct output to a specific display device, or
hierarchical, interconnected list structures. The authors provide anotherlist structure, but one with less structure provided within the data. They
achieve this by defming a set of primitives that operate on the data todevelop the more complex structure. These primitives constitute the
Pictorial ENCodIng Language (PENCIL), major portions of which havebeen implemented on an IBM 7040. In PENCIL, the unit of data is a
picture composed of points, lines, and other pictures. A picture is

146

RALPH M. SHOFFNER

represented by a Control Item and associated Line Items, Text Items, andInformation Items. Through the use of PENCIL primitives, one can define

or establish new data in the form of points, lines, etc. One can also
manipulate the data to move lines and points on the screen and canperform affine transformations. Control can be used to clear working
storage, assign names to pictures, retrieve pictures from storage, display
their on an output device, delete components of the picture, etc.
The PENCIL approach is particularly interesting because it is integrated

within the more general MULTILANG system at the University ofPennsylvania. Though MULTILANG is experimental, such mixed datasystems soon will be common. Therefore, file system designers must begin
to give more attention to the storage and mtrieval of graphical information
as well as of alphanumeric information.
The Logical Grouping of Records

To facilitate retrieval, it is desirable to group records that have the same or

related content. Where the content of a record is identified by a set of
assigned index terms, a common method of grouping the records is to list
them under each assigned index term. For systems of this type, Zunde,et al. (94) define the distribution of index terms assigned to documents tomaximize information transmission. On the basis of information theory
argument, they obtain an equation for the desirable number of records towhich a term is assigned as a function of the average number of records
assigned per term for the entire file. This distribution is then comparedagainst the distribution of terms in two systems. Although one would
expect a uniform assignment of terms to be the most efficient, they do not
explain the reason this does not occur.
Long, et al. (53) have analyzed the rate at which the number of words ina dictionary increases with the amounts of text analyzed. Their analysis ispart of an attempt to develop a set of keywords for the indexing ofradiological patient records. They have ercountered difficulty in the use ofword-rank order to establish access in that, beyond the first several
hundred words, about 70% of the remaining words occur with a frequencyof less than one in 10,000. They anticipate continuing the study todetermine whether filter techniques can be developed to give (approxiniately) 2,500 words with which to characterize the file.
In a variation of analyzing text in order to determine appropriate terms,

Armitage & Lynch (4) present the development of articulated subject
indexes by manipulating phrases. They show that for subject entries from
Chemical Abstracts, at least half of the articulated terms (i.e., a main termwith a subhead) can be generated from more general phrases by the use of
their rules. They have also applied the technique to 479 abstracts from

ORGANIZATION, MAINTENANCE AND SEARCH OF MACHINE FILES

147

Documentation Abstracts, and they conclude that an index can be developed from the titles of these abstracts.
Taken together, these papers show that a limited analysis of text can beused to determine the logical grouping of similar records in diverse subject
areas.

Beyond the grouping of records that have individual index terms incommon, it is possible to group records based upon a determination ofgeneral similarity. One of the continuing studies is carriM on at the
Cambridge Language Research Unit. Sparck Jones & Jackson (75, 76)
report on the current status of the Unit's clump-finding investigations.

Clustering is another term that has been used to describe grouping
procedures. Ide, et al., and Grauer & Messier (in Salton, 71, same as 26)'
report on the current state of the clustering programs under investigation

at Cornell. These programs are based upon the clustering algorithm
developed by J. J. Rocchio. This clustering algorithm is an alternative to
the Cambridge Language Research Unit's clumping approach. It is being

tested on the same data base used in the clumping investigation:
Cleverdon's collection of 1,400 documents from the Cranfield project.
From this, a comparison of the other two approaches should be possible.
A different approach to the grouping of records is that provided by adocument's citations. These citations provide linkages between records
that can be used to infer similar content. Part of the current work iS
concerned with the processing of common citations to infer similar
content. Thus, the objective is the same as that of clustering, but a different
kind of linkage information is used. Chien & Preparata (22, 69) use graphtheory to define a procedure for grouping documents by their "distance." It
is not known whether any test has been made of the algorithm developed.
However, it would be most useful to apply the Chien-Preparata algorithm
to the index terms of the Cleverdon documents in order to compare themwith the methods of clustering and of clumping.
The citation approach is based upon the processing of "citing/cited by"relations that exist between records in a file. Levien & Maron (52) havegeneralized this approach in their discussion of a system for inference
processing. In this system, relations that exist between records in the fileare processed to derive information not explicitly contained in the file.
Although a wide range of problems remain to be studied, their approach isimportam because it reduces the necessity to establish explicitly all of therecord groupings in the file structure.

Two papers are concerned with the grouping of records representing
chemical compounds. Uchida, et al. (84) are concerned with the evaluationof fragmentation codes, linear ciphers and atom-by-atom topological codes.
Particularly, they have been concerned with a system capable of retrieving

'1

ii

Ili

148

RALPH M. SHOFFNER

material having common substructures. They discuss the results of 28
searches in a flie containing 841 alkaloids and their derivatives. Annitage,et al. (3) report on the development of algorithms to detect similarities
among chemical compounds. Their purpose is to determine the largest
connected set of atoms and bonds common to any pair of chemical
structures. As investigations such as these proceed, the concepts ofsimilarity of chemical compounds should be developed enough to allow
grouping of compounds with similar substructures. This grouping will
reduce considerably the required search effort.
File Structure

Given that the records and the logical grouping of these records have been
defined, the task remaining is to obtain a machine file to map this logicalstructure into the physical structure of the computer system. In contrast torecords that are characteristically free in form with many partial connections, the physical records of a computer system are normally fixed andrigid. Thus, for example, the physical records of a computer system's massstorage have a given number of bits for each physical record. In thissituation, it is most unlikely that the logical records of the external world
will match the physical records of the computer system. Thus, computer
programs must be provided to translate the records from one form to theother. The term "file structure" refers to the methods by which these
programs retain the logical records and their groupings within the rigid
system structure.
Two major concerns are reflected in thc current literature. One is thedevelopment of more general, and more complex, file structures. The otheris the analysis of the effect of structure on access time.

Most of the current structure development involves variants of astructure in which each logical record contains the information needed togenerate the address of the next logical record linked to it. Such structures
have been referred to as structured files, threaded-list files, multi-lia files,

etc. Gray (35) provides a brief review of many of the list-structure

approaches that have been used in computer-aided design. Ihay i nclude
the sketch-pad ring structures, the coral-ring structures, and numerous
others. This is a most useful paper because it shows the many similaiities
between these various approaches.
A number of specific list-linking techniques have been reported on
during the year. Gabrini (33) reports on an application of Multi-list, alinked-list file system developed at the University of Pennsylvania, to theProject TIP file of Physical Review articles. Ross (70) describes the AED
free storage package, which is part of the AED-1 complier system. The
package is general in that the blocks may be of any size and the programs

S.

ORGANIZATION, MAINTENANCE AND SEARCH OF MACHINE FILES

4

149

provide several different strategies for maintaining storage accounting.
Storage accounting and the reassembly of released storage into usable
units is generally a problem in such systems. Haddon & Waite (36) discuss
their procedure for reallocating variable-length records in order to freespace that was previously unallocatable because of its distribution between
these records. They describe the general procedure and give times for its
use on an English Electric KDF9. As described, the procedure is applied
to the fast memory of the central processing unit. Its extension to massstorage is not discussed.
As list-processing languages make increasing use of mass storage, they
become more like general processors using list-linked structures. Two
papers related to list processing are concerned with the allocation of data
to mass storage or to fast memory. Cohen (24) discusses the results of aprogram on an IBM 7044 with disc storage that allocates the records on thebasis of their frequency of utilization. Bobrow & Murphy (11) discuss asimilar application that utilizes a DEC PDP-1 with a drum memory. Bothconclude that frequency can be effectively used to allocate space.
Just as both of these approaches were heavily influenced by the
MULTICS system, so too was the system described by Barron, et al. (5).
They summarize the file handling facility that is provided on the Titan
computer at the University Mathematical Laboratory, Cambridge. In thissystem, files are maintained over indefinite periods on a three-level systemhaving core, disc, and magnetic tape. Access is provided on the basis of thename of a desired logical block of informationi.e., a group of records.
The system maintains necessary indexes tot provide the physical records
associated with the name, and to reallocate the physical records among theperipheral storage units.
Morenoff & McLean (63) discuss multi-level storage organization in amore abstract fashion. They suggest the definition of levels of the file interms of the accessibility characteristics of the organization. On this basis,
they obtain something around 30 to 40 levels, with access times fromnanoseconds to minutes or hours. They suggest that the data have homeaddresses in the system that may be specified by the users to provide thedesired accessibility. Beyond this, the statistics of frequency of use could
be used to make blocks of information more accessible and thus make theoperation more efficient. Whether this approach has been implemented is
not indicated.
Several papers have been concerned with the quantitative effect of thefile structure on the expected average access time to obtain records fromthe file. Lowe (54) has written a very good doctoral dissertation in which
he characterizes the average access time in terms of the number of index
S.

terms per physical record that can be obtained in an inverted file

150

4

RALPH M. SHOFFNER

organization, as opposed to a linked-list structure. He then extends his
consideration to the effects of truncating the index terms so that the fullindex term does not necessarily appear within one physical record. Thislatter portion of his work has been published as a separate paper (55).
Thompson, et al. (81) have produced a paper closely related to Lowe's,in which they are concerned with the number of logical branches that need
to be included at a decision point to minimize search time. Although they
deal entirely with logical records, the limitation of the number of branches

that can be provided at a given decision point is a function also of thephysical record size.
Leimkuhler (51) takes a different approach. He considers the probabilityof use of the logical records and organizes the file into zones of roughly
equivalent probabilities, in order to minimize the access time. He proposesa distribution function for the spread of useful material through a file, andhe suggests that empiricai information indicates the desirability of a
two-level organization for scientific; literature. in this organization, 15 to
20% of the most useful documents would be examined first and would
provide a success probability of approximately 0.67. The search time isapparently considered to be that required for reading and reviewing the
documents retrieved rather than the time for machine retrieval of the
document references. The additional effort that would be rewired toretrieve all document references at the same time is not considered.
Although a number of papers relevant to file structure have been
published during the year, the most needed paper has not appeared. That
paper will provide a unifying quantitative model of file structure that will
cover the range from inverted file to linked-list structures and iclate thestructures to space, search, and update requirements.
File Search

As was indicated earlier, a measure of association is necessary both for
determining the logical grouping of the records and for matching thoserecords with requests. In general, the match criterion specified has been an
exact match of the logical combination of the specified values of the fields
and subfields of a record. Brandhorst (14) has suggested a method of using
term weights to bypass the use of Boolean logic. The determination ofproper relative term weights appears fairly straightforward and theprogramming for such an approach reasonably simple. First, the proper
Boolean specification is set up. Then arbitrary weight values are assigned
to search terms, and the minimum total weight any document must achieve
in order to become a "hit" is specified. The following are two examples(the plus denotes union and is read as "or," while the dot denotes
intersection and is read as "and"):ORGANIZATION, MAINTENANCE AND SEARCH OF MACHINE FILES

151(A + B) + (C D) A = 2, B = 2, C = 1, D = 1, WEIGHT Minimum = 2(A + B) (C D) A = 1, B = 1, C = 2, D = 2, WEIGHT Minimum = 5
Brown (15) reports on the use of a retrieval system with approximately
60,000 compounds in operation at Eli Lilly. While there are some logicalcombinations of search terms that cannot adequately be reflected in theweighting technique, such as a union of intersections (A B) + (C D),
the technique may still be useful for operating retrieval systems.
The translation of the request from the user's language into the structure

required for the search system is a continuing problem of search
specification. Much work has been put into developing systems to handle
the natural language of the system user. Since the general problem has
been so intractable, the current emphasis is upon the use of limited subsets
of natural language.

Kellogg (43, 44) and Tonge (83) are both concerned with limited
processes for translating from the user's language to the system's internal
search control. Both are concerned with the search of relational data files
and both have arrived at an approach by which the user is provided withthe translation of his request for his approval or modification prior tosearch. Kellogg's system, CONVERSE, is being constructed at System
Development Corporation (SDC) under the time-sharing system implemented on the IBM Q-32. It appears that the portion of the CONVERSE
program concerned with the translation of the question to the searchcontrol is in operation. It is not clear whether searches are being driven bythecontrol information obtained. Tonge's work appears to be inapproximately the same stage of development. It will be interesting tofollow both approaches as they develop.

A problem usually encountered in systems using a tiubset of natural
language is the tendency of the user to put in additional words that are
unnecessary and not defmed in the system. It is desirable to have thefacility within the system to ignore such words. Easy English, underdevelopment at the Moore School and reported by Cautin, et al. (19),removes noise words and questions the user when it encounters words thatit cannot recognize. Such facility will be most important for the infrequent
users of the console system.

Closely related to the specification of the search is the strategy ofcarrying out the search. Where probabilistic or associative indexing isutilized, the opportunity exists for modifying the search request on thebasis of the results obtained. Another search of the file can then be madewith the new request. One o' te most interesting programs in this area isthat of Salton's group (71) in vAtich the measure of association is the cosineof the angle between the vectors represented by the index terms assigned

152

RALPH M. SHOFFNER

to the request and the file record. The subsequent automatic adjustment of

the retrieval request vector is achieved by modifying the vector withanother vector chosen to maximize the difference between the retrieved
documents that the user felt to be relevant and those that he deemed
irrelevant. The results to date indicate that it is possible to develop
procedures for controlled search in a file with associative indexing. Thus,

one does not have to obtain ever-larger amounts of material in anever-larger sphere around the original request. As file sizes increase and as

associative techniques are used more, greater attention will be given to
search strategies and their relation to the objectives of the ffle system.
List-Oriented Programming Languages

As file systems deal with the variability in nonnumeric records, more
complex data structures are required. As complex structures are incorporated into general-purpose systems, list-processing techniques will haveto be incorporated into the system in order to provide satisfactory
manipulation of these records.
Foster (30) has written a very good monograph on list processing. He
discusses the various aspects of these techniques: the data representation,
the operators available in the system, and special features such as "garbage
collection." This brief monograph is particularly useful because of his useof ALGOL as a single language in which to define the list-processing
operations. He also discusses some of the well-known list-processing
languages, such as IPL-V, LISP and COMIT.

List-processing systems have had a significant impact on standard
computer languages. Lawson (47) discusses list processing in PL/i. While
the facility in PL/1 will certainly be most useful, it is well to note thecaution with which Lawson concludes his summary: "The user should be
careful to understand the structure and content of all list elements, since hemay inadvertently reference a value which has differing characteristics

from the based variable declaration." This warning is particularly apt,
since the user is developing his own list-processing macros out of the basic

PL/1 language. Thus, he must maintain the caution appropriate to any
systems programming.

The development of new list-processing languages is continuing.
Blackwell (8) provides a brief discussion of a new list-processing language,
LOLITA. This language is being implemented in the Culler-Fried system,
which uses two keyboards and a display scope connected to a BunkerRamo 340 computer. Many of the list-processing operations are controlled
by the console operator through a set of function keys. Blackwell discusses
the various function keys available to the operator. With these function
keys he can define, load, store, or concatenate list symbols. The major

ORGANIZATION, MAINTENANCE AND SEARCH OF MACHINE FILES

153

difference between LOLITA and other languages, such as LISP or IPL-V,
is the availability of these function operators to the man at the console. It

appears that the language structure is similar to an elementary CAI
language, but with the addition of a number of special-purpose algebraic
operators to make the system amenable to computational applications.
The investigation of list-processing techniques is continuing at the
Moore School of Electrical Engineering. Carr & Gray (18) report on thecurrent status of this work, in which they are atternnting to extend thelimits of list processing in several directions. Among the issues they are
investigating are: the Growing Machine, a relocatable software systemwith simple procedures for addition; an "all-pushdown" computer; and asoftware system to produce software to specification. This project is quite
ambitious, and it should produce results of importance to file systems.
FILE SYSTEMSThe previous section reviewed the contributions that have been made to
individual aspects of file systems. This section reviews the operating filesystems on which there have been reports during the past year.
The section is divided into two partsData Management Systems andSpecial-Purpose Systems. Data management systems are those designed tobe general purpose in the sense that they can be used to maintain andsearch files with many different types of information in different filestructures. Special-purpose systems are those developed for specificpurposes, even though they may have some generality. The discussion ofthese systems is divided into two parts, dealing, respectively, withchemical information systems and other systems.
Data Management Systems

Most of the literature of the current year consists of reports on data
management systems that have been previously announced. 01 le (66, 67)
has written two review articles in which he discusses GIS, IDS, INFOL,
and TDMS. Though brief, they serve as a good introduction for those whowish to know how these systems relate to each other.
The Time-Shared Data Management System (TDMS) is continuing

under development at SDC (91). This system, an outgrowth of the
TSS-LUCID system, was discussed by Minker & Sable (61) and by Bleier(9). The latter article provides information on the data structure definitionof TDMS. Data structure definition is critical to a data management
system, since it determines the amount of structural variation the data canhave and implies the corresponding processing effort requinxi. From
Bleier's article, the TDMS data structure appears to be quite flexible fromthe point of view of the definition of the individual logical record. Variably

154

RALPH M. SHOFFNER

appearing fields and subfields can be maintained and searched with nodifficulty. Also, it appears th.it variable-length fields can be handled. With
respect to the logical grouping of records, indexes referred to as
concordances can be maintained as specified by the user. However,
Bleier does not discuss the structure of this concordance or the method of

specifying which elements of a logical record will be included inconcordances.
Williams & Bartram (93) discuss another aspect of TDMS, that of report

generation. The report generation programiscalled COMPOSE/

PRODUCE. The COMPOSE phase of the program is used for the on-line
development of the report specification. The PRODUCE phase of the
program obtains the information from the report files and produces thereport desired. This phase of the process is carried on with little or nointeraction with the user. From the paper it appears that the capabilities of
the report generator are standard. The significant difference between it andtraditional report generators lies in the user interaction in the COMPOSE
portion of the program. TI-ere is little information about the nature of thisinteraction beyond the statement that there has been an attempt to use a
restricted subset of English af, the control language for the console user. In
the examples given, this language appears to be very much like COBOL in
the English it uses. There is no indication whether the sophisticated user
can bypass the verbosity requIred by such an approach.
Another group of SDC personnel have reported upon the problems ofstatistical routines for data analysis in an on-line system. Shure, et al. (74)call these routines TRACE, for Time-Shared Routines for Analysis,

Classification, and Evaluation. This system is implemented on the Q-32
Time-Sharing System. As the authors indicate, such on-line general
analysis programs will be of considerable importance to the researcher
who is attempting to perform on-line analysis of very large quantities of
information such as that derived from experiments in the behavioral
sciences. As a result, it would seem most desirable to incorporate thisfacility into TDMS.
In last year's Annual Review, Minker & Sable discuss several other data
management systems that were in the developmental stage, including
DM-1, ADAM, and COLINGO. Dixon & Sable (29) provide a general
discussion of DM-1 and its current capabilities. They do not indicate thestage of development of DM-1, but their discussion in the present tense
implies that DM-1 exists. In providing an overview of the system, they
discuss briefly the languages used, the organization of the indexes needed
to provide for record definition and maintenance, the types of program
tasks that can be carried out, and the search capability of the system. The
structure of the indexing provided is such that records with variable-length

ORGANIZATION, MAINTENANCE AND SEARCH OF MACHINE FILES

155

fields and nested sub-fields can be specified to the system. Linkage ofseparate records can also be performed. Thus, for example, the specification of the vendor in an inventory record might be made by the vendor
number. This provides a reference to a separate vendor record in which thevendor's name, address, etc., are provided. Programs would then accessboth records, to retrieve needed information. To indicate the flexibility ofthe structure, it appears that the Multi-list structure could be duplicatedusing this facility.
It is interesting to compare the index structure used in DM-1 with theindex structure provided in TDMS. On the surface they are very similar, inthat each goes through several decoding stages to convert a field name to acode representing the position of that field and then to a record. To control

the highly variable data structure, every field in a record is separately
indexed so that access to the records in the file can be obtained via any ofthe logical fields defmed.
In DM-1, it seems that there is no way to define certain fields as reportfields only. If so, there would be an unnecessary expansion caused byincluding these fields in the indexes.
Connors (25) provides an initial discussion of some of the experienceand results with ADAM (Advanced Data Management), the MITRE

Corporation's experimental general information processing system. Char
& Foreman (21) provide a final report of the experiments utilizing ADAM.
These articles are refreshingly candid, and the system planner should readthem carefully before embarking on the development of a data management system.
One of the central problems mentioned is that a generalized system
requires a very sophisticated user if some ridiculously excessive machinetimes are to be avoided. Connors indicates these requirements: "The user
must be aware of the implications of how his data is structured; he must
understand and control the optimization." He then goes on to give an
example of a trivial but actual user-implemented problemto multiply
price by quantity in order to obtain total price. Unfortunately, as a result of
the combination of the data organization and the file generation statements
he used, the fields were placed into different files and more than one hourof running time was consumed on the IBM 7030 before the procedure waskilled. Connors points out that the user can specify a different data
structure that would make this search and computation faster, but that hemight at the same time be making the procedure more difficult for otheruses of the information. Only the user is in the position to estimate what
the mix of his processing is going to be, and therefore only he can structure
his control of the system. All of this is based upon the assumption that heknows how the system will perform in response to the processes that he

156

RALPH M. SHOFFNER

specifies, and thus can control the system to make it effective in response
to his requirements.
Char & Foreman also encountered the problem of the sophisticated user.
Plans were developed for the installation of a remote station at
Headquarters, Air Force Logistics Command. This terminal was going to
be used so that all committee members, after training, could be making
their own queries and interrogations of the system.
Although a number of attempts were made by most members of thecommittee at writing their own queries, they were successful at only

the simplest queries, and then quite often made mistakes ofpunctuation or in typing. In nearly all cases, the mission personnel
were inclined to turn their more complex qucries over to a committee
member who was a programmer and had mastered the Fable language
and remote equipment to a greater degree . . . some mission personnel, as a matter of principle, did not believe it should be their function
to learn a programming language, and others who would try did nothave the time and/or patience to master the intricacies of the language

and equipment, and became discouraged after repeated failure inattempting to use the English like syntax query language. (p. 52)

When taken out of context these remarks may seem very negative withrespect to the capability of the ADAM system. However, the ADAM
system is reasonably capable as a data management system. In addition,

the experiment was well run, and its findings do indicate a need forcontinued development of data management systems. There are, however,serious problems in the proper use of such systems, and further controlled

experimentation with them is needed to provide guidelines for theirdevelopment and use.
Another line of activity at MITRE has been the development of AESOP(An Evolutionary System for On-line Processing). Summers & Bennett(78) provide a final report on this prototype interactive system. AESOP isan experimental system implemented on the IBM 7030 (STRETCH)
computer and using CRT consoles. Although the article describes AESOP
as a data-base-oriented system, it is concerned with the terminal control
aspects of the system, display, editing, printing, and programming. The
artic!,,: gives no information about the system's data management capability.

Two new data managemeht systems implemented on IBM 1400-seriessystems, have been reported, along with additional reporting on two earlier
systems. The new systems are C-10 and GIPSY (Generalized Information
Processing System). The two older systems are MADAM (Moderately

ORGANIZATION, MAINTENANCE AND SEARCH OF MACHINE FILES

157

Advanced Data Management) and CFSS (Combined File Search System).

Steil (77) describes work that was originally intended to implement
MITRE's COLINGO (now called C-10) as a new version on the 1410. He
gives a very good history of the sequence in development objectives andthe changes in direction that took place as a result of their fmdings. For
example, they decided to program the system in a LISP-like interpreter.

When they began debugging their programs, they found that theperformance was off by several orders of magnitude. As a result, what
began as the development of another file management system became anexperiment to improve the performance of the USP-like language. The
paper is of interest because the author chooses to discuss some of thethings that were done wrong as well as those things that were done right.
GIPSY is being used by the International Atomic Energy Agency in
Vienna. Del Bigio (27) has developed a program manual for the system,
which was developed to process bibliographic infonnation. Because of thisoriginal objective, it has certain rigidities in terms of the way that fields are
identified. However, it does include the capability for variable-length fields
and repeating fields, and it permits specification of the fields that will serve
as indexes to the records. While a program manual is not meant for casual

reading, any who are interested in implementing an IBM 1400 file
management system would benefit from this document,

Freeman & Atherton (32) report on the application of another file
management system for the IBM 1400 to the problem of bibliographic
records. Specifically, they are concerned with organizing and searching
document files utilizing CFSS, previously discussed by Climenson (23).
Although the primary interest of the authors is the use of the Universal
Decimal Classification for retrieval purposes, they indicate briefly themethod by which the file was mapped into CFSS and the characteristics ofthe system that made it beneficial to use for the experimental system thatthey set up. CFSS is being reprogrammed for the IBM 360 by Service
Bureau Corporation. Its availability should be checked for the next review
article.
Franks (31) has provided a nontechnical article on the characteristics ofthe MADAM system. He indicates that MADAM is now available on the

IBM 360/30, but does not indicate whether this is through emulation or
reprogramming.

Given the development of on-line data management systems and theproliferation of 1401-based systems, what else could possibly be done in
the development of general systems? Vinsonhaler (89, 90) has written asystem in FORTRAN IV. His system, BIRS (Basic Indexing andRetrieval System), is used primarily for document control. The system isorganized to utilize a 32,000-word core machine and six tape drives. The

1 58

RALPH M. SHOFFNER

description given is for the implementation on a CDC 3600 at Michigan
State University. This system provides for access to the file by index keys.
Such access can be on the basis of providing a printed listing of indexes, orby automated search of the file. BIRS/II has been released for national
distribution. In addition to the capabilities of BIRS/I, it will include theability to handle coordinate and weighted indexing and will incorporate
sophisticated relevance-searching systems. Finally, it will include wordroot analysis and a synonym dictionary for improved searching capability.

While the inherent problems of tape-oriented systems for machine
searching have been well documented, there are at the same time many
applications in which such systems al e extremely useful. In this situation,
the BIRS/II system will provide a rapid method for a person to try out thismethod of information control on his own files.
Special-Purpose Systems

Chemical Information Systems.

Several of the organizations that aredeveloping chemical information systems have been mentioned in theearlier description of the representation of chemical compounds. Hoffman(38) discusses the chemical structure storage and search system now
operating at Du Pont. Climenson (23) discussed an early version of thissystem. To provide access to the basic documents of the file, there are four
separate files: the compound file, the general term file, the thesaurus file,and the fragment file. Chemical structures are stored in a separate registry
file, which is used for topological substructure search. Hoffman gives more
recent information on the file characteristics, size, growth rate, the cost ofdata preparation, and the cost of file search operations. The registry files

contain 69,000 compounds, and the total input cost per compound is 88
cents. While the computer cost for searching is, of course, a function of thenumber of questions per batch, the indications are that for searches of the
nonpolyrner registry file the cost is $55 (for 55,000 nonpolymers). The
incremental cost per search question is estimated to be $12. At the present
time, the system is implemented on an IBM 7010 with 1301 disc. During
1967 a major study of file organization was undertaken to lead to thestorage of their files on direct access devices in preparation for on-line
searching. No report is available yet.
Registration and searching of chemical compounds is performed using
conventions that define a connection number and oxidation state for each
item. Polymers are described in terms of significant repeating units andend g rou p s. Characters in each registry record indicate the presence orabsence of information relating to the compound in the document systemfiles.

Van Meter, et al. (87) describe an experimental chemical information

ORGANIZATION, MAINTENANCE AND SEARCH OF MACHINE FILES

159

system developed under the Army Chemical Information and Data
Systems Organization. The goal of the work is practical: to develop asystem to satisfy the needs of the Army for cJ:emical information and data.

The input for the system consists of 230,000 chemical compounds fromthree separate filesthe Toxicological Information Center of Edgewood
Arsenal (TOXINFO) file, the Chemical-Biological Coordination Center(CBCC) file, and the Chemical Abstracts Service (CAS) file. The contentsof the system include thf:, molecular formula, structural formula images,
connection table representation of structural formulas, CIDS registry
number, nomenclature, liWature references, file code, and file registry

number. The paper is heavily oriented toward the logical aspects ofrepresenting information and the nature of the searches that are possible asa result of the logical information included in the file. The physical aspectsof the operation are not included. The authors indicate that these aspects
will be discussed in a forthcoming report on the system.
Matthews & Thomson (59) dicuss briefly a COBOL-organized system
to carry out weighted coordinate term searfth utilizing an index stored onmagnetic tape. A weighted swch Is based on selecting a group of searchterms that express each concept ol the inquiry, assigning each a weightindicative of its relative importance, and computing a "score" for answers
resulting from combinations of these terms. While it may appear that thissearch method is similar to that proposed by Brandhorst, that is not theintention. Rather, the logic is a conjunction of the terms, and the weighting

indicates the relative importance the user attaches to these individual
terms. This system has been used :o retrieve patents from the Information

for Industry patent file. It is repoited that "A typical question with ten
terms having an average frequency of posting would require 12 minutes on
an IBM 1410 (40K) computer, for which current charges are $15. This time
assumes that the inquiry is one of about five inquiries which have beenprocessed together." Such information would be more meaningful if it
indicated also the size of the tape file over which the searches are being
processed.
Other Systems. Kessler (45) gives an overview of the on-line aspects ofproject TIP (Technical Information Program) at M.I.T. There are now
60,000 articles, covering 32 of the physics journals, in the system. Half are
on disc. and half are on tape ready to go to disc. Access over the system is
to citations backed up by two microfilm-based printout facilities for accessto the documents. Through Project MAC, the experiment has a wide range

of users because access to the IBM 7094-based system is through 150
on-line keyboards.

Anderson, et al. (1) discuss the experimental literature system that isavailable for reference retrieval experimentation at Lehigh University's

It

160

RALPH M. SHOFFNER

Center for the %formation Sciences. The file at the present time contains
2,500 document references in the information sciences. These art beingtransferred to disc, and on-line search programs are being set up to operate
through GE Data Net-15s and Model 33 Teletypes. The access will be toboth serial and inverted files, which will be maintained on the GE 225. The

search programs will be capable of both exact match and associative
searching. The conversation routine for specifying a search uses normal
search terms, not search codes. It appears, however, that the searches areplaced entirely in terms of conjunction of terms, without "or" logic
available.

The Information Systems Laboratory (ISL) of the Moore School ofElectrical Engineering, University of Pennsylvania, has been described by
Rubinoff (62). One of the interesting assertions made is that computer aid

must be provided not only for searching a document file but also forlibrarian-like assistance that the user may obtain when he is operating inreal time with the computer system over his on-line console. In otherwords, it must also be able to indicate to the user such information as how
the file is organized and what the meanings are of index terms that wereadopted Iby indexers at the time of indexing. The planned system will havethe following features: The user wilt have direct console access to thesystem. He will be able to obtain not only catalog and index data, but
descriptions of the system itself. He will be permitted an unrestricted
search vocabulary, and it will be the responsibility of the system tointerpret search terms, to request clarifications, or to provide meanings ofterms upon request.
Much like Project INTREX, the ISL document system is expected toincorporate many different access points, such as language, color of the
document, etc. In its first form, this system is implemented on an IBM

7040 computer with an IBM 1301 disc. A DEC PDP-8 serves as aninterface between the users and the 7040 system. This laboratory will be
an outstanding place within which to do the research that will continue theexcellen t reputation of the Moore School.

A set of reports by Magnino (56, 57) and Nelson (65) discuss anoperational system that uses text processing of input documents anddisseminates the information to 2,400 user profiles for IBM and World
Trade Corporation. The text processing is based upon the entire amount oftext available in machine form. In most cases this is an abstract of 200 to
300 words, plus title and other bibliographic data. It appears that searchingis based upon matching the text of the documents with the precise searchterms provided by the user. This requires the user to set up hll of thealternative terms that he wishes to have used in the search.
From an input of 12,500 documents, more than 500,000 abstl acts were

ORGANIZATION, MAINTENANCE AND SEARCH OF MACHINE FILES

161

sent out during 1966, to approximately 1,400 users. Thus the ratio of
notifications per document per user was .030, meaning that the averageuser received abstracts of 3 % of the documents announced during the year.

The 1966 data provided show that over 83 % of the notifications (on a
70% return of evaluation information) were judged by the users to be
relevant to their interests. While this judgment of relevance is extremely
different from the relevance determined when a person is making a specificsearch Yequest, it shows that the system is providing information the user
wants to see. It is most encouraging that an operation of this magnitude isproviding useful service to its users.
Two organizations that have been mentioned several times are MITRE
and Cornell University. Both organizations have made significant contribu-

tions to file organization and search, and both have embarked uponsignificant efforts in text processing in file systems. From the two
documents provided on SAFARI (20, 92), the present concerns ofinvestigators are for text analysis and they have not attempted toincorporate search and retrieval capabilities. By contrast, Salton's work at
Cornell is well advanced. The SMART system is a proven research vehicle

within which the text processing is being incorporated. In the past,
excellent work has been provided by the many organizations in the field,
and it is likely that as they enter new areas of inquiry, they will similarly
make further significant contributions.
CONCLUSION

Because this has been a review of a single year's literature, it has necessarily covered only small portions of many ongoing projects. As a result,
it is difficult to isolate a single "significant event" for the year. Nevertheless, it is possible to derive an implied significance for the file system literature of 1967 by projecting what should occur within the next three years.
All programs in information processing education will incorporate atleast one course devoted to the creation and maintenance of file systems.
Quantitative models of file systems will incorporate characteristics of both
structure and use and will be used to predict the following: response time,storage requirement, processing effort, and transaction capacity. Finally,
operational file systems will commonly utilize statistical analysis, such asthat of the co-occurrence of indexing terms, to improve their retrieval
performance.
In three years, the principal research concern will be user interaction
with on-line graphic display file systems. As a result of its dynamic aspect,
this research will necessitate further work on the issues of structure, searchstrategy, quantitative analysis, and, of course, education. Overall, ourunderstanding of file systems is reasonably good. The work, in fact, is162

RALPH M. SHOFFNER

more extensive and better than this reviewer anticipated. There are now anumber of strong individuals and groups concerned with file systems.

From this, we can anticipate that knowledge and application will be
extended rapidly.
REFERENCES(1) ANDERSON, RONALD R.; AMICO, ANTHONY F.; GREEN, JAMES S.
Experimental retrieval systems studies. Report no. 2; Systems manual for experimental
literature collection and reference retrieval system. Center for the Information

Sciences, Lehigh University, Bethlehem, Pa., 15 April 1967, 59 p. (AFOSR-724-65)(AD-652 279)

(2) ANDERSON, RONALD R.; KASARDA, ANDREW J.; REED, DAVID M.
Experimental retrieval systems studies. Report no. 3. Center for the Information
Sciences, Lehigh University, Bethlehem, Pa., 15 April 1967, 88 p. (NSF-GE-2569)(AD-653 280)

(3) ARMITAGE, JANET E.; CROWE, J. E.; EV ANS, P. N.; LYNCH, M. F.;McGUIRK, J. A. Documentation of chemical reactions by computer analysis of
structural changes. Journal of Chemical Documentation, 7 (November 1967) 209-21 5.

(4) ARMITAGE, JANET E.; LYNCH, MICHAEL F. Articulation in the generation ofsubject indexes by computer. Journal of Chemical Documentation, 7 (August 1967)
170-178. Presented at the 153rd Meeting of the American Chemical Society, Divisionof Chemical Literature, Miami, Fla., 9-14 April 1967.

(5) BARRON, D. W.; FRASER, A. G.; HARTLEY, D. F.; LANDY, B.; NEEDHAM,
R. M. File handling at Cambridge University. In: AFIPS Conference Proceedings, vol.
30;

1967 Spring Joint Computer Conference, Atlantic City, N.J., 18-20 April.

Thompson, Washington, D.C., 1967, p. 163-167.

(6) BEMER, R. W. Do it by the numbersdigital shorthand. Communications of theACM, 3 (October 1960) 530-536.

(7) BENNER, FRANK H. On designing generalized file records for management
information systems. In: AFIPS Conference Proceedings, vol. 31: 1967 Fall Joint
Computer Conference, Anaheim, Calif., 14-16 November. Thompson, Washington,
D.C., 1967, p. 291-303.

(8) BLACKWELL, FREDERICK W. An on-line symbol manipulation system. In:
Proceedings of 22nd National Conference, Association for Computing Machinery.
Thompson, Washington, D.C., 1967, p. 203-209.
(9) BLEIER, ROBERT E. Treating hierarchical data structures in the SDC Time-Shared
Data Management System (TDMS). System Development Corp., Santa Monica, Calif.,

29 August 1967, 23 p. (SP-2750) Also published in: Proceedings of 22nd National
Conference, Association for Computing Machinery. Thompson, Washington, D.C.,
1967, p. 41-49.

(10) BOBKA, MARILYN E.; SUBRAMANIAM, J. B. A computer oriented scheme forcoding chemicals in the field of biomedicine. Center for Documentation andCommunication Research, School of Library Science, Case Western Reserve University, Cleveland, Ohio, July 1967, 22 p. (Comparative Systems Lab. Technical report no.
1I)(11) BOBROW, DANIEL G.; MURPHY, DANIEL L. The structure of a LISP systemusing two-level storage. Bolt Beranek and Newman Inc., Cambridge, Mass., 4
November 1966, 26 p. (Report no. Scientific-6) (AFCRL 66-774) (AD-647 601) Also
published in: Communications of the ACM, 10 (March 1967) 155-159.

(12) BOURNE, CHARLES P. FORD, DONALD F. A study of methods for systematically abbreviating English words and names. Journal of the Association for Computing
Machinery, 8 (October 1961) 538-552.

(13) BOWMAN, CARLOS M.; LANDEE, FRANC A.; RESLOCK, MARY H. A

ORGANIZATION, MAINTENANCE AND SEARCH OF MACHINE FILES

163

chemically oriented information storage and retrieval system. I: Storage andverification of structural information. Journal of Chemical Documentation, 7 (February1967) 43-47.

(14) BRANDHORST, W. T. Simulation of Boolean logic constraints through the use ofterm weights. American Documentation, 17 (July 1966) 145-146.
(15) BROWN, WILLIAM F. A matrix information storage and retrieval system utilizing an!BM-360, Model-30 Computer. In: Proceedings of the 30th Annual Meeting of theAmerican Documentation Institute, New York. October 1967. Thompson, Washington,
D.C., 1967, p. 36-40.

(16) BRYANT, EDWARD C.; SEARLS, DONALD T.; SHUMWAY, ROBERT H.;
WEINMAN, DAVID G. Associative adjustments to reduce errors in document
screening. Westat Research, Inc., Bethesda, Md., 31 March 1967, 78 p. (Final report
no. 66-301) (AFOSR 67-0980) (AD-65I 630)
(17) CALESS, T. W.; KIRK, D. B. An appiication of UDC to machine searching. Journalof Documentation, 23 (September 1967) 208-215.
(18) CARR, J. W., III; GRAY, H. J.; et al. List processing research techniques. Third
quarterly report, December 1966April 1967. Moore School of Electrical Engineering,

University of Pennsylvania, Philadelphia, September 1967. 120 p. (Moore School
Report 68-03) (Technical report ECOM 02377-3)(19) CAUTIN, HARVEY; LOWE, THOMAS C.; RAPP, FREDERICKA; RUBINOFF,
MORRIS. An experimental on-line information retrieval system. University ofPennsylvania, Moore School of Electrical Engineering, Philadelphia, April 1967,
107 p.; 1967, 20 p.

(20) CHAPIN, P. G.; GROSS, L. N.; NORTON, L. M.; BELLER, R. J.; BROWNE, C. T.
SAFARI, an on-line text-processing system user's manual. MITRE Corp., Bedford,
Mass. March 1967, 34 p. (Information System Language Studies no. 15) (MTP-60)(MITRE Project 1108)(21) CHAR, BEVERLY F.; FOREMAN, ALLING C. Joint AFLCIESDIMITRE Advanced Data Management (ADAM) Equipment. MITRE Corp., Bedford, Mass.,
February 1967, 124 p. (Final report no. MTR-285) (ESD TR-66-330) (AD-648 226)
(22) CHIEN, R. T.; PREPARATA, F. P. Topological structures of information retrieval
systems. University of Illinois, Coordinated Science Lab., Urbana, October 1966, 19 p.
(Report no. R-325) (AD-642 501)
(23) CLIMENSON, W. DOUGLAS. File organization and search techniques. In: Annual
review of information science and technology. Carlos A. Cuadra, ed. Interscience, New
York, 1966, vol. I, p. 107-135.
(24) COHEN, JACQUES. A use of fast and slow memories in list-processing languages.
Communications of the ACM, 10 (February 1967) 82-86.
(25) CONNORS, THOMAS L. Software concerns in advanced information systems. In:
Walker, Donald E., ed. Information system science and technology. Papers prepared
for the Third Congress. Thompson, Washington, D.C., 1967, p. 395-398.
(26) CORNELL UNIVERSITY. DEPARTMENT OF COMPUTER SCIENCE. Informa-tion storage and retrieval. Scientific report no. ISR-12 to the National Science

Foundation. Reports on evaluation, clustering, and feedback. Gerard Salton, Project
Director. Ithaca, N.Y., June 1967, 1 vol. (various pagings)(27) Del BIGIO, G. GIPSY: Generalized Information Processing System. Program manual.
Internal publicption of International Atomic Energy Agency, Vienna, Austria, 16 May
1967, 92 p.

(28) DeMAINE, P. A. D.; KLOSS, K.; MARRON, B. A. The SOLID system. II: Numericcompression. III: Alphanumeric compression. National Bureau of Standards, Washington, D.C., 15 August 1967. (Technical note 413)
(29) DIXON, P. J.; SABLE, J. DM-I: A generalized data management system. In: AFIPS
Conference Proceedings, vol. 30; 1967 Spring Joint Computer Conference, Atlantic
City, N.J. 18-20 April. Thompson, Washington, D.C., 1967, p. 185-198.
(30) FOSTER, J. M. List processing. American Elsevier, London and New York, 1967,
54 p. (Macdonald computer monographs)

164

RALPH M. SHOFFNER(31) FRANKS, E. W. The MADAM System: data management with a small computer.
System Development Corp., Santa Monica, Calif., 8 September 1967, 16 p. (SP-2944)(AD-658 472)

(32) FREEMAN, ROBERT; ATHERTON, PAULINE. File organization and searchstrategy using the Universal Decimal Classification in mechanized reference retrieval
systems. American Institute of Physics, UDC Project, New York, 15 September 1967,
30 p. (Report no. AIP/UDC-5) (PB-176 152)(33) GABRINI, PHILIPPE J. Automatic introduction of information into a remote-accesssystem: a physics library catalog. University of Pennsylvania, Moore School ofElectrical Engineering, Philadelphia, I November 1966, 79 p. (Technical Report, no.
67-09) (AD-641 564)(34) GRAUER, ROBERT T.; MESSIER, MICHEL. An evaluation of Rocchio's clustering
algorithm. In: Cornell University. Dept. of Computer Science. Information storage andretrieval. Report no. ISR-I2 to NSF. Gerard Salton, Director. Ithaca, N.Y., June 1967,
sec.-6 (39 p.)(35) GRAY, J. C. Compound data structure for computer aided design; a survey. In:
Proceedings of the 22nd National Conference, Association for Computing Machinery.
Thompson, Washington, D.C., 1967, p. 355-365.
(36) HADDON, B. K.; WAITE, W. M. A compaction procedure for variable-length storage
elements. Computer Journal, 10 (August 1967) 162-167.

(37) HAYES, ROBERT M. A theory for file organization. In: Karp lus, Walter J., ed.
On-line computing: time-shared computer systems. McGraw-Hill, New York, 1967, p.
264-289.

(38) HOFFMAN, WARREN S. An integrated chemical structure storage and searchsystem operating at Du Pont. In: American Chemical Society. Abstracts of papers,
154th Meeting, Chicago, 10-15 September 1967. Washington, D.C., 1967, 16 p. (Sec.
G-Div. of Chem. Lit., paper 15)(39) HYDE, E.; MATTHEWS, F. W.; THOMSON, LUCILLE H.; WISWESSER, W. J.
Conversion of Wiswesser Notation to a connectivity matrix for organic ccmpounds.
Journal of Chemical Documentation, 7 (November 1967) 200-204.
(40) IDE, ELEANOR; WILLIAMSON, R.; WILLIAMSON, D. The Cornell programs forcluster searching and relevance feedback. In: Cornell University. Dept. of Computer
Science. Information storage and retrieval. Report no. ISR-I2 to NSF. Gerard Salton,
Director. Ithaca, N.Y., June 1967, sec.-4 (13 p.)(41) JONES, PAUL E.; CURTICE, ROBERT M.. A framework for comparing term
association measures. American Documentation, 18 (July 1967) 153-161.
(42) JONES, PAUL E; GIULIANO, VINCENT E.; CURTICE, ROBERT M. Papers on

automatic language processing. Arthur D. Little, Inc., Cambridge, Mass., February
1967, 3 vols. (ESD TR-67-202, vol. 1-3) Vol. I: Selected collection statistics and data
analyses. (AD-649 073); Vol. 2: Linear models for associative retrieval. (AD-649 038);
Vol. 3: Development of string indexing techniques. (AD-649 039)(43) KELLOGG, CHARLES H. CONVERSEa system for the on-line description andretrieval of structured data using natural language. System Development Corp., Santa
Monica, Calif., 26 May 1967, 16 p. (SP-2635)(44) KELLOGG, CHARLES H. On-line translation of natural language questions into
artificial language queries. System Development Corp., Santa Monica, Calif., 28 April
1967, 47 p. (SP-2827)(45) KESSLER, M. M. The "on-line" technical information system at M.I.T.Project TIP.
In: 1967 IEEE International Convention Record. Institute of Electrical and Electronics
Engineers, New York, 1967, part 10, p. 40-43.
(46) LAMB, SIDNEY M.; JACOBSEN, WILLIAM H., JR. A high speed large capacity
dictionary system. Mechanical Translation (MT), 6 (November 1961) 76-107.
(47) LAWSON, HAROLD W., JR. PL/I list processing. Communications of the ACM, 10(June 1967) 358-367.

(48) LEFKOVITZ, DAVID. A chemical notation and code for computer manipulation.
Journal of Chemical Documentation, 7 (November 1967) 186-192.

ORGANIZATION, MAINTENANCE AND SEARCH OF MACHINE FILES

165(49) LEFKOVITZ, DAVID. The impact of third generation ADP equipment on alternative

chemical structure information systems. Presented at the I53rd Meeting of theAmerican Chemical Society, Division of Chemical Literature, Miami, Fla., 9-14 April
1967, 31 P.

(50) LEFKOVITZ, DAVID. Use of a nonunique notation in a large-scale chemical

information system. Journal of Chemical Documentation, 7 (November 1967) 192-200.
(51) LE1MKUHLER, FERDINAND F. A literature search model. [Abstract] Bu/letin of

the Operations Research Society of America, 15, supplement 1 (Spring 1967) p.

B56-57. (PB 174 390)
(52) LEVIEN, R. E.; MARON, M. E. A computer system for inference execution and data
retrieval. Communications of the ACM, 10 (November 1967) 715-721.
(53) LONG, JOHN M.; BARNARD, HOWARD J.; LEVY, GERTRUDE C. Dictionary
buildup and stability of word frequency in a specialized medical area. American
Documentation, 18 (January 1967) 21-25.
(54) LOWE, THOMAS C. Design principles for an on-line information retrieval system.
University of Pennsylvania, Moore School of Electrical Engineering, Phil. lelphia.
December.1966, 136 p. (Technical Report no. 67-14) (AFOSR 67-0423)(AD-647 196)(55) LOWE, THOMAS L. Direct-access memory retrieval using truncated record names.
Software Age, I (September 1967) 28-33.
(56) MAGNINO, JOSEH-I J., JR. IBM's unique but operational international industrial

textual documentation systemITIRC. Presented at 33rd Conference of FID andInternational Congress on Documentation, Tokyo, 12-22 September 1967. (Preprint, 9p. and appendix)(57) MAGNINO, JOSEPH J., JR. Information technology and management science. Paper
presented at the Institute of Management Sciences, 14th International Meeting, Mexico
City, August 1967. (Preprint, 8 p. and appendix)(58) MARRON, B. A.; DeMAINE, P. A. D. Automatic data compression. Communications of the ACM, 10 (November 1967) 711-714.
(59) MATTHEWS, F. W.; THOMSON, L. Weighted term search: a computer program for
an inverted coordinate index on magnetic tape. Journal of Chemical Documentation, 7(February 1967) 49-56.
(60) MEADOW, CHARLES T. The analysis of information systems; a programmer's
introduction to information retrieval. Wiley, New York, 1967, 301 p. (Information
Sciences Series)(61) MINKER, JACK; SABLE, JEROME. File organization and data management. In:

Annual review of information science and technology. Carlos A. Cuadra, ed.
lnterscience, New York, 1967, vol. 2, p. 123-160.

(62) MOORE SCHOOL OF ELECTRICAL ENGINEERING. The Moore School

Information Systems Laboratory. Morris RubinolY (principal investigator). Universityof Pennsylvania, Moore School of Electrical Engineering, Philadelphia, May 1967,
28 p. (AFOSR 67-1952) (AD 657 809)
changing to a(63) MORENOFF, EDWARD; McLEAN, JOHN B. Application of level
of
the
ACM,
10
(March1967)
multilevel storage organization. Communications
149-154.
(64) MORENOFF, EDWARD; McLEAN, JOHN B. A code for non-numeric information
processing applications in online systems. Communications of the ACM, 10 (January1967) 19-22.
(65) NELSON, PAUL J. User profiling for normal text retrieval. In: Proceedings of the
30th Annual Meeting of the American Documentation Institute, New York, October1967. Thompson, Washington, D.C., 1967, p. 288-295.
data(66) OLLE, T. WILLIAM. Generalized systems for storing structured variable lengthand retrieving information. Presented at IFIP/FID Conference on Mechanized
Information Storage and Retrieval, Rome, Italy, 14-17 June 1967. (Preprint, 18 p.)(67) OLLE, T. WILLIAM. IDS and GIS: Chalk and untasted cheese. Prepared for
publication in: Newsletter of the ACM Special Interest Committee in Business Data
Processing, 28 September 1967. (Preprint, 7 p.)

If

166

RALPH M. SHOFFNER(68) POLLOCK, STEPHEN. Measures for the comparison of information retrieval
systems, and the normalized sliding ratio. [Abstract] Bulletin of the Operations
Research Society of America, 15, supplement I (Spring 1967) p. B-57.

(69) PR EPARATA, F. P.; CHIEN, R. T. On clustering techniques of citation graphs.
University of Illinois, Coordinated Science Lab., May 1967, 25 p. (Report no. R-349)(AD-652 593)
(70) ROSS, DOUGLAS "I. The AED free storage package. Communications of the ACM,
10 (August 1967), 481-492.

(71) SALTON, GERARD. The SMART Projectstatus report and plans. In: Cornell
University. Dept. of Computer Science. Information storage and retrieval. Report no.
ISR-I2 to NSF. Gerard Salton, Director. Ithaca, N.Y., June 1967, sec.-1 (12 p.)(72) SCHECTER, GEORGE, ed. Information retrieval
critical view. Based on Third
Annual Colloquium on Information Retrieval, 12-13 May 1966, Philadelphia, Pn.
Thompson, Washington, D.C., Academic Press, London, 1967, 282 p.

(73) SHUMWAY, R. H. On the expected gain from adjusting matched term retrieval
systems. Communications of the ACM, 10 (November 1967) 722-725.

(74) SHURE, GERALD H.; MEEKER, ROBERT J.; MOORE, WILLIAM H., JR.
TRACETime-shared Routines for Analysis, Classification and Evaluation. In:
AF1PS Conference Proceedings, vol. 30; 1967 Spring Joint Computer Conference,
Atlantic City, N.J., 18-20 April. Thompson, Washington, D.C., 1967, p. 525-529(75) SPARCK JONES, KAREN; JACKSON, DAVID. Current approaches to classification and clump-finding at the Cambridge Language Research Unit. Computer Journal,
10 (May 1967) 29-37.
(76) SPARCK JONES, KAREN; JACKSON, DAVID M. The use of the theory of clumps

for information retrieval. Report on the O.S.T.1.-supported project. Can bridge
Language Research Unit, Cambridge, England, June 1967, vol. (various pagings)(M.L. 200)(77) STEIL, GILBERT P., JR. File management on a small computer: the C-10 System. In:
1

AFIPS Conference Proceedings, vol. 30: 1967 Spring knit Computer Conference,
Atlantic City, N.J., 18-20 April. Thompson, Washington, D.C., 1967, p. 199-212.

(78) SUMMERS, J. K.; BENNETT, EDWARD M. AESOPa final report: a prototype
on-line interactive information control system. In: Walker, Donald E., ed. Information
system science and technology. Papers prepared for the Third Congress. Thompson,
Washington, D.C., 1967, p. 69-86.
(79) SWETS, JOHN A. Effectiveness of information retrieval methods. Bolt Beranek andNewman Inc., Cambridge, Mass., 15 lane 1967, 47 p. (AFCRL-67-0412)(80) TATE, F. A. Handling chemical ccavounds in information systems. In: Annual
review of information science arid teclutology. Carlos A. Cuadra, ed. Interscience, New
York, 1967, vol. 2, p. 285-309.

(81) THOMPSON, DAVID A.; BENN1GSON, LAWRENCE; WHITMAN, DAVID.
Structuring information bases to minimize user search time. In: Proceedings of the 30th
Annual Meeting of the American Documentation Institute, New York, October 1967.
Thompson, Washington, D.C., 1967, p. 164-168.

(82) THOMSON, LUCILLE H ; HYDE, E.; MATTHEWS, F. W. Organic search anddisplay using a connecdvity madix derived from Wiswesser Notation. Journal ofChemical Documentation, 7 (November 1967) 204-209.
(83) TONGE, FRED M. A simple scheme for formalizing data retrieval requests. RAND
Corp., Santa Monica, Calif., May 1957, 31 p. (Report no. RM-5150-PR) (AD-652 201)(84) UCHIDA, H.; KIKUCHI, T.; HIR.AYAMA, ;:. Mechanized retrieval system fororganic compounds An evaluation of the fragmentation code system. In: 33rd
Conference of FID and International Congress on Documentation, Tokyo, 12-22
September 1967. Abstracts. [Tokyo], 1967, 13 p.

(85) Van DAM, ANDRIES; EVANS, DAVID. A compact data structure for storing,
retrieving and manipulating line drawings. In: AFIPS Conference Proceedings, vol. 30;
1967 Spring Joint Computer Conference, Atlantic City, N.J., 18-20 April. Thompson,
Washington, D.C., 1967, p. 601-610.

N,L

ORGANIZATION, MAINTENANCE AND SEARCH OF MACHINE FILES

167(86) VANDER STOUW, G. G.; NAZNITSKY, I.; RUSH, J. E. Procedures for converting
systematic names of organic compounds into atom-bond connection tables. Presented

at the I53rd Meeting of the American Chemical Society, Division of Chemical

6J

Literature, Miami, Fla., 9-14 April 1967.
(87) VAN METER, CLARENCE T.; LEFKOVITZ, DAVID; POWERS, RUTH V. An
experimental chemical information and data system. Status report, JanuaryDecember
1966. University of Pennsylvania, Philadelphia, January 1967, 220 p. (Report no.
CIDS-4) (Contract DA-18-035-AMC-288 (A)) (AD-657 575)

(88) VENNER, FRANK H. On designing generaiized file records for management
information systems. In: AFIPS Conference Proceedings, vol. 31; 1967 Fall Joint
Computer Conference, Anaheim, Calif., 14-16 November. Thompson, Washington,
D.C., 1967, p. 291-303.

(89) VINSONHALER, JOHN F. BIRS: a system of general purpose computer programsfor information retrieval in the behavioral sciences. American Behavioral Scientist, 10(February i967) 12, 21-24.
(90) VINSONHALER, JOHN F. BIRS: a system of general purpose computer programsfor information retrieval. Learning Systems Institute, College of Education, Michigan
State University, 9 February 1967, 19 p. (Papers of the Institute #39, revised)(91) VORHAUS, ALFRED H.; WILLS, ROBERT D. The Time-shared Data Management
System: a new approach to data management. System Development Corp., Santa
Monica, Calif., 13 February 1967, 11 p. (SP-2747)(92) WALKER, DONALD E. SAFARI, an on-line text-processing system. In: Proceedingsof the 30th Annual Meeting of the American Documentation Institute, New York,
October 1967. Thompson, Washington, D.C., 1967, p. 144-147.
(93) WILLIAMS, WILLIAM D.; BARTRAM, PHILIP R. COMPOSE/PRODUCE: A
user-oriented report generator capability within the SDC Time-shared Data Management System. In: AFIPS Conference Proceedings, vol. 30; 1967 Spring Joint Computer
Conference, Atlantic City, N.J., 18-20 April. Thompson, Washington, D.C., 1967, p.
635-640.

(94) ZUNDE, PRANAS; ARMSTRONG, FRANCES T.; STRETCH, TERRANCE T.
Evaluating and improving internal indexes. In: Proceedings of the 30th Annual
Meeting of the American Documentation Institute, New York, October 1967.
Thompson, Washington, D.C., 1967, p. 86-89.