[GRAPHIC] \WP14CVR.GIF
                  MEMBERS OF THE FEDERAL COMMITTEE ON
                        STATISTICAL METHODOLOGY
                              (June 1986)
                       Maria E.Gonzalez (Chair)
                    Office of Management and Budget
Barbara A. Bailar                                    William E. Kibler
Bureau of the Census                             National Agricultural
                                                    Statistics Service
Yvonne M. Bishop                                          David Pierce
Energy Information                               Federal Reserve Board
Administration
Edwin J. Coleman                                         Thomas Plewes
Bureau of Economic Analysis                 Bureau of Labor Statistics
John E. Cremeans                                             Jane Ross
Office of Business Analysis             Social Security Administration
Zahava D. Doering                                   Wesley L. Schaible
Defense Manpower Data Center                Bureau of Labor Statistics
Daniel E. Garnick                                       Fritz Scheuren
Bureau of Economic Analysis                   Internal Revenue Service
Terry Ireland                                         Monroe G. Sirken
National Security Agency                    National Center for Health
                                                            Statistics
Charles D. Jones                                     Thomas G. Staples
Bureau of the Census                    Social Security Administration
Daniel Kasprzyk                                      Robert D. Tortora
Bureau of the Census                             National Agricultural
                                                    Statistics Service
                                PREFACE
The Federal Committee on Statistical Methodology was organized by OMB
in 1975 to investigate methodological issues in Federal statistics.
Members of the committee, selected by OMB on the basis of their
individual expertise and interest in statistical methods, serve in
their personal capacity rather than as agency representatives.  The
committee conducts its work through subcommittees that are organized
to study particular issues and that are open to any Federal employee
who wishes to participate in the studies.  Working papers are prepared
by the subcommittee members and reflect only their individual and
collective views.
The Subcommittee on Statistical Uses of Microcomputers in Federal
Agencies organized a one-day workshop held on April 24, 1985.  This
working paper is based on the workshop and discusses four topics:
planning to buy and use microcomputers for statistical purposes;
electronic data dissemination; applications of microcomputers; and
expert systems.  The report is intended to provide helpful guidance to
Federal agencies in purchasing and using microcomputers for
statistical purposes.
The Subcommittee on Statistical Uses of Microcomputers in Federal
Agencies was chaired by Terry Ireland of the National Security Agency,
Department of Defense.
                    MEMBERS OF THE SUBCOMMITTEE ON
              USES OF MICROCOMPUTERS IN FEDERAL AGENCIES
                         Terry Ireland*, Chair
                       National Security Agency
Ken Berkman                                             Michael Leszcz
Bureau of Economic Analysis                   Internal Revenue Service
Jay Casselberry                                              Tom Nagle
Energy Information Administration             Internal Revenue Service
Frederick J. Cavanaugh                                   Ronald Steele
Bureau of the Census                  National Agricultural Statistics
Service
Lawrence H. Cox                                          Peter Stevens
Bureau of the Census                        Bureau of Labor Statistics
Richard Engels                                   Linda Bouchard Taylor
Bureau of the Census                          Internal Revenue Service
Maria E. Gonzalez* (ex officio)                             Mark Winer
Office of Management and Budget        Office of Management and Budget
*Member, Federal Committee on Statistical Methodology
                           ACKNOWLEDGEMENTS
The idea of a workshop as a focal point for proceedings on Statistical
Uses of Microcomputers was suggested by Maria Gonzalez, Chairperson of
the Federal Committee on Statistical Methodology.  She also provided
contacts in many Federal agencies, which made possible a broad Federal
participation in the workshop.
The planning of the workshop was done by the Subcommittee.  Four
topics were selected for the sessions of the workshop.  The
chairpersons designated by the Subcommittee organized each session.
They were:
                                                           Chairperson
Session on Planning                                      Lawrence Cox,
                                                  Bureau of the Census
Session on Electronic Data                                Ken Berkman,
Dissemination                              Bureau of Economic,Analysis
Session on Applications                                 Ronald Steele,
                                                 National Agricultural
                                                    Statistics Service
Session on Expert Systems                               Terry Ireland,
                                              National Security Agency
The proceedings were prepared by the chairpersons and rapporteurs of
each session based on input from the speakers.  The Subcommittee
thanks all the speakers in the workshop for their participation.
Terry Ireland, who chaired the Subcommittee, and Norman Glick edited
the final report.
Linda Taylor ably handled all the organizational and administrative
details of the workshop the real basis for a very smooth-running
conference.
                                 -iii-
             FEDERAL COMMITTEE ON STATISTICAL METHODOLOGY
            WORKSHOP ON STATISTICAL USES OF MICROCOMPUTERS
                          IN FEDERAL AGENCIES
                            April 24, 1985
                           TABLE OF CONTENTS
                                                                  Page
Preface. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . i
Members of the Subcommittee on Statistical Uses of
Microcomputers . . . . . . . . . . . . . . . . . . . . . . . . . . .ii
Acknowledgments. . . . . . . . . . . . . . . . . . . . . . . . . . iii
Introduction.  MARIA E. GONZALEZ, Office of
Management and Budget. . . . . . . . . . . . . . . . . . . . . . . . 1
Session on Planning. . . . . . . . . . . . . . . . . . . . . . . . . 3
     Summary. Prepared by FREDERICK J. CAVANAUGH,
     Bureau of the Census. . . . . . . . . . . . . . . . . . . . . . 3
     Introduction. LAWRENCE H. COX, Bureau of the
     Census. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5
     The Census Bureau Microcomputer    Information
     Center. RONALD SWANK, Bureau  of the Census . . . . . . . . . . 6
     The National Security Agency Personal Computing
     Information Center. KATHY SCHNAUBELT, National
     Security Agency . . . . . . . . . . . . . . . . . . . . . . . .11
II
     Use of Microcomputer Technology at the Bureau of
     Labor Statistics. PETER STEVENS, Bureau of
     Labor Statistics. . . . . . . . . . . . . . . . . . . . . . . .13
     Discussion. LAWRENCE H. COX, Bureau of the
     Census. . . . . . . . . . . . . . . . . . . . . . . . . . . . .23
     Questions,and Answers . . . . . . . . . . . . . . . . . . . . .25
Session on Electronic Data Dissemination . . . . . . . . . . . . . .29
     Summary. Prepared by JAY CASSELBERRY, Energy
     Information Agency. . . . . . . . . . . . . . . . . . . . . . .29
     Use of Microcomputer Disks to Disseminate
     Information. STUART WEISMAN, National
     Technical Information Service . . . . . . . . . . . . . . . . .29
Cendata:  Development and Implementation.
     BARBARA ALDRICH, Bureau of the Census . . . . . . . . . . . . .34
     Electronic Dissemination of Perishable
     Information. ROXANNE-WILLIAMS, U.S.
     Department of Agriculture . . . . . . . . . . . . . . . . . . .38
     Questions and Answers . . . . . . . . . . . . . . . . . . . . .40
Session on Applications. . . . . . . . . . . . . . . . . . . . . . .45
     Summary. Prepared by THOMAS NAGLE, Internal
     Revenue Service . . . . . . . . . . . . . . . . . . . . . . . .45
                                 -iv-
Spreadsheets and Statistical/Econometric
     Applications in Econometric Research. LINDA
     P. ATKINSON, U.S. Department of Agriculture . . . . . . . . . .46
Spreadsheets and Data Base Applications Used by
     the Crop Reporting Board in Reviewing Survey
     Indications and Preparing Publications. GARY
     NELSON, U.S. Department of Agriculture. . . . . . . . . . . . .50
Manager's Perspective on the Acquisition and Use
     of Microcomputer-Based Graphics Packages.
     RICHARD W. HAYS, Internal Revenue Service . . . . . . . . . . .51
Current Applications of UNIX-Based Microcomputer
     Systems. BRIAN CARNEY, U.S. Department of
     Agriculture . . . . . . . . . . . . . . . . . . . . . . . . . .54
Equipped for the Future? PAUL DOBBINS, U.S.
     Department of the Treasury. . . . . . . . . . . . . . . . . . .56
Concerns About Data Integrity, Security, and
     Accessibility in an Environment Where
     Microcomputers and Mainframes Are Interfaced.
     DICK SHIVELY, U.S. Department of Agriculture. . . . . . . . . .58
Questions and Answers. . . . . . . . . . . . . . . . . . . . . . . .61
Session on Expert Systems. . . . . . . . . . . . . . . . . . . . . .67
     Summary. Prepared by NORMAN GLICK, National
     Security Agency . . . . . . . . . . . . . . . . . . . . . . . .67
Introduction. TERRY IRELAND, National Security
     Agency. . . . . . . . . . . . . . . . . . . . . . . . . . . . .69
Expert System Tutorial. GEORGE LAWTON, Army
     Research Institute. . . . . . . . . . . . . . . . . . . . . . .70
An Extension of Statistical Software to Expert
     Systems. JAMES J. FILLIBEN, National     Bureau of
Standards. . . . . . . . . . . . . . . . . . . . . . . . . . . . . .78
Editing and Imputation. BRIAN GREENBERG, Bureau
     of the Census . . . . . . . . . . . . . . . . . . . . . . . . .85
Discussion. MARK WINER, Office of Management and
     Budget. . . . . . . . . . . . . . . . . . . . . . . . . . . . .93
     Questions and Answers . . . . . . . . . . . . . . . . . . . . .94
Appendix. Announcement of Workshop on Statistical
     Uses of Microcomputers in Federal Agencies. . . . . . . . . . .97
                                  -v-
                             INTRODUCTION
          Maria E. Gonzalez, Office of Management and Budget
A subcommittee of the Federal Committee on Statistical Methodology
organized a one-day workshop on statistical uses of microcomputers in
federal agencies.  The purpose of the workshop was to share
information among federal agencies on the statistical uses of
microcomputers.
About 200 persons from federal agencies attended the workshop.  The
audience had an opportunity to ask questions and make comments in the
discussion period of each session.  All were acquainted with the uses
of microcomputers.  Some were also responsible for the planning of
statistical uses of microcomputers in their agencies.  The
announcement of the workshop is included in the Appendix.
Four topics were discussed at this workshop.
1.   Planning of Statistical Uses of Microcomputers.  The first
session described three microcomputer information centers in federal
agencies.  The purpose of personal computer (PC) information centers
is to familiarize the agency users with the PC potentialities.  This
session focused on planning#, implementation, and evaluation within
federal agencies of statistical uses of microcomputers.  The main
questions asked were: Who should have microcomputers? For what
purposes should microcomputers be used? In what configurations? At
what costs? How will microcomputers coexist with central automatic
data processing services?
2.   Electronic Data Dissemination.  This session dealt with different
data dissemination methods.  The discussion covered each agency's
approach to data dissemination and the problems encountered in
implementation.
3.   Applications of Microcomputers. This panel discussion focused on
the usefulness and weaknesses of microcomputer software and operating
systems, the interface of mainframes and microcomputers, and factors
affecting data integrity, security, and accessibility.
4.   Expert Systems The methodological basis for expert systems was
discussed and several examples were given.  The examples describe
current expert systems with statistical applications.
The proceedings of this one-day workshop follow.  For each session
there is a summary, the presentations, and the discussions that
followed.
                                  -1-
                          SESSION ON PLANNING
                           SESSION SUMMARY*
The microcomputer technology of the 1980s is a personal and,
therefore, a user-oriented technology.  However, planning for
microcomputer,technology is often very complex and causes many changes
in the workplace.  Program planners must take many factors,into
account when planning the introduction of a microcomputer system into
their organization.  Three personal computer information centers were
described:
The Census Microcomputer Information Center of the Bureau of the
Census
The Personal Computer Information Center of the National Security
Agency
The microcomputer system of the Bureau of Labor Statistics
The planning, management and evaluation of microcomputer technology at
the Census Bureau officially began in 1983 with a meeting of the
Executive Staff.  Prior to that time, microcomputer technology testing
and evaluation work was ongoing at the Census Bureau, but this was the
first time that agency-wide distribution of microcomputers was
discussed.  The Census Microcomputer Information Center (CMIC) was
established as a result of this meeting.  To give greater emphasis to
the importance of microcomputer technology, the Census Bureau located
the Center in the Office of the Director with its manager reporting
directly to the Associate Director for Administration.
The purposes of CMIC are to assist employees in learning about
microcomputer technology -- both from a user point of view and a
manager/procurer point of view -- and to reduce the overall costs of
microcomputer technology purchase and maintenance.  Employees
are.given access to various brands of hardware and software to test
prior to purchasing.  They are also given "hands-on" experience in the
use of the newest in microcomputer hardware and software through
special arrangements made with the various vendors and manufacturers.
On-site training in the use of hardware and software is provided
by,outside trainers, with the divisions paying the costs for their
employees.  Costs currently range from about $100 to $125 per person
per day, which are quite favorable in comparison with commercial costs
of similar training.
The activities of the National Security Agency's Personal Computer
Information Center (PCIC) started approximately 18 months ago, when
NSA
----------------------
*Frederick J. Cavanaugh, Bureau of the Census.
                                  -3-
established the PCIC to train employees in the use of PCs and vendor-
developed software.  It did not take long to discover problems of
compatibility among various brands of microcomputers.  Therefore,
standards were established to ensure that:
1.All microcomputer systems at NSA are compatible with one another for
effective communications and portability.
2.All systems are able to function using the UNIX operating system --
again, to allow for communications and portability.
3.The microcomputer systems-are supportable; that is, they must be
easily and cheaply repaired.
4.The systems are secure, so as not to divulge secret information.
NSA has set its microcomputer standards around the IBM PC and PC/XT in
a UNIX-based environment (IBM's PC/IX) and its office automation
standards around the Wang PC.
BLS's microcomputer system is essential for efficient office
operation, and BLS has kept this in mind in designing-and developing
its system.  The BLS Executive Staff is very supportive of the
microcomputer system.
In designing the microcomputer system at BLS, several critical needs
have to be met.  These include:
     1.The need for a system that can readily provide terminal
communication with mainframe computers.
2.The need for a system capable of communicating among various
machines and those located in field offices as well.
3.The need to provide security for confidential information.
BLS undertook research and experiments to determine which
microcomputer system best met its needs.  Upon completion of the
research, a single system comprised of machines from a single
manufacturer was implemented and a set of standards was developed
around its operation and use.  The present system includes over 100
IBM PC/XTs and three Ethernet (FIPS 1O7) local area networks.
The microcomputer systems described in the presentations form a
continuum from the experimental or user-oriented approach to the more
standard production or program-oriented approach.  However, despite a
commonality of needs and objectives, each agency has chosen a
different approach to planning and managing microcomputer technology.
                                  -4-
                             INTRODUCTION
                 Lawrence R. Cox, Bureau of the Census
Welcome to the Workshop on Statistical Uses of Microcomputers in
Federal Agencies, sponsored by the Federal Committee on Statistical
Methodology.  We begin with this session on planning.
Microcomputer technology is the technology of the 1980's.  It is a
personal and, therefore, a user-oriented technology.  However, its
focus on the individual often can be misleading from a planning
perspective -- at the agency or office level, planning and managing
the use of microcomputer technology becomes very complex very fast.
While encompassing important technical issues concerned with hardware,
software and communications networks, this technology also quickly
brings the planner face-to-face with the.business of managing and
deriving improvements systematically from technological change.
Inevitably, the introduction of microcomputers into an organization
changes the workplace and the skills and orientation of workers.  It
presents new choices and often demands that these be made swiftly.
In large organizations and small offices, the following.questions must
be addressed:
     - where does microcomputer technology fit into the agency or
office?
-how should it be introduced?
-how can the organization experiment and grow with this
          technology?
-what must the organization do to plan and manage this technology
effectively?
     - should standards be set for its use? which 'standards? how
should they be set? by whom? how should they be enforced?
     - what sort of future decisions need to be made, and who should
make them?
We are fortunate today to have a panel-of experts in this field, whose
experience should shed light on answers to these and other important
questions facing the statistical program manager about to embark on
the introduction of microcomputers into his or her organization.
They speak-with the experience of individuals tasked with managing
groups assigned these responsibilities in three different Federal
agencies: the Bureau of the Census, the National Security Agency, and-
the Bureau of Labor
Statistics.
                                  -5-
The speakers are:
     - Mr. Ronald Swank, Manager, Census Microcomputer Information
Center, Bureau of the Census.
     - Ms. Kathy Schnaubelt, Chief, Information Resources And New
Technology Branch, National Security Agency.
and
     - Mr. Peter Stevens, Chief, Division of Communications and
Computing Technology, Bureau of Labor Statistics.
Until recently Kathy was Chief of NSA's Personal Computer Information
Center and had direct responsibility for the functions we will discuss
this morning.  Ron and Peter have had these responsibilities on a
continuing basis for some time.
Each speaker will make a brief presentation on how the problem of
planning the use of microcomputers was addressed in their agency.  I
will follow with a few comments by way of formal discussion, and we
will then open the floor for discussion and questions from the
audience.
          THE CENSUS BUREAU MICROCOMPUTER INFORMATION CENTER
                  Ronald Swank, Bureau of the Census
The words "microcomputer" and "personal computer" are often used in a
manner that blurs their intended use.  In the true sense of the word
"microcomputer," the Bureau of the Census has been using
microcomputers since 1968.  The FOSDIC (Film Optical Sensing Device
for Input to Computers) allowed us to film and input census forms to
computers without manual data entry.  In 1973 we attached IBM 6250
tape drives to Sperry mainframes, approximately three years.before
Sperry announced similar availability, and experimented with the ATL
automated tape library.  In 1982, eight Apple II+ personal computers
were used to do the Puerto Rican Economic and Agriculture Census data
checking and editing.  These projects established the feasibility of
using microcomputers in much of the Bureaus work.
The Bureaus organization (3500 employees at headquarters, 9000
nationwide) can best be described as 35 separate companies (divisions,
in our parlance) sharing the same resources, computers, management
services, etc.  You can imagine the problem this presents in setting
priorities, standards and general directives.  All of the Census
Bureaus funding does not come directly from Congressional
appropriations.  So there has been a great deal of discussion on:the
best way of introducing microcomputer technology to the Census user
community, funding it and not intimidating or alienating Bureau users.
In 1983, a joint decision was made to establish the Census
Microcomputer Information Center (CMIC).  The Center with a staff of 4
was placed organizationally in the Director's Office for two reasons:
(1) to show Executive staff support for technology and.encourage users
to make
                                  -6-
active efforts to become familiar with its capabilities and (2) to
avoid turf battles.
The CMIC is a clearinghouse of information for use by Census Bureau
employees.
The goals and objectives of the CMIC are to:
     - assist Bureau employees in their analysis of microcomputers;
     -provide access to and demonstrations of a variety of hardware,
software and peripherals;
     -provide hands-on experience with microcomputers without capital
investment by the individual divisions;
     -provide training on microcomputer hardware and software;
     -provide a clearinghouse for documentation, catalogs, and
pointers to knowledge  for microcomputers, end-user computing and
office automation;
     - decrease the cost of hardware and software through more
informed procurement decisions.
With the direction of program managers, Census Bureau employees may
visit the Center for information about microcomputers, for discussions
of the characteristics of particular computers and the applicability
of microcomputers to projects, or for hands-on experience on a variety
of machines in an attempt to implement those projects.  One can use
the computers in the Center for weeks if necessary, experimenting with
various software on different machines.  The role of the Center is to
help Census Bureau staff define their processing needs, advise them of
applicable software and guide them towards suitable computer
equipment.  The CMIC contains the more popular microcomputers and the
more popular software.  Yet, there are significant numbers of
microcomputers that may provide a unique perspective in the industry
and may offer the best overall systems for a particular problem.
Therefore, the CMIC also sponsors product demonstrations about those
microcomputers that are not currently on display in the Center.
CENTER OPERATION/USE
The Center's hours of operation are 9:00 a.m. to 4:00 p.m. Census
Bureau personnel may schedule time to use a particular machine,
software package, tutorial or specialized peripheral device for one-
hour segments.  They may also request one of the Center support
personnel to work with them.  We generally have personnel in the
Center from 7:00 a.m. to 5:30 p.m. Time before and after hours of
operation is devoted to Center personnel, allowing us to gather and
exchange information on the day's occurrences and to provide
specialized support to executives.
                                  -7-
Some of the typical questions arising on a given day may be as simple
as:
What's the difference between a hard disk and a floppy?
How can I get specific information on a specific product and its
capabilities?
What kind of tutorials/training are available for Lotus, dbase, etc.?
Why doesn't a package perform in a specific manner?
WHAT's AVAILABLE IN THE CENTER
The Center subscribes to approximately 40 periodicals dealing
primarily with microcomputers and associated technology.  About 20% of
these magazines are provided free.  Also the Center has a library of
300+ books dealing with microcomputers, hardware, software, peripheral
devices, etc.  These books are directed at all levels of personnel.
The magazines and books are available for checkout by Census Bureau
employees.
The Center subscribes to Data Pro for microcomputer hardware and
software.  There are numerous other vendor- or industry-provided
catalogues available for review in the Center:
IBM Personal Computer and XT Software Guide The Blue Book for IBM
Engineering and Scientific Progress The Book of Apple Software
The Ratings Newsletter
IBM Software Directory
Many of the supply and peripheral device catalogues are provided by
vendors Public domain software is available in the Center.  Most of it
was acquired from Capital-PC for IBM's and compatibles and the
Freeloader 500 software for the Apple machine.  This software is not
copyrighted and is available for the cost of reproduction.  We have
found many useful utilities available that have saved our users much
development time.
Microcomputer software in the following categories is available:
     Communications software            Mathematical
     Database management systems        Specialized
     Electronic spreadsheets            Statistical
     Integrated software                Word processing
     Presentation graphics              Utilities
     Programming languages
This software is available for user evaluation.  The end user
determines whether the product will produce the required results.
About 15% of our software was provided by vendors for use in the
Center -- but only in the Center -- for evaluation purposes and not
for production work.
The Bureau's policy on copyrighted software is that it is not to be
copied for any reason other than backup.
                                  -8-
HARDWARE
There are 2 IBM PC/XT's hooked to a local area network.  A Sperry
Model 50, a Wang PC a Grid, Apple Macintosh, peripheral devices,
plotters,- printers, Polaroid palette, etc. are available.  Many
microcomputer vendors (43 to date) have come to the Census Bureau to
demonstrate their products, and many have loaned their products for
evaluation from 30 to 60 days, depending on product.  Some of the
vendors are:
          A & F Computers          Sony
          Digital Equipment        Fujitsu
          Olivetti                 Motorola
          Hewlett-Packard          Exxon
          Data General             Radio Shack
ELECTRONIC BULLETIN BOARD
In February 1985, an electronic bulletin board was placed into service
to facilitate information interchange on product evaluations, user
projects, etc.
PROCUREMENT POLICY
The Census Bureaus procurement policy evolved because of our
organizational structure and our funding.  While all procurement
actions are to be processed and controlled through the Bureaus
Procurement office, requests for ADP-related actions will continue to
require some specialized processing.
The justification and acquisition approval for microcomputer equipment
and off-the-shelf software and supplies totaling less than $10,000 is
delegated to the Associate Director level.  The ADP staff no longer is
required to review and approve such purchases.  When the purchase
order is sent to the Procurement Office, requests for sole source and
brandname purchases costing more than $500 must include a brief
justification. When the purchase order is received in the Procurement
Office, information copies are forwarded to the Census Microcomputer
Information Center to be used to update the Census Bureau inventory of
microcomputer equipment and software.
MICROCOMPUTER    MAINTENANCE POLICY
The Census Bureau's microcomputer maintenance policy is based on cost.
For every six machines purchased we purchase a spare machine because
the cost of a one-year maintenance contract on the first six equals
the cost of the. spare.  These machines are not just stored; they are
used in noncritical environments where they can be removed to replace
a critical machine as needed within one hour.  When a user encounters
an equipment problem that is beyond the users capability to resolve,
he or she contacts, our Technical Services Division (TSD) service
representative who will respond by sending a technician to the user's
site to isolate the cause of the equipment problem.
                                  -9-
If it is something simple that the technician can repair on the spot
(such as replacing a fuse, resealing a loose board, or tightening a
plug), the technician will make the repair.  If the problem cannot be
resolved by the technician on site, the technician will telephone the
CMIC to request that a replacement computer or input/output device be
loaned to the user until the user's machine is repaired.  TSD will set
up the replacement equipment for the user (if necessary) and take away
the machine that needs repair.  The user should be able to resume
normal operations with minimal delay, aggravation and frustration.
If the device is still covered by its original warranty, TSD will
arrange to have it repaired under the terms of the guarantee.  If the
warranty is no longer valid, TSD will arrange to take the machine to a
designated dealer for a repair estimate.  When the machine is left
with the dealer, a hand receipt will be signed by the dealer and
returned to TSD.  When the dealer calls the estimate to TSD, TSD will
prepare a purchase request and forward it to the user's division.  The
division will insert the appropriate accounting code, approve the
action, place a priority flag on it, and send it to the Procurement
Office.  The Procurement Office will expedite all micro maintenance
requests by calling the dealer with a purchase order number.  When the
repairs have been finished and the machine is ready for pickup, a
driver will take the purchase order to the dealer and pick up the
machine.  This procedure is valid for any repairs totaling less than
$1,000.  In cases involving repair estimates in excess of $1,000, TSD
will contact the microcomputer user to discuss whether the repairs
should be authorized and, if so, what procedure must be followed.
The loaner machine will be under the control of the CMIC with the
following priorities governing their use:
Top priority -- to any user where TSD has removed a machine for
authorized repairs.
Second priority -- for use in support of hands-on training classes
sponsored by CMIC.
Third priority -- for use by someone who wants to do small projects on
a borrowed machine.
Priority will mean exactly that A broken machine will be replaced with
a loaner from the CMIC even if it means having to take the loaner away
from someone who is using it under a lower priority.  I want to
emphasize that this is our current policy, but it can be changed very
quickly. We are constantly monitoring this procedure and continually
reassessing our options (i.e., outside service contract).
MICROCOMPUTER TRAINING SUPPORT
We established a classroom with 16 machines for hands-on-training.  We
did this because of the numbers of people requiring training and the
cost of sending people to outside courses.  The types of courses
taught are: Introduction to Microcomputers, Databases, Word
processing, Spreadsheets, Graphics, etc.
                                 -10-
Originally there were requests for training of 3000 persons in all
aspects of microcomputers.  That has been reduced to approximately
2100.  We believe this training demand will be high initially and then
will drop off dramatically.  Outside instructors have been hired to
teach our classes. We have had a great deal of success with this
process because of the quality of instructors acquired.  To pay for
this training facility we charge back directly to the user division
the cost of the instructor, software purchased and maintenance cost of
the classroom.  This cost goes to a maximum of $125 per class,
significantly cheaper than to send all people to outside training.
NOTE:     Many vendors sell at a small cost educational licensing
agreements providing copies of their software for each machine in the
classroom. some vendors will not do this; then we must purchase copies
for each machine at full price.
OFFICE AUTOMATION
I have specifically not addressed the topic of office automation, as
we are still planning and discussing exactly what office automation is
going to mean at the Census Bureau.  Our primary planning focus at
this time is to determine what functions need to be provided Bureau-
wide and what functions will be left to individual operating units.
                     THE NATIONAL SECURITY AGENCY
                 PERSONAL COMPUTING INFORMATION CENTER
              Kathy Schnaubelt, National Security Agency
The National Security Agency established a Personal Computing
Information Center (or PCIC for short) approximately a year and a half
ago.  This action was taken in response to the Agency's growing demand
for personal computer products.
In the year prior to the opening of the PCIC, many new personal-
computer products and vendors were reaching the marketplace.  A
growing number of these products were in turn being purchased by a
cross-section of Agency elements.  This mix of products across the
Agency began surfacing problems such as that of system
incompatibility.  This may be illustrated by, the example of a
diskette of data or software running on one computer brand but not on
a different brand of computer.  The PCIC was designed to assist Agency
personnel in the selection, acquisition and use of an established set
of "standard" personal computer products.
The basis for the selection of standard products was determined by the
Agency's needs as a whole.  One such requirement was for the UNIX
operating system.  Hardware selected as the Agency standard
workstation would have to be able to run under the UNIX operating
system.  At the root of decisions of this nature was the concept of
compatible hardware and software products that would be easy for
people to acquire.
                                 -11-
Another important concern for us was security.  By going to
standardization, that problem may be minimized by the selection of
products that meet this requirement and then training personnel to use
them.
A third consideration was supportability.  Maintaining a variety of
microcomputers, or personal computers, can be a logistics nightmare;
stocking of parts, replacing them, etc., in any number can be
devasting.
Finally, there is cost.  By limiting the number of kinds of personal
computers and software products that we use, we are able to buy large
numbers of each at a lower per-unit cost.  Right now we have thousands
of microcomputers in the Agency, and we have plans to buy many more,
which should result in a significant savings from bulk buys.
The PCIC was established to meet the following objectives: 1) to
promote the use of standard equipment; 2) to share an centralize our
small systems resources (like everyone here, we have a limited number
of people to support these products); 3) to minimize the end-user
application load; 4) to maximize cost effectiveness; and 5) to
centralize product registration (providing anonymity in our
workplace).
The PCIC has become a focal point for all Agency standard products,
and to date these products include: an Agency standard
terminal/workstation which is an enhanced IBM XT; the standard office
automation equipment which is the WANG Professional Computer; an
interim standard local area network.  So there will be a family of
Agency standard host computers.
The PCIC provides its customers with information on all of the
standard products that are available; and this includes a reference
collection of books, periodicals, in-house-developed working aids,
research guides, comparison charts of the capabilities of the
different products, and a referral service for technical questions.
It also provides demonstrations of standard products.  Anyone can go
down to the PCIC and use one of the standard products, whether it's
hardware or software.
To encourage the use of the PCIC by Agency personnel, the PCIC tries
to make the acquisition of standard commercial products as simple as
possible.  Rather than have each office go out and do their own
purchase request, an authorized individual can come into the PCIC and
request commercial software. The software is actually stocked in the
PCIC.  We have licensed some items (like CONDOR and MICROPRO products
for example).  By doing that, we have actually reduced some costs by
70%.
Non-standard products may still be purchased, but on a limited basis.
A non-standard product must be requested in writing.  This request is
reviewed by a software evaluation team to determine the validity of
the purchase request.  When a product offers a unique capability, it
is purchased and evaluated.  A favorable evaluation results in the
product's being added to the list of standard products.  A product.
which does not offer any capabilities beyond the standard product
line, or in fact is defective, would be placed on a prohibited-
purchase list.  In any case, the PCIC still does the actual
purchasing, whether it's for a standard product or, an evaluation copy
of a non-standard product.  This saves the requester from, the
paperwork of writing a purchase request document.
                                 -12-
While the purpose of the PCIC is to furnish standard products, it also
functions in identifying products that meet certain minimum
requirements for Agency use.  These products are added to the list of
standard products to provide a flexible work environment for Agency
personnel. The goal is not to restrict what people do or how they do
it, but to make sure that the products they use are compatible with
other products used throughout the Agency.
                    USE OF MICROCOMPUTER TECHNOLOGY
                   AT THE BUREAU OF LABOR STATISTICS
               Peter Stevens, Bureau of Labor Statistics
I made the discovery when putting this talk together that I could take
the various displays and shuffle them and present them in almost any
order I chose.  I not quite sure what the conclusion from that would
be, but with this heady sense of freedom, I decided to start in the
middle.  Therefore, the first display you see discusses a brief
introduction as to where we are now.  The Bureau of Labor Statistics
has approximately, 100 microcomputers, almost all of them standard IBM
PC/XT's (see Display 1).  We also have three Ethernet Local Area
Networks two in D.C. and the other in the San Francisco regional
office.  We have network licenses and centralized software libraries
for all of these machines.  This is one point, and the first of the
points which I will be emphasizing, where some of the things that we
are doing that are, perhaps, different from what is commonly done.
Floppy disks have no essential role in the entire operation.  If I had
my way, I wouldn't have them.
                      Bureau of Labor Statistics
                      Networks and Microcomputers
                           Where we are now
Approximately 100 microcomputers in use (mostly highly modified IBM
PC/XT3)
Three Ethernet (FIPS 107) Local Area Networks, two in DC, the
other in San Francisco.  Software libraries are centralized.
We are close to completing the "large scale pilot" stage of our
development effort.
For each application area our goal is to identify and validate quality
products which can be made a part of the standard BLS microcomputing
environment.
                              Display 1.
In general, the way people get software onto their machines is through
local area networks from centralized storage devices. We are getting
to the end of what might be called the "research phase" of this entire
new technology operation.  The three networks were all acquired by a
competitive
                                 -13-
procurement which we ran a couple of years ago and which is, in
effect, a large-scale+west.
That gets to the last point on Display 1, which is the basic goal for
what we are trying to accomplish right now: to identify and validate
quality products which can be made A part of the standard BLS
microcomputing environment; then, in the next stage of our operations,
to make standards for use throughout the Bureau.
When I looked at Display 2, I decided I could put it up and talk about
it for twenty minutes without any trouble at all because it enumerates
the applications and I think that gives some scope of the project.
But given the terrible time constraints that we are under, I will
spare you a lot of discussion here.
              The following are major application areas:
Word Processing
Graphics
Spreadsheets
Statistical Analysis
Data Base Management
Survey Data Collection
Survey Control
Project Management
Calendar Management
Network Services, including Electronic Mail, Shared Data Management
and Inter-network Routing.
National Communications via Public Value-Added Networks (X.25 & FIPS
100 standards).
Mainframe Communications Gateways for Interactive and Batch
Operations.
Access to the Local Networks from remote (usually portable)
microcomputers.
                              Display 2.
However, there are two things worth pointing out.  Some may know from
the previous references that "FIPS" stands for Federal Information
Processing Standards, which are produced by NBS and which we are
trying to follow. We have more standards than FIPS 100, and those
things are, in general, a significant part of our operation.
One other point, before moving on here, that I think is worth some
mention: applications like word processing, graphics, and spreadsheets
are stantard and well known; but the applications that I call here
Survey Control, Project Management, and Calendar Management get into a
function for the microcomputer which I don't think has gotten the
emphasis it deserves. This is a Control and Management function.  In
the same sense tnat a microcomputer is a useful tool to use with a
project management package, It is also used and useful for keeping
track of one's personal calendar and the
                                 -14-
ordinary flow of activities through the division. responding to
technology, this is definitely a growing area.
Anyway, enough for the present.  The reason for Display 3 is not so
much a chance to give you the details of how the Bureau operates, but
to make a point that our efforts, in these areas were started in
response to a serious and well-understood operational problem that we
are having.  The large, centralized mainframe computer provides, in
our view, a very poor, very weak environment for the general area of
interactive applications.
                       How This All Got Started
Throughout the 70's the Bureaus approach to computing relied upon two
large, IBM-mainframe, computer centers accessed via dial-up telephone
lines.
While this environment served the large-scale, batch-oriented, survey
processing well, other applications were served poorly:
Interactive applications were very hard to develop, and
response from the mainframe computers varied widely.
Data communications were a constant source of problems, especially
those with our Regional Offices.
The proliferation of incompatible word processing equipment caused
continuing operational problem and prevented any more ambitious office
automation efforts.
The most promising technical approach to solving these problems was:
Powerful microcomputers for interactive processing.
Local Area Networks for the heaviest communications and for
configuration management.
Internetwork and Mainframe Gateways for extended
communications.
Public    Data Networks for national communications.
                              Display 3.
Again, I'm sure you wouldn't like to see me stand here and cry, so
I'll spare you the details of the problems we have had with data
communications since the AT&T divestiture.
The final point under the problem areas is again worth some emphasis.
We have, I think, some thirteen odd different brands of word
processors in place.  None of them communicate with each other.  This
is a story that has
                                 -15-
been, again, welltold.  There was, in the Bureaus top management and
operations management, a perception that this had caused us a great
deal of difficulty and a very strong desire not to perpetuate that
same sort of incompatibility and lack of communication in the new
technology.
The lower part of Display 3 shows briefly what we have selected as the
technological underpinnings of the steps we are taking.  Again, we
could have, a long discussion on say, minicomputers versus
microcomputers and the local area network services, but it is beyond
the scope of this panel. will only mention that these issues were very
seriously considered, and the choices listed were not made lightly.
I would like to draw your attention to the phrase "configuration
management." Having, let's say, several hundred microcomputers all
using the same software packages would not be, in our view, sufficient
to guarantee compatibility.
Companies are constantly issuing new versions, and these new versions
are frequently incompatible with each other.  So you need not only to
standardize with the level of machinery, but you need to do version
control and configuration management to insure that the potential of a
standard environment endures.  One of the major functions of the local
area network is that it makes it really possible to do this.  If we
wish to put up a new version of a particular procedure, we can do so.
We can test it and then make that transition very easily.
Back when I was planning this, I had visions of myself running down
the hall with 500 floppy discs trying to distribute them. It was the
horror of that nightmare that led us in that direction.
Display 41 "How This All Got Started" is from a configurations
perspective.  I urge you not to take this too literally, but, in
conjunction with Display 3, it does demonstrate the basic structure of
the communications and technical environment.  The large, vertical
black bars indicate the local area networks themselves (that is, cable
connections between machines in a single area).  We use two computer
centers: National Institute of Health and Optimum Systems, Inc.  Those
dotted lines indicate communications through the public telephone
system.
On the networks themselves we basically have two types of devices: The
workstations (that is, machines that people use) and network services
for file storage, printing, Communications, etc.
Now we are at the point where we can get down to the most important
part of this presentation.  One of the things that I would like to try
and share with you, from our experience, is an idea that I call, on
Display 5, "Important Operating Assumptions." An assumption here means
about the same thing that "theory" means in physics or chemistry.  It
means an idea that we believe and accept as true and act upon, but at
the same time are constantly retesting and reevaluating.
                                 -16-
[GRAPHIC] \WP1417.GIF
                    Important Operating Assumptions
No single supplier can come even close to supplying top-quality
products for all our requirements.
The best quality and most creative software development now is being
done by independent (and frequently quite small) Software Vendors.
Standards, de facto and formal, play a much more important role for
the microcomputer market than they do for the mini or mainframe
market.
We can increase effectiveness and reduce risk by emphasizing, open
systems and standards rather than by becoming locked in to one
manufacturer's product line.
The most reliable source of information about new products is our own
testing.
The selection, testing and integration of hardware and software are
professionally very demanding tasks.  Statisticians and economists
should not have to become Microcomputer experts to use the equipment
well.
Quality in the initial selection of hardware and software is only the
start of an effective operation.  Support, maintenance, and especially
release control for software are essential to long-term effectiveness.
Planned and controlled redundancy is the best and, in many cases, the
only way to achieve high reliability.
                              Display 5.
The first four items are a basic description of why we are interested
in open systems" or open-systems interconnection. We have substantial
experience with being in the tender and enveloping grasp of a single
manufacturer and in discovering that manufacturer's products don't
meet new needs, or that there is no way to interface some new piece of
equipment to the existing equipment.
THE MOST RELIABLE SOURCE OF INFORMATION ABOUT ANY PRODUCT IS OUR OWN
TESTING. This point belongs in bold print because that is probably the
essence of the whole project.
The computer business has always been full of what I will call "hype":
statements of doubtful truths, made just to sell equipment.  The
microcomputer business is, if anything, worse than the mainframe side
of the business.  We have found that things like articles and
advertisements in magazines, the flowing promises of salesmen, and
similar frivolities are simply not a basis upon which we can operate.
We have certain responsibilities to our users in the, Bureau so that
when we say something is going to work, they can expect that it will
work.  We can't then turn and
                                 -18-
say that the salesman said it will work.  Much of our validation is
this testing of the product claims.
The next two items on Display 5 deal with another very important
aspect of our work.  Doing the kind of validation that will cut
through the hype is, in our view, a demanding task and not one which
need be or should be placed upon the working statistician and
economist.  We have a very large number of users that want to use this
technology.  We have a much smaller number that wish to become
microcomputer experts.  We are trying to create an environment in
which economists, statisticians, managers, clerical personnel, and the
whole BLS community can use microcomputers effectively without having
to go through the struggle and pain that is associated with selection,
testing, and integration of the underlying technology.
The last item in Display 5, I think, is very similar to the ones
already expressed by Census.  The way you get the reliability is
through redundancy.  One of the conclusions that followed from that
idea is to use a standard configuration.  Even though a particular
machine may be intended for word processing and the machine next to it
may be intended for statistical analysis, the underlying hardware will
be the same.  So that, if on the day the analysis is due, that
particular machine decides to go out to lunch, the other machine can
be used to finish the job.
We are getting down toward the end, so we can summarize this by
talking about the Project Goals and Current Policies (Display 6).  You
may remember that I mentioned there were three important problems that
this research effort was attempting to address: the need to have an-
environment in which we could create good interactive systems; the
need to deal with our data communication flows;  and a need to provide
effective intercommunication between machines when used for
statistical survey work, office automation, or any other purpose.
Those were the goals and the motivation to start the project.  They
remain the goals.  Every product we distribute must be thoroughly
tested before full regional use.  Some of the regional offices have
very little background in data processing.  What we put there had
better work, because we don't have the travel budget to fix the mess
if it doesn't.
                                 -19-
                  Project Goals and Current Policies
Project Goals:
To solve the identified major problems with communications and
interactive computing.
To ensure that new products are thoroughly tested before being put
into production system or into all Regional Offices.
To open up new application areas, especially in the areas of end-u3er
computing and office automation.
To establish the basis for the continuing, orderly introduction of
improved hardware and software.
Current Policies:
The selection, evaluation, procurement, and support of new products is
centralized.  Strong, de facto standards exist.
The development of end-user applications is decentralized.
The introduction of new products to Bureau production systems is
closely managed.  Pilot tests are required and high-level approval
must be gained before production commitments are made.
The emphasis on compatibility, full communications, and Bureau-wide
usage is quite strong.
                              Display 6.
Finally, we see this whole technology as having opened up the
potential to get into kinds of applications, that simply weren't being
done at,all by any type of computer,,such as some of those personal
and local organizational ones that I mentioned earlier.  We now need
to establish a basis so that we can continue to introduce, in an
effective and orderly manner, new products and new technology that
continue to pour out of the industry.
From that, we have certain policies: the centralized selection,
evaluation, procurement, and support of new products.  There is some
doubt as to whether we will be able to sustain a centralized
procurement function because of some of the problems in government
procurement which are beyond the scope of this presentation.  In
contrast to this centralization, the development of end-user
applications is decentralized.  That is, the way that persons use the
machines for a particular personator organizational task is a matter
of their judgment and their discretion.
When we are talking about introducing this technology into Bureau
production statistical systems, there is much stronger management
control; and developments are closely watched.  We insist on Bureau
testing and evaluation before committing important Bureau projects to
the new technology.
                                 -20-
I think I have said enough about the need for compatibility.
Finally, on Display 7, under the heading of Where We Are Going, there
is basically more of the same.  I mentioned we are getting toward the
end of the large-scale research phase.  We are planning to add local
area networks into all eight regional offices instead of just San
Francisco.
We have one aspect of the Bureau which may be unique in that the
Commissioner of Labor Statistics has A PC in her office.  She also has
one at home and uses them both.  She has an intense personal interest
in what I call here, "Management Communications." Through the local
networks we have possibilities that we never had before.
Through the research phase of this work, we have not had what I might
call "traditional government procurement cost/benefit justification
analysis" very much.  I expect" as we move to the broader expansion of
microcomputers into Bureau activities, that analyses of that nature
will become important.  There are many areas about procurement issues
that are, at the moment, looking through a glass very darkly.
                          Where We Are Going:
As the performance of specific hardware and software products is
validated, their use will be expanded to production tasks.
The number of Local Area Networks will be expanded to include all
Regional Offices.
The communication facilities will be expanded to include Cooperating
State Agencies for data collection and'survey processing.
Management communications, among the Commissioner, Office
Chiefs and Division Chiefs, will become increasingly important.
The number of microcomputer workstations will be significantly
expanded.  Obsolete or ineffective equipment will be replaced by
microcomputers.
New hardware and software developments will be watched for possible
replacements to standard products.
As the new technology replaces existing equipment and applications,
greater emphasis will be placed on cost/benefit justifications.
                              Display 7.
Display 8 shows where we expect to go technologically.  I ask you not
to take that too literally.  This is not a technical model, but rather
a demonstration of the way we see things getting done with each of the
regions having its own network communicating to our network in
Washington.
                                 -21-
[GRAPHIC] \WP1422.GIF
                              DISCUSSION
                 Lawrence R. Cox, Bureau of the Census
I will attempt to keep my comments brief so that we can have a full
interchange between the speakers and the audience in proper "workshop-
fashion.  In proper "discussant" fashion, I will highlight what I see
as the major similarities and differences among the three approaches
taken, in the context of what I have learned from the presentations
collectively and from my experiences at the Census Bureau.
I have learned that microcomputer technology is a must for statistical
programs.  Automated, interlinked statistical program offices are more
efficient and effective than those which are not.  Users of
statistical information have discovered microcomputer technology; and,
so, statistical data providers have a responsibility to keep pace.
Data review and analysis at its best is an interactive process between
the expert data analyst and the data, supported by statistical
software.  Mainframe computing cannot offer these services on a large
scale in a realistic manner or at a competitive price.
I have learned that an organizational focus is needed to provide
information and support both to management and users as this new
technology becomes introduced and assimilated within the organization.
We have seen that such a group can have any of several functions,
depending upon organizational size,.needs, goals and objectives:
     -user education and handholding
     -repository of literature
     -source of hands-on experience
     -maintenance
     -training
     -develop and distribute product lists and recommendations
     -establish guidelines for microcomputer procurement, use,
maintenance, training, etc.
     -recommend standards for microcomputer hardware, software and
uses of microcomputer technology
     -establish and enforce such standards
     -aid in the procurement process
     -evaluate procurement requests
     -decide upon procurement requests
     -advise in the management of this new technology
     -play an active role in its management
These functions, as I have presented them, lie on a continuum from the
more passive, permissive or experimental approach to the more
standardized, structured, or production-oriented approach.  These
needs and the management philosophies underlying them seem to me to be
well-represented on that continuum by the three agencies represented
here today.
The free-market or laboratory approach adopted by the Census Bureau
says, in effect, let's provide our diverse group of programs and users
with the information necessary to begin to explore uses of
microcomputer technology.  Let's minimize the procurement obstacles to
doing so, and let's work closely
                                 -23-
with users in their applications and see what lessons are to be
learned and what patterns emerge.  In effect, as an organization,
let's not force microcomputer hardware and software choices, but-let's
closely manage and monitor several experiments and learn from each of
them.
At the National Security Agency, decisions were driven by the
overriding need to standardize on hardware and software choices-
sufficiently to allow diverse and distant groups to talk to each other
and access the same data and programs, but stopped short of imposing
inessential standards.  Within a predefined architecture of standards,
NSA users are free to experiment, to share information and to tailor
choices to programmatic and individual needs.
At the Bureau of Labor Statistics, the requirements for good and
standard communications between offices and geographic areas were
paramount.  Experiments were conducted to fix upon the best choices,
from which standards are to emerge.  The environment is intended to be
uniform and capable of supporting continuing, production-oriented
work.
Reflecting upon this continuum for a moment, I could equally describe
it as being from user-oriented to program oriented, reflecting a
progression defined in terms of the number of diverse programs and
functions within these agencies which each agency seeks to address
with automation at the microcomputer level.
Interesting, all three organizations share several characteristics:
they are not small, they deal routinely with massive amounts of data,
their paramount concern is improved and broader access to their own
data, their systems require mainframe-gateways or links, and they
operate under strict data security requirements.  However, for reasons
which we have heard and others you may explore in open discussion,
they have chosen three different approaches to tackling the problem of
planning and managing microcomputer technology.
                         QUESTIONS AND ANSWERS
Ql:  How was the Census Bureau able to acquire 500 microcomputers in a
little over one year given GSA guidelines?
Al (Mr. Swank): The Census Bureau did not go around GSA guidelines and
standards, but worked within the existing regulations. Most
procurements are off the GSA schedule.
Q2: What variety does the Census Bureau have in their brands of
microcomputers?
A2 (Mr. Swank): Currently there are 25 different brands of
microcomputers in operation at the Census Bureau.
Q3: Does Census go through the "GSA microcomputer store"    in
procuring its microcomputers?
                                 -24-
A3 (Mr.  Swank): Yes, when possible.  However, the GSA microcomputer
store does not stock all brands, and this forces the Census Bureau to
go elsewhere.
Q4:  Why did Census create a separate staff for microcomputers when
they already had an established automatic data processing staff?
A4 (Mr.  Swank): The Executive Staff of the Census Bureau wanted to
show support for microcomputer technology and to give it high
visibility and, therefore, created the Census Microcomputer
Information Center and placed it in the Director's Office.
Q5:  Has the Information Center taken an active role in education of
upper-level management in the uses of microcomputer technology,
A5 (Mr.  Swank): Yes, each member of the Executive Staff has been
given at least an introductory course on microcomputer usage.
Q6:  The presentation left several unanswered questions that should be
addressed:
1. What about the lack of a management system for electronic files?
2.   How are archiving and disposition of files handled?
3.   What about programming for the PC's?
A6 (Mr.  Swank): Electronic filing systems will come in the near
future.  There are several such systems in existence now, but the
costs are astronomical.
A6 (Mr.  Stevens): Software for record retention currently exists but
the big problem is file retention for which very little software is
available
.Q7: Are the PC's at Census "stand-alone" or are they networked?
A7 (Mr.  Swank): Some PC's are networked others are "hardwired" to the
mainframe; the majority are "stand-alones."
Q8:  Two questions regarding the presentations:
1.   What is meant by "software standards"?
2.   Some software packages need improvements, corrections, etc. In
each agency, does anyone speak to the manufacturers as a
representative of the agency?
A8 (Mr.  Stevens):  "Software standards" means software standards.
     For example, there are at least three subcategories of word
processing software, and each would have a   separate software
standard at BLS.
A8 (Mr.  Swank):   Corporate licensing would be the answer.  Those
manufacturers that will not discuss corporate licensing have so much
business they do not need to help and keep the client happy.
                                 -25-
A8 (Ms.  Schnaubelt): The focal point for NSA is with the vendor
rather than the manufacturer.  NSA has had problems with RUBIX from
IBM.  The smaller vendors are much more eager to get the business and
give better contractual terms than the large firms.
Q9:  Is there a very strong recommendation from the panel for a PC
information center?
A9 (Dr.  Cox) An independent PC information center is an absolute,
necessity in a large organization.
A9 (Mr.  Swank) Each agency definitely needs at least a resource
person if not a center.
Q10 Would a small group need a PC information center?
A10 (Dr.  Cox): Not necessarily a center, but at least a reference
person.
Qll: Regarding machine-oriented versus people-oriented use of
microcomputers, what would the individual agencies do for the people?
What are the goals?
All (Mr.  Swank): At the Census Bureau, if the individual divisions
have the budget, they will get the microcomputers they ordered within
3 0 days of the request.
All (Ms.   Schnaubelt): The goal is to have a PC on each desk.
All-(Mr.  Stevens): At BLS, the only drawbacks to a microcomputer on
every desk are budget and procurement.
Q12: With the advent of work-at-home, is there a use of portable PC's
for this purpose?
A12 (Dr.  Cox): The major problem with portable PC's for take-home use
is data security -- a large problem for each of the agencies
represented.
A12 (Ms.  Schnaubelt): At NSA, portable microcomputers are used by
executives and others, but these machines are kept "clean" (i.e., they
have never had any sensitive data on them).  The portables are used
for training purposes only.
A12 (Mr.  Swank): The Census Bureau has many "checkout" machines, but
some" of these are secure-machines and cannot be taken out of the
building.
A12 (Mr.  Stevens): BLS definitely believes in the work-at-home
concept and has machines for this purpose.  However, precautions are
taken to protect confidential data.
Q13: How are services provided to field operators?
A13 (Mr.  Stevens): The regions do their own training on the uses of
the BLS system.
                                 -26-
A13 (Mr.  Swank): There is a standardized configuration of
microcomputer technology in each regional office with a nationwide
company contracted to carry out maintenance.
A13 (Ms.  Schnaubelt): Data and software are transmitted world-wide by
mail or other secured means of communication.
                                 -27-
               SESSION ON ELECTRONIC DATA DISSEMINATION
                            SESSION SUMMARY
The second session dealt with electronic data dissemination, focusing
on disseminating information for use with microcomputers.  While the
first panel discussion focused on.how agencies use microcomputers
within their own internal environments, this session deals with the
impact of microcomputers on users of federal agencies' data and the
possibilities for agencies to make information available for
microcomputer users (that is, dissemination of data using floppy discs
or through telecommunications).
There are some very interesting opportunities for federal statistical
agencies to use new media to provide data to users more quickly and in
a form that is more highly usable than current printed methods.  The
three speakers will deal with these issues.  The first speaker is from
the National Technical Information Service (NTIS) which is primarily
an archival-type agency for disseminating federal data and
information.  The NTIS program to disseminate data on floppy discs,
the problems encountered, and the various issues surrounding this area
will be discussed.
The second speaker is with the Bureau of the Census and works with
their telecommunications system called CENDATA.  CENDATA is used to
distribute perishable Census information to users.
Our final speaker is from the Department of Agriculture.  She will
describe the current, ongoing process to implement a contract with the
Martin Marietta Corporation to establish a telecommunications system
for the dissemination of large databases containing agricultural
information.
         USE OF MICROCOMPUTER DISKS TO DISSEMINATE INFORMATION
        Stuart Weisman,  National Technical Information Service
The history of the National Technical Information Service (NTIS) dates
back to 1945 with the establishment of a publication board to assist
in making unclassified government documents available to the private
sector.  The program went through various transformations, reaching
its current status as an agency of the Department of Commerce in 1970.
----------
*Jay Casselberry, Energy Information Agency
                                 -29-
The law creating NTIS states that NTIS is to search for, collect,
classify, coordinate, integrate, record, catalog, and disseminate
information.  In the early 1970's, NTIS received its first machine-
readable information product.  In 1981 a new unit was established
within NTIS to manage its product line of data base files and
software.  In the summer of 1984 NTIS began to sell data on floppy
discs.
The current NTIS machine-readable-products program contains about 10
bibliographic data bases, 300 source-text non-bibliographic data
bases, 800 numeric and statistical data bases, and 1300 computer
software programs.  With this substantial amount of information
available, NTIS began a review of procedures for disseminating
information products for microcomputers.
The following criteria were considered when NTIS reviewed the
potential for disseminating their information products on
microcomputer diskettes:
     -Forecasts of the number of microcomputers
     -Forecasts of,the primary type(s) of microcomputers being used by
business and professionals
     -Physical size of the computer diskette
     -Microcomputer operating systems
     -In-house and/or contractor production of diskettes
     -Information products to be made available on diskettes
     -Entire and/or subsets of information files made available
     -Production of microcomputer software
     -Whether to reformat the data for use with popular data base
spreadsheet formats
NTIS has decided to make information products available on 5 1/4 inch
diskettes for IBM and IBM-compatible microcomputers.  Diskettes are
produced by A contractor, and costs are determined based on the number
of diskettes required.
The main problems that have been encountered are in the loss or
incorrect conversion of data when tapes or diskettes are produced,
mishandling of diskettes during shipment, and improper use of the
diskettes by customers.  The way to overcome these problems is to
establish procedures for checking a diskette against the original
magnetic computer tape, and to instruct transportation companies and
end-users on the proper handling of diskettes.
In the future NTIS will consider producing information products on
high density diskettes, hard discs, and, where it is practical,
optical or video discs.
With the future increases in microcomputers by business and
professionals, NTIS is making a long-term commitment to having
information products available for microcomputer users.  With -the
proliferation of data
                                 -30-
management and analysis being done with microcomputers, NTIS
recognizes the needs of this user community.  Displays 9 through 16
illustrate the work of NTIS.
           HISTORY OF MACHINE-READABLE INFORMATION PRODUCTS
Late 60's      First machine-readable products arrive at NTIS
Early 70's     Production Group formed to process orders for machine-
               readable products
Late 70's      Concept of Product Management introduced
1981           Office of Data Base Services
1983           Video disc products from NASA
1984           Data files,available on diskette
                              Display 9.
                              DATA TAPES
Over 1,000 Titles           32 Source Agencies
40 Titles Updated Annually  25 Titles Updated 2-6 Times a Year
15 Titles Updated monthly   Remainder Updated Less than Annually
Standing Orders Available
                              Display 10.
                        MAJOR DATA COLLECTIONS
National Center for Health Statistics (NCHS)
Federal Communications Commission (FCC)
Energy Information Administration (ETA)/ U.S. Department of Energy
National Bureau of Standards (NBS)
Human Nutrition Information Service/ U.S. Department of Agriculture
Defense Logistics Supply Center/ U.S. Department of Defense
Federal Reserve Board (FRB)
Environmental Protection Agency (EPA)
                              Display 11.
                                 -31-
                   DECISIONS, DECISIONS, DECISIONS!!
Size:     5 1/4" vs. 8 1/2" (3 1/21, not readily available)
Density:  Double vs. single aided;
Single vs. double-density (quad-density not readily available)
MS-DOS vs.  CP/M (or MS-DOS Vs. PC-DOS)
Total in-house vs. contracting-out vs. in-house/out-house balance
Products pre-selected vs. demand-driven selections
Complete files only or subsets/extract3
Software
ASCII only or various DBMS/spread3heet/fo   ta
                              Display 12.
                            DATA DISKETTES
5 1/4" Diskettes         Standard ASCII Fo-t
For IBM-PC Microcomputer      Unique Accession Numbers Assigned
Data Tapes Converted to Diskettes       Documentation Required
                                Display 13.
                        PLAYER RESPONSIBILITIES
     NTIS                     Contractor          Source Agency
Order Input & Control    Create diskette master   Provide master tape
                                                  diskettes (with
                                                  appropriate
                                                  documentation)
Copy tape to be used     Archive Master           Available for
conversion                                        consultation
Ship Orders (with        Duplicate Master
documentation)
Available for            Get duplicates to NTIS
consultation
                         Available for consultation
                              Display 14.
                                 -32-
                              The Action
               Customer contacts NTIS   "Available on Diskette?"
     YES                                          NO
1. Price                                1. Estimate price (based on
                                             #of diskettes)
2. Customer orders                      2. Customer orders
3. Order to contractor                  3. Copy master tape
4. Contractor duplicates master         4. Order to contractor with
                                             tape
5. Duplicate to NTIS                    5. Contractor creates master
   diskette and duplicates master
   for customer order
6. NTIS mails (with documentation)       6. Duplicate to NTIS (price
   is to customer--overnight delivery    actual # of diskettes)
                                         7.  NTIS mails (with
                                             documentation)
                                             to customer--overnight
                                             delivery
                              Display 15.
                               Problems
Original tape            ----------------    Bad tape from agency
Copy tape at NTIS        ----------------    NTIS error in  copying
tape
Contractor converts      ----------------    Contractor error in
tape to diskette master                           conversion processor
and duplicates master                             duplication process
Duplicated diskettes     ----------------    Problems created in
sent to NTIS                                 handling of diskettes
NTIS ships diskettes     ----------------(magnetic field, dropped,
to customer                                  smudge, coffee, etc.)
Customer receives        ----------------Customer mishandles diskettes
and processes diskettes                 (see above) plus diskette
                                        processing
                              Display 16.
                                 -33-
                CENDATA: DEVELOPMENT AND IMPLEMENTATION
                 Barbara Aldrich, Bureau of the Census
CENDATA is an information system for disseminating Bureau of the
Census ("Census") information electronically.  Development of CENDATA
began in mid-1983 when Census decided that certain data, especially
time-sensitive economic data, should be available on-line.  CENDATA
was developed under the guidelines that the data should be available
on-line as soon as possible after release and that the system
developed should be done at no cost to Census.
The system was proposed as non-sole source, (i.e., not limited to only
one contractor).  In addition, no money was to be involved in the
arrangement with any contractor, and Census was to have control over
the information made available.  During the entire process of
developing the specifications @and establishing memoranda of
understanding with qualified vendors, Department of Commerce lawyers
assisted in refining the language and procedures.
Census' list of qualifications for vendors wishing to access CENDATA
and make the information available included:
     -A CENDATA user should only have to pay for time used accessing
CENDATA
     -CENDATA should be available separate from other data bases, be
clearly identified, and include the entire CENDATA package
     -CENDATA must be available seven days a week
     -A CENDATA vendor must be willing-to accept data delivery via
telecommunications
     -A CENDATA vendor must be able to offer its users the services of
national telecommunications networks
     -The system must be an end-use-based, user-friendly system
The reasons behind the above qualifications were to:
     -ensure that vendors did not add hidden fees or package CENDATA
with other services
     -enable users to use major telecommunications networks to
minimize costs
     -obtain vendors with the capabilities to handle a large-scale
data base such as CENDATA
-    increase dissemination of Census information products.
Of the dozen vendors who have shown interest in the CENDATA system,
four met the criteria established and; memoranda of understanding have
been signed with two.
                                 -34-
The first vendor, Dialog Information Services, went on-line with
CENDATA on August 1, 1984. (Dialog is extremely prominent in the
library community.) Dialog has CENDATA available using the standard
menu-based system and also makes the information available in a full-
text-searchable format.
In-mid-October, 1984, the Glimpse Corporation made CENDATA available.
Glimpse, in cooperation with the Chemical Bank of New York, markets
data to the financial community.
With the success achieved by the first two vendors in expanding the
dissemination of Census data, Census is anticipating adding new
vendors who service different sectors of the public.  With the
inherent advantages of CENDATA over traditional publications, Census
hopes to continue to expand its user network.
The primary advantages of CENDATA are the timeliness of the data and
the ease of using the system.  One of the first goals of CENDATA was
to have sensitive economic information available within minutes after
any embargo on the information is lifted.  Examples of the- type of
sensitive information. available are manufacturers' and shippers,
orders, retail sales, housing starts, and balance of payments.
Having this information available electronically assists users who are
located away from Washington where the information is initially
disseminated in press releases.  The data are available weeks before
users would receive it in published form, and it can be downloaded
into a user's standard information system for review and analysis.
Census also maintains an inventory of its products on CENDATA.  This
allows a user to quickly determine if a particular publication has
been released, and, if so, the price, source, and Government Printing
Office stock number.
The illustrations that follow, Displays 17 through 21,.show how
CENDATA has been developed for ease of use.  Menus are designed to
provide an inexperienced user with a choice of selections, and to move
from general to the more specific.  In addition, instructions are
provided to help a user move through the system.
                    THE CENDATA INTERACTIVE SYSTEM
The Online Information Utility at the U.S. Census Bureau.
A very small portion of the Census Bureaus vast data holdings has been
included in this "information utility."
Do you wish to see the CENDATA menu? If yes, enter Y or (return).  If
not, enter LOGOFF to end session.
?Y
                              Display 17.
                                 -35-
                         -- CENDATA MAIN MENUS
1    Introduction to Census Bureau
          Products and Services
2What's New in CENDATA
3U.S. Statistics at a Glance
4Press Releases
5Census User News
6Product Infoxmation
7CMMATA User Feedback
8    General Data
9    Agriculture Data
10   Business Data
11   Construction and Housing Data
12   Foreign Trade Data
13   Governments Data
14   International Data
15   Manufacturing Data
16   Population Data
Enter item number or ? for help.
?15
                              Display 18.
15--MANUFACTURING
1    Introduction to the Manufacturing Statistics Program
2M3 Preliminary Report, July 1984
.
.
8    Aluminum Ingot and Mill Products,
          June 1984 (CIR 1433-2)
Enter item number or ? for help.
?2
                              Display 19.
                                 -36-
15.2--MX3 PRELIMINARY REPORT,
     JULY 1984
1    M3 Narrative Summary
2value of Manufacturers Shipments
3value of Manufacturers New Orders
.
.
7    Ratio of Manufacturers Inventories an Unfilled Orders to
     Shipments
Enter item number or ? for help.
?3
                              Display 20.
15.2.3--August 30, 1984
TABLE 2, PART 1: VALUE OF
MANUFACTURERS
NEW ORDERS FOR INDUSTRY GROUPS, MARKET
CATEGORIES, AND SUPPLEMENTARY SERIES
                                                --Seasonally adjusted-
                                                               Monthly
(                                                 Millions of dollars)
SIC                                     Jul.      Jun.      May
Code      Industry                      1984(p)   1984(r)   1984
All  manufacturing industries.          192,450   190,620   193,680
Manufacturing industries
with unfilled orders..............      103,496   102,051   104,482
Durable goods industries............    100,489   99,171    102,,256
     --more-
     Display 21.
After moving through the choices of information topics, the user is
presented with the information requested.
An experienced user may move through CENDATA more.quickly by
specifying all parameters of its search at the same time.  For
example, by specifying 15.2.3 initially, all menus may be bypassed;
and the user moves directly to manufacturing (15), the M-3 report (2),
and specifically the value of manufacturers new orders (3).  This
development allows CENDATA to provide the necessary information and
instructions for novice users without unduly hindering more
experienced users.
                                 -37-
As with any developing system, Census is soliciting comments from
actual and potential users to determine possible system improvements
and expansion of the data base.  The primary users at the current time
are economists,, industry analysts, and market researchers.
Future plans are to, expand the data base with additional Census
products.  Upcoming products to be added are 1984 country population
estimates and statistical profiles of every country in the world.
With the addition of the statistical profiles, CENDATA moves into a
new area since the information is from the International Data Base
rather than from a publication, and the profiles are not readily
available outside the system.
ELECTRONIC DISSEMINATION OF PERISHABLE INFORMATION
Roxanne Williams, Department of Agriculture
The Department of Agriculture has as a primary function the
dissemination of information about conditions related to Agriculture.
The Extension Service is one way the Department uses to get
information disseminated at the local level.  In addition, the
Department has long utilized the printed media for the dissemination
of information around the nation.  A few years ago, a number of
agencies in the Department became dissatisfied with the print media
because of the difficulty in getting information to interested parties
as quickly as necessary.  The agencies, acting independently, tried,
electronic communication of data.  Use was made of a number of
commercial services such as 'DIALCOM, AGNET, and AGRADATA.  DIALCOM is
equivalent to an electronic bulletin board.  AGNET is an on-line
information system developed at the University of Nebraska.
About two years ago, the Department started to have problems with the
use of these services.  Other information companies wanted the
Department to provide the data going to existing services.  They did
not want to have to go to competitors for the information for a
variety of reasons.  One reason was that they wanted to be able to say
they obtained the data directly from the USDA.  Supplying each
potential vendor with USDA data was just too much of a burden for the
Department.
In order to continue to get data to the ultimate end user and at the
same time meet the needs of commercial vendors, it was decided to
establish a single department-wide .system of electronic data
dissemination.  No agency will be forced to use this system; but if an
agency decides to use electronic media, it must use the Department's
system.  This central system will then service the commercial vendors,
including DIALCOM, AGNET, and AGRADATA.
The Department decided to limit the scope of the project to what we
call "time-sensitive perishable data." One example of this type of
data is the agriculture marketing reports.  These Are perishable
because they contain the current prices and the current sales of all
the different commodities around the country.  The data are in
constant demand and they are constantly changing as new reports arrive
continuously.  The demand for the quick and timely dissemination of
these data is very high.
                                 -38-
The Department is utilizing commercial vendor, Martin Marietta
Corporation, to provide this service.  This maintains a Department
policy of not allowing public access to the Department's computer.  It
also keeps the Department from establishing a service that can be
adequately provided by the private sector.  Martin Marietta acts as an
agent of the Department and has agreed not to use its position in
order to benefit itself in the dissemination of these data to ultimate
users.  Martin Marietta can only disseminate these data through the
system established for the Department.  Other commercial vendors (we
call them Level I users) can tie into the system with auto-dial or
auto-set facilities.  For a price, they can even have the main
system's computer call their computer as soon as data ate released and
transfer those data immediately.  Thus all vendors will have excellent
and "equal" access to USDA's perishable data.
Equal access also meant to us that Martin Marietta would not charge
other commercial vendors outrageous prices for access to the system.
We wanted to keep the costs to Level I users reasonable.  Martin
Marietta was very reasonable and agreed to modest and uniform charges.
Ease of access was also important to the Department.  In order to
maintain simplicity and keep programming costs low, we decided to use
a straightforward file structure for the data with access obtained
through a menu-driven system.  The resulting simplicity of the system-
not only makes -for easy access by users, but it also allows
originating offices within the Department to upload files with a
minimum of effort.
Further, the originating offices maintain complete control over their
own data in the system.  They determine when data go into the system,
when they are to be released, and when they are to be deleted.  Martin
Marietta only maintains the hardware and software of the system.
In addition to meeting the requirements of outside (Level I) users,
the system has been designed to the Department's own intern al
requirements for information.  A second type of user (Level II) has
been defined.  Level II users are primarily offices within the
Department and the Extension Service.  Other Federal agencies which
make heavy use of agriculture data will be included.  In order to
service the Level II users, we asked Martin Marietta to allow access
to smaller segments of data.  These users do not need to obtain bulk
data by telecommunications.  The system allows us to break down bulk
reports into smaller segments all of which are accessible via simple
menus.
The Department anticipates that the effect of the new system will be
manifold.  Users should have much better access to a wider range of
information.  Internal communication of information within the
Department should improve significantly.  The demand for hard copy
should be significantly reduced.  All of these effects should help to
reduce the cost to the Department of data dissemination.
                                 -39-
                         QUESTIONS AND ANSWERS
Ql:  What were the particular problems with mailing floppy discs; what
kind of reject rates were encountered; and, if the discs are used for
data transfer, how much of a backup do you need?
Al (Mr.  Weisman): Some problems in handling of the discs during
shipment may have been avoided because we chose to use an overnight
delivery service instead of the Postal Service.  The quality of the
service has been very high, there is very little handling required,
and the service has not failed yet.
Q2:  Did you mention that there were some bad discs that needed to be
replaced?
A2 (Mr.  Weisman):  Yes.  It is very difficult to track down where the
mishandling of discs actually occurred.
Q3:  Is there a flat percentage of reliability?
A3 (Mr. Weisman): The percentage of problems is very small, but it
does occur.
Q4:  Has NTIS considered direct phone transmission of data; that is,
could users call directly to the NTIS computer, similar to commercial
data bases?
A4 (Mr. Weisman): We did make our bibliographic data bases available
similar towhat Census is now doing (as mentioned in the talk by
Barbara Aldrich). That was started around 1974, or perhaps earlier. I
believe there are now four vendors carrying our data base. In
addition, NTIS encourages vendors to carry its statistical files and
source files.  To date, no vendor has elected to carry these files
because it is more difficult to carry these files than a bibliographic
data base.  NTIS has no plans at this time to make these files
available through telecommunications.
Q5:  What are the plans for disseminating data from the 1990 decennial
census?
A5 (Ms.  Aldrich): In terms of data dissemination for 1990 decennial
census data using CENDATA, there are no solid plans, but it is an
issue for thought.  The product information section of CENDATA could
be used as a daily update or product release for 1990.  I believe that
there will be some electronic dissemination, but the amount and the
level are not really being addressed at this time.
Q6:  Please tell us more about the software available through NTIS; is
it public-domain software, software that the agencies have written for
their own use, or some other type of software?
A6 (Mr.  Weisman): While I am the manager for data files and data
bases and there is a separate product manager for software, I will try
to answer your question.  The criteria that NTIS uses for handling
software are the same as, those used, for data files; that is, the
software must be Government-produced. the software must also have a
common usage and be useful to others.
                                 -40-
Q7:  NTIS currently sells a catalog of public domain software for $40
that includes quite a lot of information. Why doesn't NTIS publish
separate catalogs of microcomputer software and mainframe software?
A7 ( Mr.  Weisman): At the present time NTIS only has three packages
available on diskettes for microcomputers, the rest are for
mainframes. NTIS does not convert software at the present time and may
never do so.  Currently  there are not enough diskettes available for
microcomputers to justify a separate catalog.
Q8: Does Census have any feedback from CENDATA, users on the services
and charges?
AB (M.  Aldrich): Yes, based on discussions with users, the charges
seem reasonable.  DIALOG priced CENDATA at $36 per hour, their most
inexpensive commercial rate.  That price does not include the
telecommunications network charge which, with discount, is generally
about $6 per hour.  The Chemical Bank version of CENDATA is priced at
$28 per hour and includes the telecommunications charge.  In addition
to the positive feedback we are receiving on prices, we receive
feedback on what is in CENDATA, what users would like to see in
CENDATA, and what they do not like.
Q9:  Is it possible to download CENDATA data and create other data
files based on this?
A9 (Ms.  Aldrich): CENDATA is all public domain and no part is
copyrighted.  Therefore, it is available for users to download to
their computers or add to other data bases.  This caused a slight
problem with DIALOG because so many of their data bases are
copyrighted.  To end any confusion, a notice was put in the DIALOG
newsletter pointing out that CENDATA is in the public domain.
Q10 (Mr.  Berkman): Would Barbara and Roxanne discuss the impact upon
their particular agencies' personnel who generate the data, in
transferring the data to the two systems they discussed?
A10 (Ms.  Aldrich): I would like to cover the impact in two areas: the
positives and the negatives.  The negative for the people generating
the data is that they must provide it to us in machine-readable form,
either in the appropriate kind of floppy disc or via
telecommunications to our microprocessor.  There are some guidelines,
with respect to designing tables that must be followed, which are
quite difficult.  The industry standard for CRT screens is 80
characters across, so any table must be defined in 75 characters since
the vendors requested five characters for control.  Often tables are
split vertically, with the first part becoming Table 1, Part A; then
the second part is Table 1, Part B; and so forth.  The positive
advantage to people preparing time-sensitive information and providing
the data to CENDATA is a reduction in the interruptions from outside
the agency with requests for data.  Prior to CENDATA, when a data
embargo was lifted, staff members would spend the remainder of the day
answering the telephones and reading data over the phone.  With the
advent of CENDATA, users have an alternative where they can quickly
receive the data.  They can copy the data from CENDATA to their
microcomputers and eliminate the need to listen to it over the phone
and record it.  There are both positives and negatives to the
individuals who provide CENDATA with the information.  In all cases
the
                                 -41-
individual division which is the source of the data provides the
CENDATA staff with the information.
A10 (Ms.  Williams): Agriculture has designed a system whereby each
agency retains control over its own data.  This is a very sensitive
subject, so the system was designed so that each agency enters its own
data into the system.  Because of the wide variety of equipment used
to process data and create reports by our agencies, the system also
needed to be designed so that the agencies did not need to change
their current methods of doing business.  To accommodate the agencies,
each agency only needs to put a header card on its report to identify
the report.  If a report is to be broken up into different levels of
service, an additional header card is necessary.  Based on the header
card(s), the system knows how to handle the report that follows. one
agency, the Agriculture Marketing Service, required another
accommodation because it used a leased wire service with a special
protocol.  Current users of these data had taps on the wire which were
usually linked .to teletype machines. A microcomputer system was
placed between their system and our system to convert the protocol and
place the headers on the data.  This allowed their system to operate
exactly as it did prior to development of our system.
Q11: Does CENDATA provide a computer tape to its vendors or is data
communicated via telec ications? Also, how often are the vendors'
files updated?
All (Ms Aldrich): All CENDATA are transmitted via telecommunications.
We use an enhanced word processor with telecommunications
capabilities.  Information initially goes into a private file where it
is integrated into our standard system. We review the system exactly
as a user would see it and,determine if there are any problems. Simple
problems are corrected using the vendor's editor; serious problems may
be corrected by deleting the file and starting over. When we give the
go-ahead, the data become available on the vendors' systems.  On
DIALOG the files ate brought up overnight so the data becomes
available the next day.  We update daily based on data to be made
available and changes in our product listings.  The update is
controlled by a vendor's software. We move records into and out of
their systems.
Q12: Does the Bureau of the Census pay for the update costs?
A12 (Ms.  Aldrich): No.  Census developed the menu.  We work closely
with the software design people at each vendor.
Q13: Do the vendors limit the amount of information?
A14 (Ms.  Aldrich) Certainly not in the case of DIALOG.  They have the
philosophy that however much information you can give them they will
accept it. They consider data storage to be cheap and pride,
themselves on being one of the largest vendors.  In the case of
Chemical Bank, they have not constrained us either.  About once a year
they request for planning purposes an estimate of how much storage we
will need in the next two years.  We have a small amount of data
available on-line with a rich potential for it out of hand, but thus
far there are no problems.
                                 -42-
Q14: What were the reasons Census decided not to go sole source?
A14 (Ms.  Aldrich): One of the primary reasons was our objective to
get the system operational as quickly as possible.  By offering it to
several vendors, we could avoid the procurement process.  Another
appeal was that by going with several vendors, CENDTA would be
available to different segments of the community.  With different
vendors it might be possible to reach users that previously had not
been Census data users.  I think that in the case of DIALOG we have
found a lot of librarians who were not previously users.
Q15: Has meeting the different protocol requirements of the different
vendors involved much extra work?
A!5 (Ms.  Aldrich): No, because we have only one system and one format
for the data; each vendor must agree to adapt that format to whatever
they see fit to use.  There is one set of, codes which are very simple
and straightforward.
                                 43 -
                        SESSION ON APPLICATIONS
                            SESSION SUMMARY
The relatively recent emergence of powerful microcomputers (micros)
coupled with the availability of specialized vendor software packages
for micros has significantly enhanced the federal statistical
community's ability to gather, manipulate and analyze data.  Today,
more than ever, it has become easier to perform data analyses
previously considered to be impractical due to resource and time
limitations associated with traditional manual and computer
methodologies.  Accompanying enhanced analytical capabilities have
improved methods for communicating the results of our data analyses.
Powerful graphics software along with improved graphics plotters and
color displays have made it possible to easily paint pictures
reflecting data. analyses, which before were only possible through
relatively expensive and involved mainframe processing.
The boom in microcomputer usage in the areas of statistical and
economic analyses is due in large part to the many advantages micros
have over mini and mainframe computers.  In particular, today's micros
have storage capacities and processing speeds which often exceed
mainframe capabilities commonly found just 10 years ago.  Micros are
generally simpler and easier to use than minis and mainframes; they
are often portable; and they cost less to procure, operate and
maintain.  Micros are usually more reliable (less down time), and they
often possess the ability to.communicate with minis and mainframes,
which permits micros to access and transfer large data files.
Along with the "hardware" advantages, there are also "software"
advantages associated with micros.  In particular, there is an
abundance of high quality and user-friendly vendor software packages
available, many of which permit the user to add his or her own code to
modify and enhance the package's capabilities.  Relative to mini and
mainframe costs, these software packages are inexpensive.
A few disadvantages of micros should be mentioned as well.  The
ability to exercise security measures and ensure control appear to be
more limited.  Today's micros are slow in comparison to current state-
of-the-art mainframes.  There exist serious compatibility problems of
file structures between vendor software packages.  Finally, there is
often an added personal cost to the micro user in the area of
additional time spent in procurement and maintenance, since these
activities are usually not required of a mainframe user.
The discussions which follow address many of the issues mentioned ----
above.
------------------
*Thomas Nagle, Internal Revenue Service
                                 -45-
         SPREADSHEET AND STATISTICAL/ECONOMETRIC APPLICATIONS
                        IN ECONOMETRIC RESEARCH
          Linda P. Atkinson, U. S. Department of Agriculture
Microcomputers are in widespread use throughout the Economic Research
Service (ERS).  I will be discussing their application not by
secretarial staff for word processing or by data processing
professionals, but rather by the economic research staff themselves.
Our economists first became involved with microcomputers through the
use of spreadsheet software, and this is still where the bulk of the
applications are. Packages such as Supercalc and Lotus 1-2-3 are used
extensively for data preparation, developing tabular reports,
producing high-quality charts, graphs, and plots, performing if-then
analyses, and interfacing with mainframe software.  Some of the
systems which have been developed with these packages are, in fact,
quite sophisticated.
One group, for example, has developed a program using Lotus 1-2-3 to
assess preliminary economic impact of foreign pests to producers,
consumers, and society in general.  A partial budget analysis is used
in which different economic scenarios are simulated by allowing
changes in costs of production, yield, and prices for the affected
crops.  The entire system is menu driven and has options for various
tables and graphs which can be produced.  The program set-up is-being
used as a template from which similar analyses can be developed, such
as a program to evaluate the impact of change in ozone concentrations
on yields.
Another group hail been using Supercalc for data entry and preparatory
calculations before running a program on the microcomputer to convert
the data to-the form required for input to mainframe packages such as
TROLL or SAS.  After running these mainframe programs, files of output
were then transmitted back to the microcomputer and reformulated for
spreadsheet entry so that tables and graphs of output were
automatically generated.  Additional changes in the form of model
output results could then be made, interfacing the flexibility of the
microcomputer with the calculating power of the mainframe computer.
Now this group has a simplified version of their model, the world
grain-oilseeds-livestock (GOL) trade model, running entirely on the
micro in Supercalc.  The GOL model is an annual simulation model
consisting of 27 country and regional models and 20 major agricultural
commodities.  The individual models are linked to solve simultaneously
for a vector of prices which clear world trade.  The global model
system has equations for 339 country-commodity combinations.  Running
a 20-year projection on the full linked model on an IBM PC/XT took 48
hours; however, an individual country model runs in about 15 minutes.
They hope to improve speed considerably by the acquisition of an IBM
PC/AT with memory upgrades.  The program has been. set up to ask
questions of the user, such as what country is to be analyzed for what
start and end dates.  Users like the flexibility of the spreadsheet
format; one can.get in and look at a simulation, watch the numbers
change and see where any problems are.  Built-in equation writers
allow you to change the structure of a model or you can edit it
directly.  You can pre-create graphs and have them contain historical
data to compare to simulated results.
                                 -46-
A good reference on building such models in spreadsheets is an article
from the February 1985 issue of Byte magazine entitled "Simultaneous
Equations with Lotus 1-2-3." The author demonstrates how to formulate
and solve a famous macroeconomic model, Klein's Model I, using
standard Lotus commands.  The Gauss-Seidel iterative method is used to
numerically solve the system, with a one-line Lotus macro written to
test for convergence.
Another example of Supercalc use is to make projections of coarse
grain production in foreign countries using population projections,
real GNP growth rate, elasticities of consumption with respect to
income, and growth rates of production.  The spreadsheet format allows
the analyst to change one item, such as an elasticity and have
everything else recalculated.  In this way it becomes easy to cross-
check to see if implications of certain assumptions are reasonable.
A planned enhancement to this analysis technique is to begin to use
the regression capabilities of a microcomputer statistical package,
ABSTAT specifically.  Regression of grain conversions over time can
yield estimated elasticities, which can then be put back into the
spreadsheet.
ABSTAT was acquired as a user-friendly package to do basic descriptive
statistics and simple linear regressions.  We have also acquired
SPSS/PC, the micro version of the popular mainframe package.  Many of
our economists are accustomed to using SPSS for analyzing survey data
and large cross-sectional data files such as those provided by the
Census Bureau.  To provide databanking of larger files of which
portions might be analyzed using SPSS/PC, we recently licensed SPSS/X
to run on our in-house minicomputer.  SPSS/PC's ability to handle
"portable" system files which can be uploaded and downloaded easily
aids in forming an interface between the large and small computers.
We will first apply this in analyzing the results of an in-house
information-needs, survey; complete questionnaire results can be
stored on the minicomputer, with data for particular groups of
respondents or selected variables downloaded to the micro for detailed
analysis without having to be redefined.
We have two packages in-house that can perform more complex
econometric estimation techniques: RATS (Regression Analysis of Time
Series) and SORITEC.   A domestic sugar model has been set up in
SORITEC.  Varioust estimations were performed, including OLS and two-
stage least squares and Cochrane-Orcutt autocorrelation correction for
each equation.  The model was too large at 15 equations for SORITEC to
do maximum-likelihood estimation of it, but the new version, when it
comes, should be able to handle it.  The model was simulated in
SORITEC with the various sets of coefficients and also with various
changes made to the model, for example perturbing an exogenous
variable by 10%  SORITEC has a command to compare actual and fitted
values, computing summary statistics to measure goodness-of-fit.
Because the model is somewhat large, it is run in a "batch" mode with
Wordstar used to edit the SORITEC program.  The model has also been
put up on Lotus 1-2-3 to experiment with the parameters.  Graphwriter
is used to output plots of results.
There is a free version of SORITEC called SORITEC Sampler which has
capabilities of the main package up through, two-stage least squares.
It cannot perform three-stage least squares maximum-likelihood
estimation or
                                 -47-
handle-nonlinear models.  It produces nice screen graphics of
regression plots including residuals, which can be dumped to a line
printer (but not at present to a plotting device).  While not of
publication quality, the plots are very useful for analytical work.
For example, as part of a farm production model, an equation was
estimated with prices paid by farmers for feed as a function of corn
price and the price,of soybean meal.  The residuals showed some
problems; an autocorrelation correction was tried and the regression
re-estimated.  The new plot showed substantial improvement in the
residual analysis.
Another analyst uses RATS to estimate import demand for wheat, corn
and soybeans in four Asian countries.  The 10-equation model has been
run through OLS, instrumental variables and Taylor-series
approximations, and he is trying to get around memory constraints
(supposedly temporary until the new release of the package) to do
seemingly unrelated regressions.  The ARIMA time-series analysis
capabilities of RATS were used in this project in determining how to
average prices on a yearly basis" looking at the cross-covariances
between prices and imports to decide on a lag structure.
RATS is also being used to estimate a Canadian grains and rapeseed
model.  Again, a spreadsheet, in this case Lotus, is being used to
update the data and provide graphical output, as well as to simulate
the results.
We have at ERS a number of other software packages for microcomputers
to perform more specialized functions.  GAUSS is a matrix programming
language that allows you to write out an analysis the way you would
write it mathematically.  You can easily write down the estimation
commands for the coefficients of a simple linear model, or the code
for a complex statistical algorithm as it appears in a journal
article.  GAUSS does not currently come with built-in statistical
routines but is planned to in the future.
Another program, TK!Solver, solves simultaneous nonlinear systems,
again allowing you to express the equations similarly to how you would
mathematically.  A package called MUMATH solves mathematical problems.
symbolically land can take derivatives, etc.  Especially useful in
macroeconomic theory, one can change coefficients or other aspects of
a model symbolically rather than numerically and see the logical
implications in terms of cross-relationships that result.
We even have some researchers who use small programs written in Basic
to perform a specific statistical function, such as regression or the
calculation of standard deviations or coefficients of variation,
rather than bother learning how to use a more complete statistical
package.
Finally, I would like to mention one macroeconomic model to which ERS
subscribes, FAIRMODEL which is a model-of the U.S. economy developed
by Professor Ray Fair of Yale University-and programmed for the IBM PC
and XT.  The model consists of  30 stochastic equations and 98
identities and is re-estimated quarterly.  It can be used for
forecasting, policy analysis, scenario development and as a research
tool.  An analyst can run experiments change exogenous assumptions,
enter adjustment factors, or exogenize an equation or block of
equations, and view the results.  An interface to Lotus 1-2-3 can be
obtained with FAIRMODEL to use for setting up an analysis and deriving
tables and graphs from the model output.
                                 --48-
These have been only a few of the very many applications of
microcomputers that we have in-house.  The use of microcomputers has
revolutionized the way our analysts conduct their research.  In the
area of econometric modeling, many more alternatives can be considered
and assumptions tested in a much shorter period of time,, taking
advantage of the interactive nature of the software oh these machines.
Researchers who in some cases had little computer experience
previously have become proficient with the easy-to-use and flexible
software available on microcomputers, particularly spreadsheets, and
seem to prefer this to the use of cumbersome statistical packages.
However, now that better statistical software is becoming available,
interest in it is growing.  The economists I spoke with seemed to want
to choose their, own components of an analysis system - spreadsheet,
statistical program, graphics package, word processor - and are
concerned with having good interfaces so they can quickly move data
from one program to another.  Some problems with memory constraints
and speed have been experienced, but hardware is rapidly improving to
alleviate this. There are worries about having errors creep into
programs, especially with spreadsheets that may not be well documented
and might be passed from one researcher to another.  These and
security of data issues will have to be addressed now by analysts who
perhaps had that taken care of for them in a mainframe environment,
but this seems to be a fair trade for the ability to interact directly
with their models and better understand what the data are saying.
Reference
Johansson, Jan-Henrik, "simultaneous Equations with Lotus 1-2-3,"
Byte, February 1985, p. 399.
Acknowledgments
Many thanks to the following researchers who shared the results of
their work on microcomputers: Walter Ferguson, Vernon Roningen,
Michael Lopez, David Weisblat, Suchada Langley, Gary Lucier, Carlos
Arnade, Larry Deaton, Clark Edwards Paul Prentice, and Merv Yetley.
Software Vendors
AbStat                                  SORITEC
Ander-Bell                              Sorites Group, Inc.
P.O. Box 191                            P.O. Box 340
Canon City, Co                          Springfield, VA 22151
81212
Graphwriter                             SPSS/PC
Graphic Communications, Inc             SPSS, Inc.
200 Fifth Avenue                        444 Michigan Ave.
Walthham, Ma  02254                     Chicago, Il 60611
                                        (312) 329-2400
                                 -49-
Lotus 1-2-3                               SuperCalc 3
Lotus Development Corporation             SORCIM/IUS Micro Software
16l First Street                          2195 Fortune Drive
Cambridge, Ma 02142                       San Jose, Ca 95131
(408) 942-1727
MUMATH                                    TK!Solver
Microsoft Corporation                     Software Arts, Inc.
10700 Northrup Way                        27 Mica Lane
Bellview, Wa 98604                        Wellesley, MA 02181
RATS                                      FAIRMODEL
VAR Econometrics                          Urban Systems Research &
134 Prospect Ave.                         Engineering
Minneapolis, Mn 55419                     2067 Massachusetts Avenue
(617) 661-1550                            Cambridge, Ma 02138
SPREADSHMT AND DATA BASE APPLICATIONS USED BY THE CROP REPORTING
BOARD IN  REVIEWING SURVEY INDICATIONS AND PREPARING PUBLICATIONS
Gary Nelson, U. S. Department of Agriculture
Our Agency, the National Agricultural Statistics Service, is
responsible for gathering crop and livestock statistics for the
Department of Agriculture.  We make forecasts of the crop size during
the growing season, and final, estimates at the end of the year.  We
have a network of 44 field offices serving all fifty states.  These
field offices regularly survey thousands of operators of fa=s, ranches
and agriculture businesses to gather information about their
operation.  Statisticians in our field offices assemble the
information and make recommendations on such items as acres planted or
harvested, yield per acre or the amount of grain that is in storage.
They then send indications and recommendations to our headquarters
office in Washington, D. C. where the data are assembled and reviewed,
and U. S. estimates are set and published.
Our state offices are connected to a large computer network, the
Martin Marietta Data System.  The indications, recommendations and
comments are submitted over the network to our office in Washington,
D.C. We have several IBM PC/XTIS in our section, which we utilize
extensively for summarizing data and weighting the data to give state,
regional and national totals, as well. as designing questionnaires and
various othee spreadsheet applications and some graphic applications.
One microcomputer application that we have developed is called the
Grain Stocks Program.  This program produces a report that we release
four times a year, that shows the amount of grain that is in storage,
both on the fa= and what is stored off the farm.  The report was
produced manually in the past and we wanted to put it on the micros.
In designing this application, we wanted a system that would: be easy
to use, be menu driven, be able to download the data from the data
base on Martin Marietta to the microcomputers; assemble the data,
provide a means for making changes, provide us with summaries and
camera copy that we could use to print the report, provide the ability
to transmit the changes back to the data base at
                                 -50-
Martin Marietta, and provide the capability to compute and print a
balance sheet.  The program uses a combination of Condor and
SuperCalc3.
Another application of the micros is in tabulating and charting data
used in making forecasts on the size of the various crops each month
throughout the growing season.  These forecasts are released on a
specific day each month.  Since the forecast of the size of the crop
can have a definite impact on the prices, it is extremely important
that strict security be maintained in compiling these statistics until
the report is released to the general public.  To insure that the data
are kept confidential, we operate under a "lockup" procedure.  The
members of the Board review the data, read charts, and recommend a
yield for each State and the Region.  The Board then jointly agrees on
a yield for each State to give the U.S. totals.  The biggest use of
the micros in this application has been to assemble the data to a
Regional level and at the same time provide printed worksheets to the
Board members for setting the estimates.  We usually have less than
one hour to prepare the data for review by the Board.  I can enter the
indications on .the PC, and within about five minutes print out the
spreadsheet with all the indications on it.  In the past it would take
almost 6 ne hour with two or three people doing the calculations and
checking the totals manually to complete these tasks.  Furthermore,
these -time savings permit extra time for reviewing the data and
ensuring they are correct.
In conclusion, we find ourselves putting almost all of our
calculations on spreadsheets, and even people who have little
experience on computers are able to effectively use the micros.  In
most cases there has been a considerable time savings, coupled with
improved data quality.
               MANAGER'S PERSPECTIVE ON THE ACQUISITION
           AND USE OF MICROCOMPUTER-BASED GRAPHICS PACKAGES
               Richard W. Bays, Internal Revenue Service
The capability to display statistical data graphically as opposed to
tabularly has been greatly enhanced with the advent of graphics
software packages which can be used on microcomputer equipment.  This
paper summarizes the experiences of one small statistically-oriented
organization, the Projections and Forecasting Group, a component of
the IRS Research Division, in using microcomputer-based graphics to
upgrade the quality and impact of its products.
Mission of the Projections and Forecasting Group (PFG)
Until 1983/1984, the Group's projection activities were completely
mainframe bound.  All consequential projections were performed at a
remote IRS -computer facility in a dumb-terminal, time-sharing mode.
There was no graphics capability in this system.  Even tabular
information was difficult to extract in a format which was ready for
camera-copy reproduction.
The introduction of 16-bit 10 MB hard-disk micros into the Group in
early 1984 radically altered work processes within six months:
                                 -51-
--   Lotus 1-2-3 spreadsheet software was used to format smaller
     projection projects.
--   A downloading- capability was created so that large-scale
     computations could be done on a mainframe, with numbers dumped
     into preformatted tables.
--   A variety of different tables were created which allowed more
     rapid scanning for errors or problems in projections.
--   Data transmission arrangements were made with key users so that
     data previously supplied in hardcopy only could be provided
     electronically, thereby facilitating further analysis by the user
     without data re-entry.
--   Experiments with Lotus 1-2-3 graphics suggested that much could
     be done to present analytical information and projection
     highlights in pictures rather than words or spreadsheets.
     Graphic representation of data would expand the managerial and
     executive audience.
Graphics Experimentation
Early experiments in presentations were done using Lotus graphics and
a dot matrix printer.  The Group found Lotus graphics satisfactory but
limited in the quality of presentation both in terms of sharpness for
reproduction purposes (a printing problem) and sophistication (a
software problem).
The problem of quality reproduction was solved by acquiring a six-pen
Hewlett-Packard plotter. Tests and discussions with other
organizations showed eight-pen plotters to be too slow, too
complicated and too expensive.  The second problem, sophistication and
flexibility of presentation, necessitated a software survey.  A number
of different packages were reviewed against survey criteria:
--   compatibility with Lotus 1-2-3 files,
--   menu structure,
--   equipment compatibility, and
--   memory demands.
Chart Star, software marketed by Micro Pro, Inc., was chosen at the
end of this survey.  Chart Star has a wide range of charts and
graphics to choose from and is in all ways superior to Lotus 1-2-3
graphics.  For example, bar graphics can be three dimensional; it has
exploding pie-charts, and a number of other options, all prefaced with
easy-to-use menus.
With both hardware and software in place, the Group began to routinely
use graphical data representations in its reports and documents.  We
discovered that once it was demonstrated what microcomputer graphics
could do, demand for such presentations increased exponentially.
Consequently, the Group added to its repertoire a software package
called Statmap which permits shaded/cross-hatched representations of
data on maps at the zip code, county, stateand U.S level.
                                 -52-
Some observations on Impact
There is no question that microcomputer graphics have greatly improved
the quality of and increased the audience for Projections and
Forecasting work productions.  Managers and executives-are more aware
of-key trends and have tended to ask for additional data and displays
on them.  There are organizational impacts, however.
Search time Finding the hardware and software which suits the
presentation requirements of the organization requires time--staff
time.  Hardware and software specifications need to be reviewed,
demonstrations arranged and procurement initiated.  Getting the right
technology for your needs requires carving out enough time from
everyday-work to do an adequate job of review.
Implementation - Training employees to use graphics software is not
usually a major issue.  However, selecting the right graphics to
demonstrate the point in question is a more substantive issue.  Doing
so requires consultation and testing and adds to production time.  The
longer the review chain, the more frequent will be requests to alter
graphic presentations or present data,in some other manner.
Integration - Good graphics create their own demand.  We found that
top managers expect textual material with high data content to be
graphically illustrated.  We have not found software which does a good
job of integrating text and graphics for camera-copy development.
This means graphics and text are separately produced, then cut and
pasted into camera copy.  The result is that making changes becomes
more difficult than simply making textual adjustments, on a word
processor.
Color - All good graphics packages can give video displays of charts
and graphics in color.  With a plotter, camera copy can also be
produced in color as can overhead projections.  The rub develops in
moving from camera copy to production.  Few organizations have the
color xerography necessary to make color reproductions, although this
may be coming.  For the interim, graphic presentations need to be
developed with a black and white final product in mind.
Competition - Good analysts quickly realize that graphic data
representations help sell their products.  Consequently, there is
competition for the use of both equipment and software.  If the
organization has micros with either built-in or external hard disks,
the equipment side of the equation can be solved by loading software
into the hard disk. The number of software packages needed will
depend on the volume output of the Group.  In our case, during peak
production, one package for five, analysts seems to meet the need.
Supervisors and/or reviewers need also to guard against "over
illustration," a problem which can occur once analysts have seen the
power of graphical presentations.
                                 -53-
       CURRENT APPLICATIONS OF UNIX-BASED MICROCOMPUTER SYSTEMS
             Brian Carney, U.S. Department of Agriculture
The situation in the National Agricultural Statistics Service-(NASS)
is unusual in that several of our microcomputers are based on the UNIX
operating system.  Instead of having just one user on a machine, we
have multiple users and multiple tasks per user.
First, a little about what the Research Division of NASS does.  There
are three branches in the division: Remote Sensing, Yield Research,
and Sampling Frames and Survey Research.  The Remote Sensing Branch
uses a UNIX system with exotic graphics hardware for satellite image
processing.  The Yield Research Branch is using a UNIX system for
editing programs that are submitted to a mainframe computer.  The
Sampling Frames and Survey Research Branch, the group I am in, works
in three general areas: nonsampling error research, area-frame design
and construction, survey design and analysis, and statistical
consulting.
Much of the work of the latter branch involves large datasets from our
agricultural surveys and requires the use of a statistical package on
a mainframe.  We use SAS primarily.
Efforts to reduce nonsampling errors in telephone interviews is what
led to our use of UNIX.  We became research partners with the Center
for Computer Assisted Survey Methods at the University of California
at Berkeley.  That group had been working on a system for computer-
assisted telephone interviewing (CATI).  Using the CATI system, we
replace paper questionnaires with a terminal, and the interviewer can
enter respondents replies directly into the computer.  Error checking
is performed while the respondent is still on the telephone.  CATI was
developed and runs under the UNIX operating system.
For a while back in 1982, when we were just starting our work with
CATI, we tried connecting the terminals over telephone lines at 1200
baud; and at that slow data rate it takes a while to paint the screen,
delaying the interview.  Shortly thereafter, some multiuser
microcomputers became available that ran UNIX; and the CATI system was
ported over to them easily.  The cost of the system was not too bad,
something under $40,000; so we were able to procure an inhouse system
for a work group of about ten people.  That was our introduction to
UNIX.
Once we had installed the machines, it was clear we could do quite a
bit more with them besides run CATI. what we have done falls into the
category of analysts' support. There is a video display terminal on
each desk, with access to the mainframe systems both interactively and
in batch mode.  Programs for analyses are written on the UNIX system,
using the native full-screen editor, and are transmitted to the
mainframe for execution.  This avoids the cost of being online to the
mainframe.  By dialing in to other systems or having them dial in to
the UNIX systems, we can transfer information on the word processors
and PC's in and out of UNIX.  The electronic mail system has been very
useful to the managers; and some of the material usually covered in
staff meetings is now mailed to the staff using UNIX, and they can
read it at their convenience.  The electronic mail system extends to
the research UNIX systems in Washington and the field.
                                 -54-
Of course, UNIX has tools to facilitate programming, technical writing
and publishing; and these are widely used on our UNIX systems.  All
this capability is right there under UNIX; many operating systems are
not so rich.
The communications capability of these systems is such that they are
accessible-from remote dial-in terminals in the same way as mainframes
and minicomputers.
I mentioned the specific use of these systems for CATI research.  The
Agency, because of the interest in making CATI operational, has
procured twenty to twenty-two UNIX-based machines for our field
offices.  Besides running CATI these will be used for direct data
entry, transcribing the responses to paper questionnaires we still
generate in field interviews.
Software costs have generally been lower since one package is purchase
per system.  The individual price is high, but is generally cheaper
than PC software on a per-user basis.
I have mentioned office automation.  All the analysts have a CRT on
their desk, and prepare reports and analyses through that terminal.
We can edit and review before the manuscripts hit hard copy.  This can
be a real time saver.
There are several spreadsheets available A number of imaginative
simulations have been done on spreadsheets under UNIX by our group.
Database software is available, but is used now primarily for
administrative purposes.
The statistical analysis capability under UNIX is a limitation right
now.  Probably one of the most complete statistical. languages
available is the AT&T Bell Labs.  However, it is very large and does
not run on every UNIX system because of certain hardware and memory
requirements.  P-Stat and Minitab both run under UNIX, but would have
to be converted to run under specific systems.  SAS might run someday
under UNIX, but probably not for a while.  The SORITEC system that
Linda Atkinson mentioned is available, too, and is good for
econometric analyses.  A system called UNIX/STAT is available for
basic statistics and psychometric analyses.  Because of these
limitations, we do not do, much statistical analysis directly on the
UNIX systems.
Now a little about UNIX itself.  Among its disadvantages is its size:
it requires as much as twelve megabytes of disk for the operating
system and utilities, of which there are hundreds.  The system can
appear to be quite complicated, and it is usually necessary to have
someone available to help out with solving system problems.  The
commands are a bit terse, most only two to four characters long, and
that can be a problem for new users.
Among the advantages of UNIX are its flexibility and power.  You have
an operating system that operates on minicomputers, mainframes and
microcomputers and has the same essential capabilities across them
all.  UNIX is powerful because you can do extremely complicated
functions with a very small number of keystrokes.  The multitasking
means that a user can have several programs operating simultaneously.
When I use viewgraphs, I
                                 -55-
set each one up interactively, then have the system actually draw the
viewgraph in the background.  While it was drawing, I could go on to
the next viewgraph.
There are hundreds of utilities native to UNIX that are useful for the
full range of tasks from text processing to database to programming.
The UNIX hierarchical file structure is important for managing large
numbers of files.
Some of the new UNIX systems feature displays with multiple windows
that can run different processes at the same time.  For applications
requiring detailed graphics or typesetting, several new systems use
bitmapped screens.  What you see is what you get, even to the
different fonts, special characters, and drawings.  The results,
printed on a laser printer, are quite good.  It is not unlike the
Macintosh, but with a more substantial operating system.
Decision support on a microcomputer is the idea of having all the
functions an analyst needs for assembling, analyzing, and presenting
data, in both graphic and text form.  The systems are flexible enough
to manage both text and graphics in the same files.
UNIX is also developing sophisticated networking to allow shared
access to file systems, but with separate processors available to each
user.
We have found the UNIX systems to be extremely useful because of their
power and the wide variety of utilities built into the operating
system.  But we have been limited, by the small number of statistical
packages available under UNIX, and still rely on a mainframe for most
of the statistical analyses.
                       EQUIPPED FOR THE FUTURE?
            Paul Dobbins, U. S. Department of the Treasury
The Office of Tax Analysis (OTA), which is part of the Treasury
Department's  Office of Tax Policy, is responsible for three major
functions.  First, providing revenue estimates for the Administration
for the budget and its quarterly reviews, as mandated by law.  Second,
providing on demand revenue estimates of tax proposals.  Third,
providing economic analysis of current and proposed tax legislation,
often on very short notice.  Timeliness is of the essence for the work
of OTA to be of any impact during the sensitive, if fast-paced
negotiations that are characteristic of mark-up whenever tax bill is
pending.
OTA specializes in what is called micro-simulation, which in our case
is simply defined as modeling the responses of tax or household units
to tax-law changes on an individual basis and weighting up the sample
resists to get population estimates.  Our data is primarily tax-return
information, but we are making increased use of other sources.
Even though our data files are relatively small samples, they are
still large data sets and have made us largely mainframe bound. (Micro
data does
                                 -56-
not imply microcomputing!) But we have begun to use microcomputers
increase overall office productivity and, having seen the light, we
are attempting to push our frontiers even further out to the limits of
the possible.
We would ideally have a triad of computers/computer systems supporting
our work.  At the top or first level would be the individual
microcomputer workstations bringing the burgeoning new wave of
software development into the hands of tax economists, and lawyers.
Currently only, our economists and computer specialists have
microcomputers: the Z80-based CP/M machine, the Superbrain (TM), a
fine machine in its day but certainly no longer in the mainstream due
to memory and system limitations.
Our economists have benefited greatly from having these
microcomputers.  Most of the staff were quickly converted into very
proficient users of Wordstar (TM) and Supercalc (TM), and several have
demonstrated considerable programming talent.  The programming staff
has found its burden lightened somewhat by a transfer of focus by
staff wherever possible from the mainframe to the micros.
But as described above, we are mainframe bound by the nature of our
micro-simulation bread and butter.  The mainframe is the third and
fundamental level of our triad.  What we're hoping to implement is a
second level: a mini- or supermicro-based network linking the first
and third levels together into a smooth, efficiently running system.
Why this is needed can be illustrated by a generalized paradigm of how
OTA often does its work.  A particular tax proposal will need to be
reviewed and analyzed in the course of an afternoon.  First, the
appropriate mainframe simulator will be run and the results brought to
the staff, who may then input the numbers into an analytical framework
they have developed (e.g., in Supercalc (TM).  These modified results
then may be inserted into a document residing on a third device, a
stand-alone word processor.  Finally, time being we have results, but
not without considerable carrying paper from office to office and re-
inputting numbers at each step along the way.  This is, I submit, an
old story for many an office.  We have duplication, wasted effort.
And yet the very presence of microcomputers has made the process
faster and more reliable.
A triad of micro-mini-maincomputers seems to fit our demands
perfectly.  The new Micro-Vax (TM) may very well fit into the-second
slot.  The workstations could easily be IBM PC's and look-alikes or
DEC PC's, while any mainframe fits the foundation (we're running on a
UNIVAC 1100/81,series and will soon have an 1100/92).
Our ideal system would make it, possible to share software and
software tools at the local and network levels, and allow us to easily
move text and estimates across and among all of its levels.  The
intermediate level also goes some way towards narrowing the software
gap between mainframes and micros by providing considerable computer
power and much of the much of the latest software design.  We can only
anticipate the hardware advances that will narrow the gap even
further.
Finally, it cannot be overemphasized that the introduction of
microcomputers has fundamentally altered the way OTA does business.
But perhaps the
                                 -57-
greatest lesson learned was how much better we can do than our current
partial solution.  In an environment that features considerable
interaction among many different specialists, a network unifying all
computer assets seems to be the only way to go.
   CONCERNS ABOUT DATA INTEGRITY, SECURITY, AND ACCESSIBILITY IN AN
ENVIRONMENT WHERE MICROCOMPUTERS AND MAINFRAMES ARE INTERFACED
             Dick Shively, U. S. Department of Agriculture
A.   Background (History)
The NASS is known as a data collection agency and reporter of
agricultural statistics.  In this line, a substantial amount of effort
is directed toward list maintenance and data collection.
As the agency moved into single- and multiple-frame probability
samples for more and more indications, large-scale computer resources
became extremely critical to allow evaluation of results in a manner
timely enough to meet the reporting schedules, as well as to support
much more exacting requirements for list maintenance.
A large proportion of the field office (SSO) efforts are directed
toward maintenance of the lists associated with their individual
state, as well as collecting and analyzing their data.  The NASS
estimating procedure normally consists of each SSO preparing the
indications and estimates for their individual state; then the Crop
Reporting Board reviews all of these estimates to arrive at regional
and national estimates.
Since many diverse commodities are estimated, and sample sizes are
fairly large to cover the desired geographic areas, data conversion is
a major task.  For this reason, all of the SSO's are mainly equipped
as remote-job-entry sites with high-volume data-entry equipment.
While the NASS has been a proponent of "generalized" software for some
time, operations of this software at multiple sites required that
installation and maintenance activities were duplicated for each site.
The NASS history in mainframe computing has progressed from each of
the field offices (SSO's) being responsible for obtaining their own
computer resource locally to the current approach of providing a
single large-scale commercial time-sharing-network vendor who can
provide adequate resources to satisfy all of the agency's
requirements.
Until the bulk of national interest surveys were processed on the.
time-sharing vendor's equipment, it was sometimes difficult to
determine what procedures were used to obtain indications and even
more difficult to review multiple state outputs from different
reporting formats.  Even shared required modifications to be used on
different brands of hardware, and there was little assurance that
these modifications would always provide identical results.
                                 -58-
The introduction of microcomputers into the NASS offers some
possibility of encountering the same problems recognized when local
computer capacity was utilized.  However, with care in selection of
commercial software to ensure standardization where necessary, these
devices offer substantial opportunities for improvement in personal
productivity.
B. Data Accessibility
The typical processing method for survey data in the NASS for national
surveys is for the Washington, D.C., staff to provide general
guidelines for the type and amount of data validation to take place,
as well as the summarization techniques.  The SSO staff provides
detailed validation specifications appropriate to their state, taking
into consideration any specialized local conditions.  SSO personnel
will collect and validate the data, and summarize to the state level.
The D.C. staff consolidates all of the state level information into
regional and national values. The majority of the post survey analysis
is also accomplished by the D.C. staff, using the data from each SSO.
For this type of approach, the data values need to be available to
people at widely-scattered locations.  Storage on the microcomputer
devices does not satisfy this requirement, since only one user at one
location can access the data.
The same accessibility requirement also holds for Crop-Reporting-Board
released values.  The D.C. staff reviews the SSO recommendations in
establishing regional and national values.  Following the Crop
Reporting Board review during lockup, the state, regional and national
estimates are made public on a known date and time.  These estimates,
normally released at 3:00 p.m., must be immediately available to each
of the SSO's to prepare reports emphasizing those items of local
interest.  In addition, many people outside of the NASS are allowed
direct access to these published values.
Microcomputer usage in the NASS appears to be best adapted to play a
support function, rather than providing a source of computational
power.  This includes primarily office-automation functions, such as
word processing and spreadsheet analysis.  Because of the volume of
data, the stringent time constraints imposed on processing the data,
and also the geographic distribution of processing, the data needs to
be stored in a common repository.  This allows each individual SSO
accessibility to their own data, while still making the same data
available to the review staff in D.C.
A recent "Viewpoint" column in INFOWORLD made the point that data
security and microcomputer-enhanced productivity are incompatible.
The relationship is very strong, in that the more stringent the
security measures used become, the less accessible the data is with
corresponding losses in productivity.  This column's analogy is that
security on microcomputers is similar to inventorying pencils and
paper.
One current weak point with data accessibility, both from the
mainframe and the micro standpoints, is a lack of communications
ability.  A reliable file transfer ability, consistent from machine to
machine, and allowing usage of the strong points, of both mainframe
and micro, will enhance our data processing capability.
                                 -59-
C. Data Integrity
Storage of data on microcomputers in the NASS environment, for more
than a temporary working basis, has a tendency to lead to integrity
problems.  This happens whenever the same data is stored in more than
one location, whether it is on a mainframe or microcomputer, or in a
file cabinet.  Anytime that a value has occasion to be changed, unless
all occurrences are simultaneously changed, someone will likely accept
the wrong value as correct.
Version control is another name for data integrity, and when using
individual stand-alone microcomputers this is difficult to handle.
Each machine has its own copy of software, so whenever a new version
becomes available every micro user must be provided with a copy.  This
is compounded by those machines having only floppy-disk drives, where
the same software and/or data may appear on many diskettes.  To
upgrade to the new version, all copies must be located and modified.
The problem of maintaining data integrity on microcomputers is the
same one we have been battling for years - a single copy must be
identified as that containing the "correct" values, and any accesses
must be directed to this copy.
Local Area Networks (LAN) provide some help in those situations where
data is needed in a small area, where all users can be contained
within the LAN.  In this way, each user of the LAN can have
accessibility to the same values, which will alleviate the integrity
problem.
D. Data Security
The most secure system is undoubtedly manual although it may not be
very productive.  Microcomputers are not the place to store data that
is, sensitive, unless special considerations are made such as a locked
environment and limited access.
Floppy disks are extremely easy to duplicate and just as easy to carry
away from an office without detection.  However, this is a human
security problem and not a microcomputer security problem.  An
authorized person can just as easily remove a printout from a
mainframe computer or pages from a record book in a file cabinet.
A "Jim Seymour" column in PC WEEK suggests increasing security by
locking up, diskettes when not in use, checking them out for usage,
and encrypting the data stored on the micro.  To me, this seems
contrary to the usage of a PC as a productivity tool.
The main security consideration that should be given to microcomputer
usage with sensitive data is the RF or radio-frequency emission
associated with them.  Using fairly inexpensive devices, these
emissions can be recorded from many feet away from the computer and
reconstructed into the data that was being processed.  This can be
solved with Tempest machines or RF shields.
In closing, I would like to say that I think that microcomputers are
an excellent productivity tool.  We need to be aware of their
strengths and limitations when designing projects for them to
accomplish.
                                 -60-
                         QUESTIONS AND ANSWERS
The following discussion reflects questions and answers related to the
"Applications of Microcomputers" Session, which involved the Chair,
Speakers, and certain members of the audience.
Mr. Steele: We have heard several interesting applications discussed
here this afternoon, and the purpose was to give you some sense of the
breadth of applications that microcomputers are being used for.  I We
have seen applications that were very simple uses of spreadsheets,
spreadsheet templates, and data bases, ranging on to econometric
projections, graphics, and then the sophisticated time-sharing
multiuser UNIX-based systems.  We wanted to give you a sense of the
kinds of things that are being done so that we can talk more about
where we are going.  What more can we expect out of microcomputers and
what kinds of developments would we like to see in terms of hardware
and software to achieve greater productivity from the use of
microcomputers?
In that light, I would like to start off and ask several questions.
Ql: Linda, you seem to be a strong proponent of the IBM
microcomputers. Could you explain why?
Al (Ms. Atkinson): Being with the Federal Government, I am probably
supposed to start with some sort of a disclaimer about products;
therefore, my comments do not constitute a product endorsement.
When we first started acquiring PC's, it was done by our Economic
Research Staff; and we had a proliferation of all kinds of machines
including some that are no longer being made.  It became clear that
not only was this hard to-support, but we were going to have problems
in being able to move data from one machine to another and possible
communications problems later should we want to network them.
Also, the situation that we are seeing how is that the newer software
such as SPSS PC and SAS, are being developed first for the IBM
machines and later, if at all, for other machines.  So if you are on
these other types of computers, you are going to lose some time during
that waiting process.  I would say by now that probably about eighty
percent of our PC's are IBM compatibles or IBM.
Q2: Brian, I see you seem to  be a very strong proponent of the UNIX-
based machines; and you give very strong arguments for, well, a good
discussion of the relative strengths of them.  We have heard a lot of
sales hype about UNIX being the operating system of the future. Is
that really going to happen?
A2 (Mr.  Carney): AT&T would certainly have us believe UNIX will be
the operating system of the future.  UNIX offers some special
capabilities that you don't get on other ones.  One in particular is
software compatibility across UNIX machines.  As an example, CATI
software will run on practically, anybody's UNIX box, anywhere.  So,
we have acquired a degree of vendor independence right there.
                                 -61-
But, as far as hardware is concerned, say for example you want to
network machines together; at that point you are down to a level where
UNIX doesn't really do you a whole lot of good, because the individual
vendors choose to implement UNIX differently on different types of
hardware.  There is some effort to relax those restrictions.  Sun
Microsystems has a hardware and software implementation of networking
that can be done that is largely vendor independent.  But that's not
something in general that you can get and walk out with today.
UNIX suffers from being extremely complicated.  As I mentioned
earlier, you have to have an expert around or available when the
machine goes down.  Because of the size and complexity of the system,
it does take the user quit a while to be productive.
In terms of the so-called popular software (Lotus, Symphony, etc.),
none of that is available under UNIX.  Whether it becomes available
depends a lot on what AT&T can pull off with their market.  They say
it will happen, but we haven't seen it yet.
Q3:  Paul, as you look towards implementing your design of the future,
having this triad or three levels, do you anticipate. making them all
one brand or one vendor and one standard software line; or what kind
of problem do you envision in the interchange of data between
packages?
A3 (Mr.  Dobbins): First of all I would say that we don't have enough
experience at the moment to say exactly what it is that we will have a
year or so down the road.  I personally would look forward to having
more flexibility and not essentially going toward one standardized
system or one standardized software.  We ultimately might, want to cut
bait with our mainframe.  As the minis become more powerful and we
have super-micros on the desk, we may begin to use the mainframe for
initial processing and then download a data base to one of our PC
machines.
I am looking forward to experimenting for a while before I will be
able to say too much more.
Q4: Rick, you mentioned that you had some problems with output
devices.
Could you expand on that a little?  What problems have you
encountered, and what could the vendors do to solve some of these?
A4 (Mr.  Hayes): What we are looking for is a good vehicle for
integrating our text with our pictures or graphics.  We haven't found
that and we still are looking.  One of the things we find is that
there are a tremendous number of products out on the market.  We are a
small shop and we are doing our own research.  So far we haven't had
any luck with Text/Graphics integration.  Once, you start
experimenting you find that you can spend a lot of time looking at
various packages.
The other thing that ou will find is that people will suggest
different software you can look at, you can try.  Each of these
experiments takes time out of your production so there is a tradeoff
between finding something that will work for you and finding the best
and latest product.  Once you have found something which works for
you, I think you should stick with it for a while and find out its
complete capabilities.  I When we talk about output devices, we are
having problems withtfinding print devices that will handle
                                 -62-
graphics, pie charts, diagrams, as well as text, at a reasonable
speed.  I think most of the products presently available have problems
with integration of text and graphics.  It is not an insurmountable
problem, but it slows you down.
Q5:  Dick, you expressed several concerns about data security when
interfaced with mainframes. Does that mean that you think people
shouldn't have microcomputers hooked to the mainframes or that we
should lock everyone's door or what?
A5 (Mr.  Shively): All I was saying on that was that the microcomputer
has a problem with security.  Mainframe computers are very well
adapted to securing data, securing devices, making things pretty
secure.  Microcomputers, by design, are a productivity tool.  If we
try to add security to it beyond what is necessary, we have reduced a
lot of the productivity gains that we have possible there.
Mr. Steele: At this time I would like to open the floor up for general
questions.  Please stand and identify yourselves by your name and
agency before asking the question.
Qg:  This is a question that is appropriate to organizations that
haven't yet capitalized on current technology and, are looking to get
into the business.  What is the appropriate level of computer power
for economic feasibility forecasts and statistical work? I wonder if
anybody can comment on the relative merits of waiting until 16-bit  --
-microprocessor technology is developed.
A6 (Mr.  Hayes: As it turns out, we recently had a contractor come
through and evaluate our operation in terms of what ought to be our
computer configuration in the future.  Basically, their conception is
that anybody who is working with any sizable base is not going to get
along without it. Their suggestion is that you I use the mainframe for
your heavy duty computations and as a storage device to deal with some
of the security problems we talked about here.  They then are
suggesting to us, at least, that we look now to microcomputers, 16-bit
machines that can network into the mainframe and be used either in the
stand-alone capacity for test purposes to try out graphics, to upload
and download, or I as remote terminals.
I know we are going to maintain our mainframe capabilities while doing
as much as we can on micros, because micros are much simpler and
flexible to use by analysts.  Where we need to spend time is on
gateway architecture that links our micros to the mainframe and allows
analysts who are not experts in programming to get in and out of the
mainframe and get data in and out of software packages.
A6 (Mr. Carney): In the UNIX-based workstations that I am familiar
with, the real 32-bit technology comes on a single chip with a
powerful box.  I think people are looking towards Motorola 68020 which
should be in production sometime in the Fall, 1985.  It takes a little
while, not I too long, under UNIX for the software to catch up with
it.  It may be a while before it's really fully mature as you are
looking for right now.
You've heard this old song before, but you really want to find what
software you want first, and then-figure out what sort of boxes you
can afford, that it will run on.  You really are looking for
productivity after all.
                                 -63-
Q7:  Each of the panelists has described a decentralized system,
especially one using spreadsheets and/or spreadsheets with something
else.  How do you sure the integrity of the data that you are using;
and second, how do you insure that whatever statistical standards may
exist in your agency are being followed in those decentralized system?
A7 (Mr. Carney): I can talk about it a little bit in the research
environment.  Basically, we always have to use the data on the
mainframe as the benchmark data.  That is the correct data, and
anything we pull off has to be pulled off checking protocols; and we
can't change those numbers.
As far as the statistical standards are for the research unit, you
pretty much have to depend on the review process, review by our peers.
A7 (Mr.  Shively): I second Brian's statement.  We basically consider
the mainframe data to be the official source unless some special
circumstances exist where there is no need for it to ever be on the
mainframe.  But for any data that is shared or that is nationwide in
scope, the mainframe is the official source of control. If you pull a
copy of that to your micro to run it through a model, you are working
with data that at that point in time is not official copy.  It may be
a copy of the official and you can Use it for your model or plan --
anything like that -- but if you want to go to publication with it you
need to go back to the mainframe for the official copy.
QB:  This is a comment. We are using more and more graphics lately.
There are some packages on the market which were released recently
that will capture the picture of the U.S. map -- then you can enhance
that map.  You can add titles, text, whatever you want to these
graphics.  The program works in the background -- one is the graphics
partner. You can call it like the Sidekick package.  It captures the
picture you have on the screen and you can enhance and modify the
picture. You have the integration of text and graphics. I suggest that
you try SMART; they have the software package.  Also, take a look at
GEM which is just coming out by DIGITAL Research.
Q9:  I have a question for Linda Atkinson. I think you mentioned a
spreadsheet that ran for forty-eight hours? Did you have an 8087 (math
ematical functions) processor.
A9 (Ms. Atkinson): Yes we did.  It is a very large model and this is
the simplified version of it on the micro, and yet it took that long.
It's very large and it ran very long; but yes, it was an 8087 chip in
there.  They are hoping that the AT is going to improve the situation.
If not, they may not be able to ultimately move to the micro but will
need to use the mainframe as well for that model.
Mr. Steele: I have one anecdote that I would like to share with, you
about computers and applications to computers.  I became involved with
microcomputers in 1978.  By 1980 I had most of my functions already
automated in my office on the computer, and I called up one of my
colleagues who was Secretary of the Crop Reporting Board to get some
information from him on when one of our employees had last been in on
the Crop Reporting Board.  He said, hold on just a minute, I need to
check my data base, and within thirty seconds he had an answer back
for me.  He read back all the
                                 -64-
times that this guy had been in and when he was next scheduled and I
was really impressed.  I couldn't believe how quickly he had all of
that information.  I didn't have a nice data base like that, so I
asked him what kind of data base he was using, and he said, "file
cards."
certainly, the purpose of telling that anecdote is to illustrate that
there are certain applications that are best left to a manual
procedure, and then oftentimes I encounter people trying to automate
procedures that aren't well defined manually.  I think that any time
people try to automate procedures that aren't well defined manually,
they are expecting magic; and they usually end up with a lot less than
what they are asking for.
Ms. Atkinson: I would-like to make one comment about acquiring
statistical software.  If any of you are looking for, software or a
good source of reviews or products that people have tried, there are
at least two electronic bulletin boards that I am aware of.  Capital
PC users group is a special interest group for statistics.  Charlie
Hallihan who is one of the chairs of that group has left some
information on the desk outside which will tell you how to access
their bulletin board or attend their meetings.
There is a SAS users group, even though SAS is not yet on the micro.
This is a group right now who likes SAS and also uses micros.  I guess
they like to discuss their applications of getting data back and forth
from SAS.  They have a bulletin board also.
A paper was presented on that at the SAS users' group meeting last
month which I think had the phone number of the bulletin board.
Otherwise, you can contact me.  As I said, there is software available
on these bulletin boards that people are willing to share.  There is a
SAS macro on the SAS users"group bulletin board.
                                 -65-
                       SESSION ON EXPERT SYSTEMS
                           SESSIONS SUMMARY*
Both the DATAPLOT and Editing and Imputation systems described here
were not developed by computer scientists or knowledge engineers but
by subjectmatter specialists who were presented with new tools to
assist them in improving their jobs.  Although "expert system" tools
and techniques were developed by a community of researchers who happen
to call their field "Artificial Intelligence," the tools and
techniques can be considered to be useful in their own right without
the necessity to call the result "Artificial Intelligence" systems.
In fact, there is good reason to say that none of the existing expert
systems is truly intelligent or even expert.  A true expert has the
ability to learn new rules in his specialty and to apply common sense
reasoning in cases where specific rules don't happen to reside in his
"knowledge base." Both "learning" and "common sense reasoning" are
areas of artificial intelligence research in which there are only a
handful of active workers and in which progress has been slow.
Contemporary "expert systems' neither learn nor exhibit common sense
behavior when it is warranted.  But, as a set of tools and techniques,
expert system technology has proved to be useful for some specific
applications.  We have seen two examples of such applications today in
Mr. Filliben's DATAPLOT system and in Mr. Greenberg's edit and
imputation software.
I think it is worth noting that these successful expert system
examples were done by mathematicians and statisticians, rather than
artificial intelligence specialists or even computer scientists.  Less
than two years ago, there were dire warnings that expert system
techniques could not be generally applied because there were so few
PhDs being granted to people who had specialized in artificial
intelligence research. what we are finding though, is that the
techniques important for developing expert systems can be taught to
people in other specialties.  In fact, many organizations (.including
the Digital Equipment Corporation and IRS), given the choice of
training artificial intelligence researchers in application domains or
training subject-matter specialists in the tools and techniques of
artificial intelligence have opted for the latter.  The Bureau of the
Census and the National Bureau of Standards show that good subject-
matter specialists are perfectly capable of learning the techniques
without any deliberate training program by their agencies.
One of the reasons  to have a panel on expert systems at a conference
on statistical uses of microcomputers is that such systems.as DATAPLOT
can be adapted to personal computers as soon as the personal computers
are powerfulenough to accommodate them.  To expand on that theme please
note that
________________________________
*Norman Glick, National Security Agency
                                 -67-
today's personal computers are already much more powerful than the
"supercomputers of the early 1960's".  We are guaranteed, given what
can already be seen in computer engineering laboratories, that the
inexpensive personal workstations of the future will be powerful
enough to accommodate the kinds of systems that need mainframe
computers today.  But the existence of that future power for the
benefit of an individual will make even more important some of the
research that hasn't been discussed today but is part of what the
artificial intelligence research community is concerned with.  The
ability to provide a user model to accompany an expert system
addresses some of the points made in the talks and the question period
today.  Users do have different levels of sophistication and expertise
of their own.  We would like the system to accomnodate to the needs of
the user, even to adapt to the changing expertise of a single user.
The same person who might need substantial help in using a system for
the first few times, might ultimately consider verbose assistance to
be a nuisance.  The ability of a system to adapt to the evolving needs
of such a user is a subject of active research in the artificial
intelligence community today.
Since it was announced that this session might be on "pure fantasy,"
and since what we have heard from the Bureau of the Census and the
National Bureau of Standards has been on eminently practical systems
(whether they are called expert systems or not), perhaps we should end
with some speculations that some might consider fantasies.  One class
of artificial intelligence research that promises to have relevance to
statistical systems of the future is automatic programming.  Both the
editing and imputation and the DATAPLOT systems required that the
statisticians and mathematicians write programs.  Whether they were
intentionally building expert systems, unconsciously building expert
systems or simply writing a program to assist in statistical analysis,
they needed to provide significant detail about how the computer
should do what they wanted.  If sufficient expertise of the
programming art can be captured in an expert system and can be
combined with sufficient expertise in a particular domain, even one of
the domains we've heard about today, then the combined system might
permit a user to state what he wants done, rather than the details of
how he wants to do it, and a program to perform the job could be
generated automatically.  To a modest extent, so-called fourth-
generation languages provide existence proofs of such systems today.
These fourth-generation systems work in very limited domains (e.g.,
payroll And inventory control), but there is substantial research
aimed at increasing the set of applications for which such approaches
are practical.  Some even see this class of activity as the future of
software engineering.  Please note that some differences exist in the
life cycle of standard software relative to the life cycle of
"artificial intelligence " software.  It is clear that current
software engineering techniques will not provide the quantity and
quality of software required in the future.  More statisticians,
mathematicians, and psychologists will need to tell computers what
they want done in the future without computer-specialist
intermediaries.  Let's hope the automatic programming "fantasy"
becomes less fanciful so that, in the future, more subject-matter
specialists can be their own "knowledge engineers' rather than to be
dependent on programming specialists.  Statisticians shoudn't have to
spend inordinate time learning the details of how to use specific
computers when their talent is to apply their mathematical and
statistical knowledge.
                                 -68-
                             INTRODUCTION
                Terry Ireland, National Security Agency
It's possible that the organizers of this workshop, and I was one of
them, wanted to have one session on pure fantasy and this might be it.
Building expert systems that model in software the behavior of human
experts, and evolve in a natural way so you can more clearly
understand the expert, is so clearly impossible that there must be--
and is--an unlimited amount of high-priced advice on how to do it.
Some statisticians may argue that random sampling and surveying
procedures on computers can already model the experts, so we really,
perhaps, have two questions:
     What do we mean by "an expert"?
     If we know what an expert is and if we have one on hand, how do
     we go about modeling his behavior?
In order to give some more practicality and reasonableness to this
presentation, we have made absolutely certain that none of the
speakers is a computer scientist.  However, they are skilled
developers and users of software systems, and they have built expert
systems.
George Lawton is a psychologist with the Army Research Institute.  He
has an interest in systems that support the interface between human
factors and computer science.  He will give the introduction to expert
systems.
Jim Filliben is a statistician with the National Bureau of Standards
who has an interest in systems that model and support statistical
expertise.  In fact, one of his software systems is said to be the
most requested piece of software from NTIS.
Brian Greenberg is a mathematician with the Bureau of the Census.  He
has an interest in expert systems for data editing and imputation.
Roughly a year ago I gave a talk on expert, systems -- an abstract
talk because my practical knowledge was limited.  After the talk,
Brian came up and observed that he wasn't sure about the jargon that
was being used, but he felt that he had built an expert system.
The Rapporteur is Norman Glick.  He and I are both computer scientists
and he may try to have the last word.  Mark Winer, an economist from
the Office of Management and Budget, is our Discussant who will keep
us honest.
Computer scientists are trying to create tools for the development of
expert systems and to make them commercially available.  They are also
trying to give the impression, that they are the most skilled at
eliciting information from experts.  Thus, the once humble programmer
now calls himself a knowledge engineer.  Ultimately, a knowledge
engineer is a person (sometimes statistician, psychologist or
mathematician) who makes the most substantial effort to understand and
model the expert.
                                 -69-
                        EXPERT SYSTEM TUTORIAL
                George Lawton, Army Research Institute
A number of things have happened recently that would lead me to change
some of the things I say today had I the opportunity to do so.
Fortunately, some of those things are available to you through your
local newsstand.
One was the publication last month of a magazine called PC, not to be
confused with PC World and PC Junior, which has a section describing a
number of proprietary software packages which are available on
microcomputers; for developing your own expert systems.  Anyone who is
inclined to go out And build an expert system may look at this article
and review the software.
The other was my attendance last week at a conference at Bell Labs
which brought together a number of computer scientists and
statisticians who were all interested in what I am going to be
discussing here this afternoon was, surprised to find that there are
at least sixty people from the United Kingdom and the United States
and at least a couple o f European countries who are interested in
this subject.  No less than John Tukey of Princeton University
believes that this is the next wave of software for statistical
applications.  It seems to me that this is something of a coming
concern.
Expert systems come out of laboratories for research in artificial
intelligence.  Artificial intelligence, as I think everybody in the
world now knows if he reads the popular press, is a line of research
developed at MIT, Carnegie-Mellon University, and Stanford University
in particular, concerned with building computing machines which will
emulate high-level human cognitive capabilities.
In the earliest incarnation of artificial intelligence, primarily at
Carnegie-Mellon University, researchers tried to develop very powerful
general-purpose problem-solving algorithms which would give a user
appropriate support in tackling a problem that a human expert could
solve.  Those programs were largely failures.  As a consequence of
those failures, research in artificial intelligence has converged on
one organizing theme in the past decade: to be intelligent, computer
programs have to be able to access large bodies of knowledge.  It
isn't their deep problem-solving capabilities that make people
intelligent, it's the fact that they know a lot about the world in
which they operate, and so it must be for programs.
An outgrowth of that discovery is a type of computer program which is
essentially nothing but a large collection of knowledge and a
relatively simple mechanism for accessing that knowledge and using it
to solve various problems.  These programs are called Expert Systems.
I intend to talk about what expert systems are, about how their
software is written, about the software techniques that expert-system
developers use, and finally about what statisticians might want to do
with expert systems.
                                 -70-
             ARCHITECTURE BASED ON THREE SEPARATE MODULES
                        1. KNOWLEDGE BASE
                         2. INFERENCE MECHANISM
                        3. USER INTERFACE
                              Display 22.
Following good programming practice expert systems are modular and
they usually have at least three fundamental modules.  The first is a
collection of facts and rules in a knowledge base; the second is a
relatively simple program evaluator we call an inference mechanism.
The third is the interface with the user which gives the user the
illusion that he is dealing with something that's intelligent (see
Display 22).  That's what, makes the machine capable of pasting what
we call the Turing Test. This test is very simple.  If a person acting
as judge asking questions cannot tell the difference between a machine
and a human any more frequently than he can tell the difference
between a man and a woman, then the machine must be considered
intelligent.  Of course certain rules prevent the obvious shortcuts
(for example, a terminal should be used to ask and to answer the
questions).  And, of course, the respondents (woman, man, machine) are
not required to tell the truth. In the knowledge base we have some
knowledge, and it has to be represented in some form that can be used
by the computer.  Almost every expert System that I have had occasion
to study uses one basic knowledge representation (possibly
supplemented by some others): some kind of conditional structure we
call production rules.  The fundamental idea of a production rule is
strikingly simple.  It has two parts.  The first asks if something --
tome state of the world -- is true. The second part takes some
specified action if that something is true.  Programmers know them as
if-then constructs in programming languages.  In conventional programs
they are usually scattered throughout the program text.  In a
knowledge base they are collected together into a list of rules.
     1. Production Rules
          There are two classes of production rules.
          A.   Situation-Action Rules (which are essentially data-
               driven procedure invocations)
               e.g., if the data are skewed, then call a re-expression
               procedure.
          B.   Conditional Assertions
               e.g., if the case has V1 = 10 and V2 = 20, then it is
               an outlier.
                              Display 23.
                                 -71-
Production rules really break down into two different classes.  One of
them is something we call a Situation-Action Rule.  It's essentially a
pattern invoked program.  The other is really a conditional assertion
(see Display 23).
Most expert systems are based on one of those two kinds of production-
rule systems.  There are at least two other ways of structuring
knowledge that are widely used in expert systems.
Conceptual Networks. First, you may have a large number of concepts to
represent in the knowledge base, and the concepts might be related to
each other (for example, a class-instance relationship or a set-subset
relationship).  There are species of animals, and each animal species
has members.  It doesn't make any sense to represent all of the
features of each of those members, so they may be organized into
conceptual, often hierarchical, networks which show these
relationships.
Frames and other structured objects.  This is really an extension of
the idea of a record with the record containing not only the usual
type of data, e.g., name or age, but also other records as data.
Moreover, the data can be specified as a computation, e.g., if you
know the person's year of birth and you know the year, you can compute
age without storing its actual value. Frames also contain default
information to be used or computed when the required data is missing.
     1. Forward Chaining
          e.g., If P then Q;
          If Q then R;,
          If R then T;
          P therefore T.
     explanation
     By P infer Q; by Q infer R; by R infer T.
     2.   Backward Chaining
     e.g., T if R;
     R if Q;
     Q if P;
     P therefore T.
explanation
     To show T, first show R; to show R, first show Q; to Show first
     show P; P is true.
                              Display 24.
Inference mechanisms.  Because we are talking primarily about
knowledge that is represented as conditional structures, we must have
some logical reasoning process to make use of them.  For example, we
can use some of the
                                 -72-
basic rules of logic: if we know that Proposition P is true and if we
know that Proposition P being true implies Proposition Q is true, then
we can conclude from these two true statements (one a fact, the other
a general reasoning procedure) that Proposition Q is true.  Or, we can
reverse the process, starting with a goal to show Proposition Q is
true and reasoning backwards, looking for a sequence of statements
that could bring us to the desired conclusion.  Again, most expert
systems use some variation of the first and second form of reasoning
shown on Display 24.
Propagation of uncertainties, statistical reasoning.
reasoning, which we will come back to in a minute, attaches to each
conditional structure something called a certainty factor.
Statisticians might think that the certainty factor might be a
probability because some vary between zero and one or maybe a
correlation because others vary between minus one and plus one.  In
general, they are not that well motivated -- they are ad hoc.  They
are just numbers that somebody pulled out of a hat, saying this is how
certain I am that this fact is true.  The question is, how do they get
propagated through a sequence of inferences? This is a difficult
problem about which there has been much recent discussion.
Inheritance of Properties: Again, we can represent objects in terms of
a network of class relationships.  By default, certain things may
inherit properties from their class.  If Fido is a dog, then Fido is
warm-blooded, because Fido is a dog and dogs are warm-blooded.
Heuristic Rules: In certain cases, no straightforward and logical
procedure may apply, in which case you may apply what we call a Risk
It Rule.  It's a rule of thumb that says "if we don't know any better,
do this.
Meta-reasoning: Last, but not least, an expert system may have rules
about rules, a form of reasoning about process often called meta-
reasoning.  It deals with reasoning about the representation of the
problem.  All of these methods have been incorporated into the
handling of expert systems.
     USER INTERFACE
     1.   Read user input.
     2.   Provide user with useful output.
     3.   Explanation facility, to give the user a useful trace of the
          program's inferences.
     4.   Knowledge-base input and editing.
                              Display 25.
This is the most important part of an expert system.  The
distinguishing feature of an interface for an expert system, and this
is a well-designed interface, is Item 3 (see Display 25).
Expert systems, unlike other computer programs, explain the
conclusions they reach.  I would say there is no other really
necessary feature for an expert system than this ability to
                                 -73-
A third form of
explain.  Previously, I showed several logical forms for the reasoning
process.  They provide examples of what an explanation might look
like.
There you have a crude expert system.  They are relatively unfriendly.
They just say "by this rule, I infer this; and by that rule, I infer
that" and so on, until-you get to a conclusion.
Other systems are much better at knowledge representations, including
diagrammatic representations of good inference and explanations in
good English (see Display 26).
Traditional Software Engineering -- Linear Program Development
1.   System requirements
2.   Software requirements
3.   Preliminary design
4.   Detailed design
5.   Code
6.   Debug
7.   Test
8    Use
9.   maintain
                              Display 26.
How do you go about building an expert system? The methodology that
most people use is a little different from the methodology you might
have learned in a basic programming or a software engineering class.
Display 27 shows the steps that a conventional programming text might
tell you to follow when building software. By the way, expert systems
are just enlarged software systems.
Alternative Approach
     Cycles of Progressive Refinement
          Preliminary Requirements
          Preliminary Knowledge Engineering
          Prototype I
               1. Design
               2. Coding
               3. Debugging
               4. Testing
          Prototype II
               1.   etc., etc., etc.
                              Display 27.
                                 -74-
This is an alternative list that is used by most people who have
developed expert systems.  Rather than starting from the
specifications and going step by step by step through the program
development, requirements analysis and the rest to a final software
product, expert-systems developers follow an iterative process which
begins with a small program that is written and tested, then
elaborated, and written and tested again.  In fact, the programming
language used in this development methodology was designed to support
the iterative and experimental development of software.  The ability
to express your ideas in a high-level flexible language enables the
programmer to develop rapid prototypes or models of the system he
wishes to define.
LISP is the language of American artificial intelligence research.
Notice that I said research.  It's the language of artificial
intelligence research.  That may not mean that it's the best language
for artificial intelligence implementation.  It is a functional
programming language.  That means that programs written in LISP are
functions.that can be passed around just as ideas are passed around
and used where appropriate.  They are more like mathematical functions
in the sense that they have the mathematical properties that you
associate with function, rather than properties that you would
associate with FORTRAN function.  That's a formal statement that I
don't really want to defend any further than to say it really is true.
          Four Basic Components At The LISP Top Level
                         (print (eval (read)))
The first three are in the top-level loop of the LISP interpreter.
1. Reader
2. Evaluator
3. Print
4. A table of LISP objects which serve as a data base.
                              Display 28.
LISP provides a series of operations that you would have to make for
yourselves if you were-going to write a program in something like
FORTRAN.  When you invoke LISP on a computer, you are invoking an
endless repetitive loop which looks like this.  That is a top-level
interactive computing environment from which you can either ask for a
computation or define a new function very much as you would interact
with another person.  Sometimes you are computing values or
ascertaining facts; other times you are developing new ideas.  Both
activities are done in the, same environment.  The innermost
expression is ready and waiting for you to type something into the
terminal which it will read.  Then there is the high-level functional
evaluator which knows how to evaluate any well-formed expression in
LISP.  Then it will print out the results of that.  It continues to go
through this loop.  It is like a conversation (see Display 28).
                                 -75-
Invoked by the functions in this loop are three programs: the LISP
reader which includes both the ability to scan the characters entered
and to format them into a LISP expression, the LISP functional
evaluator which can evaluate or compute the expression entered, and
the LISP printer which knows how to format and print the results of
the evaluation.  Moreover, the LISP reader also stores information
about the names or identifiers read in.  Most of this information is
stored in a table that holds rules, values and names.  This table is
useful as the data base.  It is more than a simple table produced, for
example, by FORTRAN.
Why would you want to use LISP? Because LISP is interactive, you can
write a program and see how it works almost immediately.  Compilation
is unnecessary.  Its modularity enables you to write small segments of
code in the form of functions, checking each one as it is written.
LISP doesn't require declarations, although good programming practice
suggests they be included in the final product.  This enables you to
develop functions quickly for experimental use.  LISP dynamically
allocates whatever kind of data structure you want to use.  That means
when you call in a function, LISP will make it immediately available
to you.  This means LISP handles all storage allocation for you,
allocating it when needed, cleaning it up when you are finished with
it.
Because there are no differences in LISP between programs and data
structures, LISP can be represented as lists just as if it were
another data structure.  Therefore, it is easy for LISP to reason
about or deal with its own functions just as humans examine their own
procedures.  As, a consequence, LISP provides sophisticated tracing
and debugging capabilities.
PROLOG
Prolog is based on a general-purpose pattern matching and inference or
theorem-proving mechanism called unification resolution. It is based
on a formalism symbolic logic called first order predicate calculus.
Basic Components:
1.   Read and Print
2.   Procedure evaluation based on unification-resolution, implemented
     as backtracking search.
3.   A data base which contains the definitions of procedures and the
     facts needed by the program.
                              Display 29.
PROLOG is a sort of second-class language for artificial intelligence
research in the United States, but it is gaining adherents (see
Display 29) It has a big following in England and in Europe.  PROLOG
stands for PRogramming in LOGic.  The idea is that PROLOG is a
language based on a subset of first-order predicate calculus called
Horn clauses.  It makes use of inference procedures used in proving
the correctness of logical
                                 -76-
statements.  It allows you to write computer programs simply by making
statements about what you want to be true in a clause form.
PROLOG provides many programming structures you would otherwise have
to build.  It can read program goals or procedures and print out the
results for you.  It's interpretive and it gives you all of the
facilities of allocation of storage and reformation of unused storage
in a manner similar to the support environment in LISP.
You do not need LISP or PROLOG to build an expert system--but it sure
helps.  The two speakers who are immediately following me are going to
talk about expert systems they built in FORTRAN.
What's important is to identify and to make use of the distinguishing
features of expert systems: the abstractions and program structure.
How do we use expert systems in statistics? This is a review of
suggestions discussed at the conference on Expert Systems and
Statistics that I attended last week.
First of all, I don't know how many of you are statisticians; but if
you are, your knowledge, as distinct from the knowledge of laypeople
who come to you for consultation or as distinct from pure subject-
matter knowledge (for example, economics) consists of a collection of
strategies for working with a set of data.  And those strategies could
probably be readily represented in the form of an expert system which
knows what test to do next.  Parts of the data need to be cleaned up.
One of the most active areas I of research that I have found is the
specification of individual statistical strategies in the form of
expert systems, either as interfaces to existing statistical acronyms
like S or SPSS or SAS, or in the form of a complete statistical
system.  Nobody wants to suggest that you can improve upon the
capabilities of these statistical packages.  What you may want to
suggest is that it may be possible to improve their usability by
adding something between the user and the package.
Another area of research concerns reasoning with uncertainty.
Statisticians have something to say about that.  I mentioned the
adding of certainty factors to knowledge bases in most expert systems.
An active area of research is to determine how those certainty factors
should be propagated through a rule system and how certain conclusions
can be based on uncertain knowledge.
Existing statistical software deals only with statistical ideas at the
lowest level.  It provides code to do things like least-squares
fitting and so on.  We want to use the ideas of expert systems to move
software up one more level to deal with the abstract ideas of
statistical strategy--the choosing of statistical methods and
selective analysis of data in light of these methods.  Our success in
this area depends on the development and use of modern programming
languages and on the development and use of expert system models.
                                 -77-
        AN EXTENSION OF STATISTICAL SOFTWARE TO EXPERT SYSTEMS
            James J. Filliben, National Bureau of Standards
The outline for this talk falls into four general areas.  We are going
to be talking about the real relation of an expert system to a
particular piece of software, namely, DATAPLOT.  I am going to speak
first about DATAPLOT to show how the expert system can be described
with respect to, DATAPLOT.  We are going to be talking about the
general structure for the intelligent subsystem -- expert subsystem --
and DATAPLOT and go into the interpretation of conclusions mode and
the analysis guideline mode.  The last mode deals with providing a
guide for carrying out data analysis.  We will go through a particular
data problem.
DATAPLOT is a high-level interactive statistical system with its own
language, a high-level language with English-like commands.  It was
designed at the National Bureau of Standards (NBS) in 1977.  The
National Technical Information Service (NTIS), has been distributing
it for the last three years.  The software is written in FORTRAN.  The
cost is $1200.  It is the most heavily distributed software of its
type at NTIS. , It has been installed at about 200 sites.  Next, year
it, will be the most heavily distributed piece of software, period.
Its primary capability is graphics.
That means it can run on Tectronix, HP and various other graphics
terminal devices and on a variety of mainframes.  It has both analysis
graphics and presentation graphics.  There are extensive additional
capabilities in graphical data analysis and nongraphical data
analysis, modeling and fitting, mathematics and diagram graphics.
At the National Bureau of Standards (NBS) we are interested in
modeling and fitting data  In particular, we are often interested in
fitting nonlinear models.  Moreover, we make extensive use of applied
mathematics and diagram graphics.  By diagram graphics I mean the
construction of schematic diagrams.  The NBS is an engineering and
scientific research organization.  We have people that like to make
schematics.  We spend our time making schematic diagrams and charts.
This component of DATAPLOT supports the automation of that work.
There is a heavy emphasis on data fitting in order to test underlying
assumptions.  The graphical displays are important because they
provide insight into the underlying structure.  Insight is important
if you must go into court, for example, and defend your understanding
of mechanisms at work in the data you have analyzed.  Three notable
cases that have arisen in our area are the analysis of the draft
lottery, the argument over the use of daylight-saving time a couple of
years ago, and data concerning the Alaska Pipeline.  Graphical
analysis was a critical component in those projects.
Display 30 shows the structure for DATAPLOT.  It is a data analysis
activity on one side and a mathematics activity on the other side.
Three common activities common to both are plotting, fitting and
various transformations and function evaluations.
                                 -78-
[GRAPHIC] \WP1479.GIF
Display 31 shows the typical commands you can issue to DATAPLOT.  They
support plotting (commands 1, 2 and 4), fitting (commands 3 and,5),
and function evaluation (command 6).
                           TYPICAL COMMANDS
1.   PLOT X Y
2.   PLOT EXP(-X**2) FOR X = -3 .1 3
3.   FIT Y = A+B*EXP(-ALPHA*X)
4.   BOX PLOT Y X
5.   ANOVA Y Xl X2 X3
6.   LET A = ROOTS SIN(X**2)+EXP(-X) FOR X = 0 TO 5
IL
                              Display 31.
Displays 32 and 33 give examples of the display capabilities of
DATAPLOT.
All the graphics shown can be generated with any sort of system.
Whether you have TECTRONIX, Spot 10, IGL or any graphics terminal, the
important question is how long does it take to generate the graphical
display.  If it takes more than thirty seconds to a minute to do so,
we lose the continuity so important to human-machine interaction.  In
data analysis the only concern is finding underlying structure, and
getting insight.  When generating graphics gets in the way of the
objective of finding underlying structure in data, we lose control of
the analysis.  Thus, the utility of graphics software is measured not
in what it can do per se, but rather in how easy it is to do it -- how
easy it is to,understand, write, modify and communicate the
instructions.
The DATAPLOT Intelligence Subsystem is an augmentation of the current
system to provide information and guidance as if a statistical
consultant were present during an analysis.  Basically we want to
provide an expert subsystem that asks the right questions as we step
through an analysis.  In order to get insight -- more than answers --
asking the right questions is just as important as coming to the right
conclusions.  The expert system interacts with the analyst, setting
the pace and posing questions along the way: have you checked this,
have you checked that, what does such and such a plot look like? It
will look like such and such so perhaps you should go in this
direction, that direction, etc.
Display 34 shows some of the human-machine interaction problems that
must be addressed in an expert system.  If the user requests an
operation like BOXPLOT, he should be able to see a one-line definition
and the rationale for its use.  In other words, if the expert system
recommends a certain course of action,  the analyst should be able to
ask questions like, "What is the penalty if we don't follow this?.
                                 -80-
[GRAPHIC] \WP1481.GIF
[GRAPHIC] \WP1482.GIF
                     HUMAN PROBLEMS (DESIGN GOALS)
DEFINITIONS
RATIONALE
LINKING TOOLS
KEY TESTS
HYPOTHESES/CONCLUSIONS
VARYING EXPERIENCE
                              Display 34.
The expert system should support the linking together of statistical
analysis tools, often in unexpected ways.  Data analysis is primarily
sequential and interactive.  We step through the data, step through
the analysis; and at each step, the next step is dictated by what we
have seen before that.
Scientist's often deal with correlation plots to see if there is any
correlation structure in the data.  DATAPLOT supports a correlation-
plot command and many other graphic commands, and analysis.  However,
if someone asks for a correlation plot, the expert system should
assist the analytic effort by carrying out appropriate statistical
tests behind the scenes.
Another important but time-consuming aspect of data analysis
(especially when you are writing research papers for the general
science community) is the need to frame your hypothesis and
conclusions in proper statistical terms.  An expert system should
support this formulation. We have found that to be very helpful to the
average scientist and engineer.  Every paper that goes out of NBS goes
through our statistics review process to guarantee that hypotheses and
conclusions have been properly stated in statistical terms.
The last aspect is the varying experience.  Any expert system is going
to have a problem dealing with different kinds of methods.  No one
expert system is going to be ideal because users have various degrees
of experience.  That's a very sticky problem.  A tough problem.  You
don't want the expert system to be so simple minded that an
experienced analyst must go through 20 menus just to carry out a legal
analysis.  On the other hand, someone with limited experience needs
the extensive guidance that 20 menus would give.
Display 35 shows the general content of the expert system component of
DATAPLOT.
                                 -83-
SUBSYSTEM OUTPUT (THE EXPERT SYSTEM)
SEQUENCE OF MENUS
DATAPLOT COMMANDS
CAUTIONS/CONCERNS
MENU EXPLANATIONS
ADDITIONAL TESTS
RIGOROUS STATISTICAL CONCLUSIONS
                              Display 35.
Sequence of menus: Each menu should have guidelines at the bottom of
the menu explaining not only the current menu but offering suggestions
as to which menus to, select next for specific analyses and why.
These suggestions can include specific DATAPLOT commands.  Moreover,
within the menu environment the cautions and concerns about the form
of the analysis should be displayed clearly (e.g., a caution about the
data not following a Normal distribution).
The user should always have access to HELP functions for each menu
These menu explanations should include a description of where the
particular menu fits within the entire collection of menus.
Additional tests: I mentioned the idea of performing statistical
computations behind the scene.  Although the analyst may be unaware of
their specifics, he may want to make use of their results at a later
time.  The expert system is aware of this and can provide them.
DATAPLOT thus has 2 expert subsystems: A consultant-style expert
system which offers expert guidance for thoroughly and rigorously
carrying out a data analysis; and a data-interpretive expert system
which chooses a test, applies the test to the data, interprets the
output, and formulates a rigorous statistical conclusion (couched in
proper statistical terminology).
The remaining displays provide some idea of the analyst's interaction
with the expert system component of DATAPLOT.
As you can see the need for a great variety of interactions in a large
expert system requires a lot of thought and a large comprehensive
software system.  If any of you want to see DATAPLOT in operation, we
are out in Gaithersburg, and we will be glad to come out and
demonstrate it locally.
                                 -84-
References
Filliben, J. J. and Fong, J. T. (1984), "DATAPLOT as an Expert System
for Data Analysis," available from American Society of Mechanical
Engineers, June, 1984.
Hahn, Gerald J. (1985), "More intelligent Statistical Software and
Statistical Expert Systems: Future Directions," The American
Statistician February, 1985.
                        EDITING AND IMPUTATION
                 Brian Greenberg, Bureau of the Census
In talking today about an application of expert-system methods to data
editing and imputation, it will be the first time that I use the words
"expert system" in describing the edit and imputation program we have
developed -- SPEER (Structured Program for Economic Editing and
Referrals).  In the past, the focus was more on describing the
underlying methodology and discussing what the edit and imputation
system could do for users.
While preparing notes for this talk, I found that the emphasis was
less on SPEER itself and more on editing and imputation as an expert
system in principle.  When work started on our project to develop edit
and imputation software we had no intention of building an expert
system.  The goal was to develop techniques that corrected - survey
and census response data and imputed for missing values.  Looking
back, one can see that as work on, this project proceeded, an expert
system was evolving; and in the talk I will describe some of the steps
in the development of this system.
The purpose of editing and imputation is two-fold.  First, if a
respondent form is received and some responses are blank (item non-
response), one tries to fill in missing values in order to create a
complete data record for tabulation purposes.  In addition, one wants
to detect-erroneous responses and correct them.  For example, a
response may indicate a fifty-acre farm with five million bushels of
wheat or a twelve-year-old grandfather.  Such problems do occur in the
response data; they can be data entry errors or errors at the source.
Which field does one adjust and what value should replace those
selected for change?
When confronted with large data sets such as one has in the Census
Bureau and many other Federal agencies, an automated system is a
necessity.  For surveys dealing with similar types of data, one would
like to have general programs to avoid continually having to reinvent
the wheel.  On the other hand, it is desirable that an edit and
imputation program incorporate has much survey-specific information as
is available, and one would like the survey-specific information, to
be exercised through a family of rules. An addition, one usually would
like a mathematical model to ensure that rules are, applied
consistently and to assist in selecting from among rules. In
particular, one wants to blend survey-specific information
mathematical procedures within a coherent framework.  The expert-
system model is a natural structure for this type of program.
                                 -85-
              FUNCTIONS IN AN EDIT AND IMPUTATION SYSTEM
                             EDIT CHECKING
                          ERROR LOCALIZATION
                              IMPUTATION
                              Display 36.
What are the functions in an editing and imputation system? (See
Display 36.) The first is edit checking.  Edits are rules that detect
prohibited response combinations; and it is easy to check when an edit
fails, that is, a prohibited combination is encountered.  Given an
edit failing record, one endeavors to change as few responses as
possible in order to make the remaining responses consistent.
Determining fields to change is called error localization.  Finally,
one wants to impute in order to allocate values for non-response and
replace responses deleted during the error localization process.
          DESIGNING AN EXPERT SYSTEM FOR EDIT AND IMPUTATION
      UNDERSTANDING THEORETICAL ASPECTS OF EDITING AND IMPUTATION
UNDERSTANDING FACETS AND NATURE OF SUBJECT-MATTER EXPERTISE
                              Display 37.
In designing edit and imputation software along the lines of an expert
system, each function that was described above should be structured in
its own module (see Display 37).  In a general system one wants to
enter as parameters the information that will be requested of all
users.  Survey-specific information, particularly decision rules, can
be entered in specified, well-defined places throughout the program.
These rules will be different for each user.  SPEER had been employed
on six segments of the 1982 Economic Censuses.  The edit-checking
routine never changed from user to user, and the area-localization
subroutine was always the same.  The imputation rules, however, varied
in each application.  How does one impute? In general, one must rely
on those with expertise about the particular survey vehicle.  One
works with. the subject-matter specialists to elicit well-defined
decision rules based on their knowledge and experience.
What does one have to do in designing edit and imputation programs
along the lines of an expert system? First, one must understand the
nature and the facets of the subject-matter expertise;  What do the
experts know?  Their experience concerning the survey vehicle is
extensive; it is often based on experience in the analysis of response
forms and familiarity with respondents.  They are knowledgeable about
the survey target population, the survey form itself, and often the
source of errors or non-response in data.
                                 -86-
As a matter of fact, for some kinds of missing data, the survey
specialist can tell you why it's missing.  For example, it may be
known by people working on a survey that when a certain field is
blank, the respondent means zero -- they just routinely skip the
question.  Other blank fields will never be zero.  The respondent
either did not know the answer to that question or did not want to
reveal it; and so that data field was left blank.  Knowledge of this
sort is certainly survey-specific.  It cannot be gleaned through
standard analysis of reported data, nor are there usually auxiliary
data sets available to design models of "missingness." The subject-
specialist, however, is a source of information that can be profitably
utilized.  Statistically-derived procedures (such as appropriate
model-based imputation techniques) can be viewed and utilized as
survey specific decision rules.
In addition to subject-matter expertise, one must incorporate
appropriate editing models.  In SPEER, the error-localization process
is basically a set-covering problem -- a mathematical model.  One
utilizes linear analysis and graph theory to select fields to delete
on edit-failing records.  Once these fields are deleted, the remaining
fields will be mutually consistent; and then one can begin to impute.
The process of imputation uses survey-specific rules provided by
subject-matter experts.  The knowledge base of decision rules can be
organized within coherent imputation modules through which they can be
applied.  That is, the system goes back and forth between the subject-
matter information land the mathematical model.  Mathematical
techniques and subject-based imputation rules are two components that
one should have in an overall edit and imputation system.
thinking of it that way, the mathematical procedure and the subject-
matter rules can be treated as separate.  One can extend the
mathematical methods and revise the flow of the system as a whole,
unencumbered by survey-specific considerations.  The survey-specific
rules can be examined in their own right, updated and revised as
needed independent from the programs through which they are applied.
On the other hand, the mathematical procedures and decision rules are
integrated.  The mathematical constructions provide a framework to
assist in choosing the most appropriate decision rule and to ensure
that the value imputed will pass all applicable edits.  Thus, an
expert system for imputation should do more than provide a vehicle for
accessing expert rules.  It should also provide a mathematical
framework to help decide from among the rules, choosing only rules
which are valid for the record under consideration.
                                 SPEER
             CONTINUOUS (ECONOMIC) DATA UNDER RATIO EDITS
                           (A(l), ..., A(N))
                          TYPICAL RATIO EDIT
                       L(ij)< A(i)/A(j).< U(ij)
                              Display 38.
                                 -87-
SPEER (Display 38) is an edit and imputation system designed along the
lines of an expert system.  SPEER was designed for economic data such
as wages, assets, inventories, etc.  The typical edit is a ratio of
two fields, called a ratio edit.  The total salaries paid to employees
divided by the total number of employees should be within some
reasonable range consistent with our knowledge of the industry and
occupation.  The amount of crop yield divided by the number of acres
should be in a certain range.  Ratio range checks are a very common
edit in economic surveys.  Given that a family of ratio edits is
failed by a response record, one must select a set of fields to
delete.  We illustrate the workings of the error localization
mechanism of SPEER on two samples below.
Let the circled numbers in Display 39 represent response fields and
the edges in the graph represent edit failures between the adjoining
fields.  For example, the value in field 6 is inconsistent with fields
1 through 5 as determined by the collection of edit rules.  If we
delete the value in field 6 -- that is, remove node 6 from our graph -
- all edges vanish.  Thus, the remaining fields are mutually
consistent because there are no edges connecting the corresponding
node, hence there are no edit failures between them.  That is, we can
delete a single field to eliminate all edit failures.  The lower graph
is a little more complicated. one can see, however, that by deleting
the values in fields 2 and 3, the remaining fields are mutually
consistent.  These simple examples capture the spirit of the error
localization methods built into SPEER -- a little graph theory is used
to find the minimal subset of field values to delete.
After error localization one has a collection of blank fields (some
due to non-response, others because fields were deleted during the
error localization process).  The remaining fields are consistent with
one another, and they must be consistent with values imputed.  The
program sets up a series of range specifications for a blank field
taking into account the value for each valid field.
                                 -88-
[GRAPHIC] \WP1489.GIF
       If A(n) is missing, and A(j) are consistent for all j n,
                   L (nj) fa A (n) /A (j) -,C U (nj)
                                  so
                   L (nj) A (j)  A (n)  (nj) A (j)
                              Display 40.
Every valid imputation for a missing field (field n in Display 40)
must lie in the overlap of the regions determined by each fixed field
on the record in order to be consistent with every other field.
Once the feasible region for a missing field,is computed the program
reaches into the imputation module for the value to be imputed.  The
first applicable rule is selected and an imputation is derived based
on this rule. if the derived value falls in the feasible region, it is
Accepted as a valid imputation.  If not, the second rule is accessed,
an imputation value is derived, etc.  The value ultimately selected as
the imputation will be chosen based on subject-matter based rules and
will also be consistent with all other fields on the record under
review, because it is forced to lie in the feasible region.
This may be a good time to provide an example of what a rule sequence
might look like.  Suppose one is to impute for a field such as Annual
Payroll (APR) on an economic census or survey.  For concreteness, let
us couch our discussion in terms of the 1982 Economic Censuses.  The
first rule might be to derive an imputation based on the 1982
Administrative Data value for APR.  If the value derived does not lie
in the feasible region, one might try the 1981 Administrative Data
value for APR.  If this value is not suitable, we pass to a third
option, etc., until a valid imputation is derived.  Some imputation
rules can be extremely field specific.  For example,.suppose some
field is to be reported in tons.  Assume that, the feasible region
allows valid responses to be between 500 and 1,000 tons and the value
1,800,000 was reported and deleted as an error.  The applicable option
might be to divide the reported value by 2,000 (subject-based
information that respondents
sometimes report in pounds rather than tons).  In this example, we
would derive 900 tons and observing that this value is feasible,
accept it as the valid imputation. A common error in reporting
economic data is that respondents provide answers in units rather than
in thousands as per instructions.  For fields in which this error may
occur, the first rule (when appropriate) is to divide the reported
response by 1,000.
The editing and imputation for the 1982 Enterprise Summary Report and
the 1982 Auxiliary Establishment Report (both portions of the 1982
Economic Censuses) was performed using SPEER.  In addition, SPEER was
used-to process the Manufacturing, Retail, Wholesale, and, Service
segments of the 1982 Economic Censuses of Puerto Rico.  In each of
these applications, the edit checking, error localization routines,
and basic system flow are the same.  Each application, however, had
its own family of decision rules for imputation.  Each application
employed different rules based on the survey-specific fields, relation
between fields, and auxiliary information.
                                 -90-
How does one implement an edit and imputation system based on expert-
system principles?  For a given application, start with the experts.
their expertise, elicits rules, and embeds those rules in the system
components requiring them.  Sample data is tested, performance
evaluated, rules are refined as needed.
Editing of economic data records at the Census Bureau is a two-phase
process.  All records are run through an automated edit and imputation
system in batch mode. Within the automated routines, selected records
are targeted as referral cases and are directed for analyst review.
An optimal strategy will include automated procedures to resolve the
majority of cases and individual review for establishments needing
special handling.  Typical referral criteria are: (1) large change to
reported data; (2) amputations for large establishments; and (3)
highly atypical combination of responses.  The analyst reviewing a
response form is a subject-matter specialist, and the review is
currently a pencil-and-paper process.  After analyst adjustments are
made to the results of automated processing on an establishment
record, the revised form is once again processed through the automated
system.
SPEER allows on-line, interactive processing of referral cases.  Used.
in this mode, the system converses with an expert using it.  The human
expert can override the decision rules residing in the system and
replace them based on his/her expertise, and auxiliary information
about the case under review.  Using this system, the analyst requests
a specific record and reviews the processing done by the automated
system.  The analyst has the original response form and hence access
to information not incorporated into the rules.  Based on this
additional information and his/her own experience, an analyst may
overrule the decision rules built into the automated system.
                      IMPUTATION OPTIONS FOR APR
A.   RANGE OF APR: (250,750)
B.   CURRENT VALUE: 375 OPTIONS
     1.   REPORTED VALUE: 82
     2.   1982 ADMINISTRATIVE DATA:
     3.   1981 ADMINISTRATIVE DATA BASED:
     4.   1977 CENSUS DATA BASED:
     5.   IMPUTATION AND TOLERANCE:
     6.   ANALYST SUPPLIED VALUE:
                              Display 41.
The display seen by the analyst looks something as in Display 41.
Using Annual Payroll (APR), this display shows an acceptable range for
APR from 250 to 750 (i.e., the feasible region).  The current value is
375, which was derived by the automated system.  The next value is the
actual reported value of 82 followed by the reported 1982
Administrative Data and other candidate amputations based on 1981
Administrative Data, 1977 Economic Census Data, etc.  The ordering
above reflects the order in which the rule options are applied.  By
requiring that the range in which the imputed value must fall be
consistent with all fields, plus a variety of options, the
                                 -91-
analyst then has a significant amount of information at his/her
disposal to assist in the decision-making process.  If there is reason
to believe that the most appropriate imputation value lies outside the
feasible region (for example, because of explanatory notes on the form
or through a call-back to the respondent), the analyst can select an
imputed value outside the feasible region.
A revised imputation for field APR is decided, and the analyst enters
it into the data record.  This value is accepted by the program, and
field APR is, considered to be completed.  Suppose there is a second
field to be reviewed on this record (for example, Number of Employees
(EMP)).  Once again, the program displays on the terminal screen the
feasible region for EMP, currently residing value, and candidate
values for imputes derived according to each option, as it did for
APR.  Note, however, that each of these values is based, in part, on
the new value of APR just entered by the analyst.  As above, the
analyst will determine an appropriate value for EMP, enter this value,
and move on to the next field, if any.  After all fields have been
examined and adjusted if needed, the review is complete.  The revised
record will be consistent, and no further batch processing will. be
required.
The important observation from the perspective of an expert system, is
that a true expert, converses with the automated expert programs in
order to augment the system expertise and override decision rules as
needed.  Initial testing has shown that analysts have found this
system easy to use.  It has the potential for making their decisions
in the review of establishment records less tenuous than is currently
the case.  Because the individual review of establishment records is a
time-consuming and costly process, one can anticipate savings of time
and money in the use of an "expert-system aided," on-line, interactive
review process.  The on-line, interactive portion of this program has
not yet been put to use for actual survey processing.  We are actively
working with potential users to incorporate this aspect of the program
in future editing and imputation processing.
In summary, an edit and imputation system should blend statistical and
subject-matter expertise in a coherent framework and integrate edit
constraints with imputation strategy.  We have described a structured
system that attempts to meet these requirements and is sufficiently
flexible to accommodate a variety of users.  Development work
continues on this system, enhancements are being made, and additional
users are being identified.  The references provide more information
about some of the technical features of the SPEER system.
                              References
Greenberg, Brian (1981), "Developing an Edit System for Industry
Statistics," Computer Science and Statistics: Proceedings of the 13th
Symposium on the Interface, 11-16, Springer-Verlag, New York.
__________     (1982), "Using an Editing System to Develop Editing
Specifications, " Proceedings of the Section on Survey Research
Methods, American Statistical Association, 366-371.
     and Surdi, Rita (1984), "A Flexible and Interactive Edit and
Imputation System for Ratio Edits," Proceedings of the Section on
Survey Research Methods, American Statistical Association, 421-426.
                                 -92-
                              DISCUSSION
              Mark Winer, Office of Management and Budget
When Terry first mentioned the idea of having a panel on expert
systems in a conference on Statistical Uses of Microcomputers in
Federal Agencies, my question was "What do expert systems have to do
with microcomputers and statistical systems?."  I decided I would take
a look at all the things I saw today to see how this session fits into
the other sessions.  The first thing we see is that you can use the
IBM PC, or other personal computers, as a terminal to a mainframe.
The mainframes have excellent software systems.  That allows you to
use both machines for the things those machines are best at. With the
large amounts of data and large amounts of information you might need
with an expert system, it is good to use a mainframe for most cases;
but for the kind of processing and quick response you might want it's
nice to use downloaded results from an expert system on a personal
computer.
The second reason this fit in is that, as I mentioned, the system
developed by Brian Greenberg has just been adapted to personal
computers.  As memory capacity and storage capacity on personal
computers increase, even the large systems like DATAPLOT could be
extended to personal computers.
The third reason that this fit in is that every couple of years there
is a real hot topic in the computer field.  In 1982 and 1983 it was
decision support systems.  If we were having this workshop in 1982 or
1983, we would have undoubtedly had a panel on decision support
systems; and since in 1984 and 1985 the hot topic is expert systems,
we are having this workshop and it is incumbent upon you to have a
panel on expert systems.  This brings up the obvious question of
whether an expert system really is something new or is it just
something old, another big word that people use to bring out high-
priced consultants to design your system.
I guess I will say that from what we have seen in these demonstrations
today, expert systems are doing more than ordinary software systems.
Ordinary software systems help the user do the things he normally does
but make it possible for him to do those things faster and save him
some of the tedious parts of the task.  Both the systems we heard
talked about today have the advantage of actually bringing additional
knowledge to the user in that he can do what he wouldn't necessarily
know how to.  Expert systems show you how to locate the error that is
the easier error to change if you are trying to do efficient editing.
You have subject-matter expertise that analysts couldn't produce on
their own.  This system does that for you. It uses the subject matter
of the expert to figure out how that record can be changed with some
help from the system.
The DATAPLOT system teaches you about the tests that are available to
you as you use them.  It suggests to you additional steps to do as you
are doing things; so even if you are not an expert statistician, you
can figure out the ways to proceed as you are, working on a problem.
So, as I say, expert systems provide something beyond what we
ordinarily have in software systems.  They are an extension of the
existing packages rather than things that stand by themselves.
                                 -93-
                         QUESTIONS AND ANSWERS
Ql:  You mentioned a new fad.  Isn't some of this just like a
sophisticated help facility?
Al:  (Mr.  Winer): Yes.
Q2:  This is this years new thing, but better help facilities have
been a growing need since computers got started.  I think expert syst-
ems are a logical outgrowth of that.
A2 (Mr.  Lawton): I think expert systems are more than that, but help
facilities are at least part of what we are dealing with here.  I
would say what makes the help facility more sophisticated is that they
have some expertise built into them, that is based on knowledge of
where in the program the user calls the help facility.  So what you
say is partly true, but I think there is some intelligence built into
the back of the help facility in the expert system that wouldn't be
there in a more conventional system or from just reading, the help
file.
A2 (Mr.  Ireland): There probably are two other issues.  First, the
help system can be changed incrementally as you come to understand
what kind of help you need.  The idea of rules makes it easy to
develop small help modules that are added to a system that already has
a help facility.  Second, for some of Brian's things, it isn't a user
help facility, but a specification of how to handle a particular piece
of data.  So, the help facility might never be seen by analysts unless
they ask to see it, but it would be used to make a proper kind of
modification to the data.
A2 (Mr.  Greenberg): Expert systems can be run in batch mode once ex-
pertise is built into it, and that bears on the use of the help
facility in a batch mode.
Q3:  I am curious about some of the details of the DATAPLOT and the
editing and imputation system.  Let me start with the editing system
since that is fresh in your memory.  How much memory on the IBM XT
does your system take up?
A3 (Mr.  Greenberg): I really don't know.  I haven't been doing the
actual transfer to the XT.
Q4: But you could fit it into 64OK?
A4 (Mr.  Greenberg): Yes, plenty of data is on one floppy disk.
Q5:  You said it was easy to use.  What did you say--a half day? How
long?
A5 (Mr.  Greenberg): I would say a half day working with somebody like
myself or someone familiar with it.
                                 -94-
Q6:  We do surveys a lot and they are typically tedious -- we have
people coming in to do error checking and editing in a rather
primitive way, so I think your product would be very useful to us and
a lot of other people. What would be a good way for us to learn more
about it?
A6 (Mr.  Greenberg): Drop me a line or give me a call.
Q7:  The degree of statistical information you obviously have in your
head goes well beyond that of everyone in our office.  The only fear I
would have would be whether those of us who have a much lower level of
statistical knowledge could still make use of DATAPLOT. What do you
think about that?
A7 (Mr.  Filliben): That is a general problem, and one of the displays
dealt with varying experience.  This addresses the point of whether
this is an extended help facility.  We tried to make sure that the
menus that came up would be a part of the education process too--a
tutorial, if you will.  We have had people use this expert system who
have very limited statistical background, and they have come, out with
good results.  It's a matter of learning, and I think the expert
systems are at the point now where it's nice to have a machine that
has an expert system, but it's also nice to have some statisticians
and other consultants around who can augment them.  One thing we did
not mention was references.  Where does someone go if he really wants
to read up on graphical or residual analysis, for example? That is one
command as far as DATAPLOT is concerned.  There should be a reference
command.  It's not in there yet, but there is a body of literature
that's out there that has a lot of details.  If people want to go in
and fill up their own base knowledge, they should have access to this
base.  It is very much, as you say, an extended help.  There are lots
of different ways these various systems can be of help because there
are a lot of different ways we can have deficiencies in our own
knowledge.
Q8: What kind of mainframe are they working with?
A8 (Mr. Filliben): All the major mainframes.  UNIVACS.
one, and in fact the default machine, is the VAX 11/780.  IBM/PC's.
The Pentagon has it on a Honeywell Multics system.  PERKIN-ELMERS.
PRIMES.  The only machine we had difficulty putting it up on was the
CYBER machine.  That problem will disappear because we are getting a
CYBER machine and we will be forced to address that problem.  They
have a hardware restriction on memory.  In UNIVAC you run into an
overlay problem.  In terms of whether it would download to a PC or
could be put up on a PC, you would need about-a half a megabyte of
memory.  Small machines--micros--are expanding to the point where it's
a real possibility to put DATAPLOT on a PC.
Q9: You   say that NTIS sells this?
A9 (Mr.   Filliben): National Technical Information Service sells it
for
$1200 --a one-time-only fee.  You get the source code.  The source
code on the file is 12 megabytes, so you have to have somewhere where
you can put it.
Q10:  Did you write this yourself?
A10 (Mr.  Filliben): Yes.
                                 -95-
Q11: How long did it take?
All (Mr.  Filliben): We started back in 1972 with a software system
called DATAPAK which is free from NBS.  That sort of got us into the
problem.  By 1975 it became clear that interactive systems were
becoming more important.  By 1977 we had the first DATAPLOT running,
and things have essentially been the same since then.  We augmented it
to include the expert system.
Mr. Winer: Perhaps less a question than, a comment.  At the end of the
last session, I asked the panelists how their decentralized and
spreadsheet-type statistical systems insure or assure data integrity
and adherence to the statistical standards.  Here I think we have had
two presentations in which one could in a sense say "Hey, that answers
the question!".  If people start using systems like Brian's, they will
have more data integrity; and if one starts using systems more like
Jim.'s, one could have more adherence to agencies present standards.
I would like to take this opportunity to thank Terry Ireland who
chaired this session, but who is also the Chairman of the Subcommittee
of the Federal Committee on Statistical Methodology who organized this
entire workshop, including this session.  We thank him and thank you
all for coming.
                                 -96-
                               Appendix
                      Announcement of Workshop on
        Statistical Uses of Microcomputers in Federal Agencies
The Subcommittee on Statistical Uses of Microcomputers in Federal
Agencies of the Federal Committee on Statistical Methodology is
sponsoring a one-day workshop on April 24, 1985, to discuss with other
Federal employees selected topics on statistical uses of
microcomputers.  The workshop will be held at the IRS Auditorium, 1111
Constitution Avenue, N.W., 7th floor, from 9:15 a.m. to 4:30 p.m.
The agenda and speakers are as follows:
9:15 a.m.    WELCOME AND INTRODUCTION
Chair:    Maria Gonzalez, Office of Management and Budget
Arrangements: Linda Taylor, Internal Revenue Service
9:20 a.m.    PLANNING
Chair and Discussant: Larry Cox, Bureau of the Census
Rapporteur: Fred Cavanaugh,.Bureau of the Census
Microcomputer technology has much to offer statistics, and many
statisticians have become microcomputer users at work and at home.
This technology and the keen interest of statisticians in it provide
statistical agencies with many opportunities, each bringing with it
responsibilities for planning, implementation and evaluation: if every
statistician (programmer/secretary) in an agency wants a
microcomputer, who should have them? For what purposes can/should
microcomputers be used? In what configuration? At what cost
(overall/per user)? How will this technology coexist with central ADP
services? What policy decisions need to be made -when -- by whom?
In this session on planning, we will explore such questions through
discussion, focusing on three different and successful approaches to
these problems -- those adopted by the Census Bureau, the National
Security Agency and the Bureau of Labor Statistics.
Speakers: Ronald R. Swank, Bureau of the Census Kathy Schnaubelt,
National Security Agency Peter Stevens, Bureau of Labor Statistics
                              DISCUSSION
10:45 a.m.                   COFFEE BREAK
                                 -97-
11:00 a.m.             ELECTRONIC DATA DISSEMINATION
Chair:    Ken Berkman, Bureau of Economic Analysis
Rapporteur: Jay Casselberry, Energy Information Agency
This session is a panel discussion of the different approaches
to.electronic data dissemination by various Federal agencies.
Different approaches will be described with particular emphasis on the
factors determining an agency's approach to data dissemination and the
problems encountered in their implementation.  The experience gained
by these agencies will be presented: National Technical Information
Service (NTIS) distribution of microcomputer floppy disks; Census
CENDATA system; and the U. S. Department of Agriculture's current
development of an on-line system.
Speakers: Stuart Weisman, National Technical Information Service
Barbara Aldrich, Bureau of the Census
Roxanne Williams, Department of Agriculture
                           ***DISCUSSION***
12:30 p.m.                  LUNCH
1:15 P.M.       APPLICATIONS OF MICROCOMPUTERS
Chair: Ron Steele,  Department of Agriculture
Rapporteur: Tom Nagle, Internal Revenue Service
This session is a panel discussion of statistical applications of
microcomputers.  The utility and weaknesses of applications software
and operating systems will be discussed.  Some examples involve
interfacing mainframe and microcomputers.  Issues to be addressed
include an assessment of the utility of microcomputers at present, the
future utility in light of new hardware and software technologies, and
considerations regarding data integrity, security and accessibility.
Speakers:      Linda Atkinson, Department of Agriculture
               Gary Nelson, Department of Agriculture
               Rick Hayes, Internal Revenue Service
               Brian Carney, Department of Agriculture
               Paul Dobbins, Department of the Treasury
               Dick Shively, Department of Agriculture.
                                 -98-
                              DISCUSSION
2:45 p.m.                    COFFEE BREAK
3:00 p.m.       EXPERT SYSTEMS
Chair:    Terry Ireland, National Security Agency
Discussant:    Mark Winer, Office of Management and Budget
Rapporteur:    Norman Glick, National Security Agency
Recently, the idea of incorporating techniques used by professional
experts into software has become popular.  This session will introduce
the basis for expert-system methodology and give several practical
examples of expert systems with statistical applications that are
currently in use.
Speakers: George Lawton, Army Research Institute
          James Fillben, National Bureau of Standards
          Brian Greenberg, Bureau of the Census
                          ****DISCUSSION****
4:30 p.m.                   ****ADJOURN****
                                 -99-
     Reports Available in the Statistical Policy working Paper series
1.   Report on Statistics for Allocation of Funds (Available through
     NTIS Document Sales, PB86-211521/AS)
2.   Report on Statistical Disclosure and Disclosure-Avoidance
     Techniques (Available through NTIS Document Sales, PB86211539/AS)
3.   An Error Profile: Employment as Measured by the Current
     Population survey (Available through NTIS Document Sales PB86-
     214269/AS)
4.   Glossary of Nonsampling Error Terms: An Illustration of a
     Semantic Problem in Statistics (Available through NTIS Document
     Sales, PB86-211547/AS)
5.   Report on Exact and Statistical Matching Techniques (Available
     through NTIS Document Sales, PB86-215829/AS)
6.   Report on Statistical Uses of Administrative Records (Available
     through NTIS Document Sales, PB86-214285/AS)
7.   An Interagency Review of Time-series Revision Policies (Available
     through NTIS Document Sales, PB86-232451/AS)
S.   Statistical Interagency Agreements (Available through NTIS
     Document Sales, PB86-230570/AS)
9.   Contracting for Surveys (Available through NTIS Document Sales,
     PB83-233148)
10.  Approaches to Developing Questionnaires (Available through NTIS
     Document Sales, PB84-105055/AS)
11.  A Review of Industry Coding Systems (Available through NTIS
     Document Sales, PB84-135276)
12.  The Role of Telephone Data Collection in Federal Statistics
     (Available through NTIS Document Sales, PB85-105971)
13.  Federal Longitudinal Surveys (Available through NTIS Document
     Sales, PB86-139730)
14.  Workshop on Statistical Uses of Microcomputers in Federal
     Agencies (Available through NTIS Document Sales, B87-166393)
Copies of these working papers may be ordered from NTIS Document
Sales, 5285 Port Royal Road, Springfield, VA 22161 (703) 487-4650
