**American Statistical
Association**![](media/image1.png){width="0.8548611111111111in"
height="0.9840277777777777in"}

1429 Duke Street, Alexandria, Virginia 22314-3415 USA

\(703\) 684-1221 Fax: (703) 684-2037 Email: asainfo@amstat.org

Web Sit: http://www.amstat.org/

September 17, 1999

National Science and Technology Council

Committee on Technology

Old Executive Office Building, Room423

Washington, D.C. 20502

VIA E-Mail and Federal Express

Dear Committee Member:

The American Statistical Association (ASA) is one of the oldest
professional societies in the United States and with nearly 18,000
members is the largest association of professionals dealing with data
collection and data analysis. We are delighted to submit the enclosed
issue paper \"Policy for Efficient Development and Management of Data
Resources to Support U.S. Technical Innovation and Competitiveness\", to
the Committee on Technology (CT) of the National Science and Technology
Council (NSTC) in support of an action plan for Federal policy and
regulatory reform that will enhance innovation.

This submission is an indication of our strong interest in the process
you are undertaking, our conviction that statistical science has an
important role to play in this process, and our continued interest in
providing further support to your committee.

Sincerely,

Jonas H. Ellenberg, Ph.D.

1999 President

Enclosure: Microsoft Word Document

PRIORITIES FOR FEDERAL INNOVATION REFORM

Policy for Efficient Development and Management of Data Resources to
Support U.S. Technical Innovation and Competitiveness

Submitting Organization:

American Statistical Association

Contact:

American Statistical Association

Ray A. Waller, Executive Director

1429 Duke Street

Alexandria, Virginia 22314-3415

703 - 684 - 1221 (Voice)

703 - 684- 2037 (Fax)

RAY@AMSTAT.org

Policy for Efficient Development and Management of Data Resources to
Support U.S. Technical Innovation and Competitiveness

######## American Statistical Association

**Introduction**

This paper explores three themes affecting Federal policies for
efficient development and management of scientific and technical data.
One is the increasing importance of data economics. Demand for high
quality scientific and technical data is rising faster than its supply.
This poses multi-dimensional policy problems, not only for the Federal
Government, but for technological innovation in general. Another theme
is the need to emphasize data utilization technologies over
inappropriate temptations to hoard or excessively protect data
resources. The third theme is the promotion of technological efficiency
and productivity in part through better and wider use of statistics and
statistical ethics.

Increasing internationalization of information will inevitably increase
global competitiveness. It would be highly counterproductive to react to
this trend by trying to restrict all U. S. scientific and technical data
in a protectionist mode. As valuable as data may become, the greater
value will always lie in the ability to absorb, digest, and utilize
data. The U. S. benefits from inexpensive or no-cost international data
sharing and allows other countries to do so in a reciprocal manner. We
should properly promote data accessibility, usability, and efficiency,
balanced with due recognition of proprietary and intellectual property
rights. (1)

The third topic of this paper is improving technological data efficiency
through better use of available and developing technology. The focus
here will be on statistics. Other contributing technologies, which are
more commonly recognized as crucial to technical data management,
involve computer science (including computer simulation),
bioinformatics, and scientific and engineering analyses generally.
Statistical ethics in particular and scientific ethics in general are
also highly relevant. For example, one way to avoid wasting scarce
scientific and technical resources on fruitless research is to evaluate
correctly that the data sample and the experimental or observational
design of a planned study will be capable of providing a satisfactory
answer to the research question(s) posed. This is especially important
where human or animal subjects are to be placed at risk in the study. It
is unethical to place subjects at risk where one knows, or should know
that there is little potential for an increase in knowledge. Federal
policy should encourage wider use of statistical methods where they
promise to improve research efficiency and reduce data costs. It should
also discourage misuse of statistical methods and basic scientific
principles.

**Policy implications of evolving data economics**

Evidence of the economic shifts occurring in technological data
management include:

Increasing costs and commercialization of scientific and technical
journals,

The increasing proportion of the research enterprise underwritten by
commercial interests rather than the Federal Government,

Efforts to promote treaties and national laws which would support
monopolization of scarce data resources while minimizing: independent
transformational uses of those data, fair use in science and education,
and public accessibility through libraries (2), and

Demands that data funded with taxpayer dollars which support Federal
policies or regulations be publicly accessible regardless of economic
value.

Each of these shifts has implications for other policy issues beyond
development of data resources and their management, but this paper
addresses only that one topic. It is also true that each of these
shifts, to a greater or lesser extent, produces its own set of
counter-forces. For example:

The commercialization of journals is partially offset by expanding
amounts of scientific and technical data being offered on the internet
at affordable or no cost.

Increasing use of government-industry-academic partnerships leverages
the impact of Federal dollars, at least in technological innovation if
not also in basic research.

Efforts to monopolize data legally are being rigorously opposed by those
who would maintain Constitutional patent and copyright policies to
balance the rights of innovators to profit and the rights of the public
at large to information.

Reasonable demands for public accessibility of data which affects
governance are being partially offset by demands for protection of
legitimate research needs. If those needs were ignored, there would
likely be a reduction in the ability of the Federal government to obtain
needed research input (3).

Apart from those factors above, much of scientific and technical data
acquisition is becoming more expensive simply because of increases in
the scope and technology needed to address issues of greater complexity.
Genome research generally requires larger databases than does equivalent
biologic research. Similarly, research in particle physics or in the
chemistry of designer molecules, or in global patterns of climate and
land use change, all require more data generally than their predecessor
areas of investigation.

In general, data acquisition costs will inevitably consume a growing
part of the research dollar. *We must continue to emphasize the
affordable accessibility of scientific and technical data for research,
education, and public information (4)*.

**Emphasis on data utilization over data protection**

The goal of competitiveness inherently implies policies which promote
improvement to the ratio of: (the value of) output relative to (the
costs of) input. Any policy which responds to evolving data economics by
greater hoarding or protectionism toward data will inevitably add to the
rise in data costs. This would increase the competitiveness problem
rather than help to solve it.

Although, for example, the European Union Database Directive is intended
to encourage technical innovation through strong, *sui generis*
protection against violations of a data producers rights, many
scientists predict that it will have the opposite impact on European
science and technical innovation. It will do so by restricting data
sharing essential to scientific and technical progress. Monopolization
of data rights would only be sensible if one could be relatively certain
that the holder or owner of those rights was assuredly the most capable
manager of future innovations the data might contribute to.

*One policy thrust, then, is promotion of affordable scientific and
technical data sharing to aid the accessibility of scientific and
technical data by diverse sources.* This policy thrust would seem to
increase the threat of foreign competitiveness. A far more salutary way
to look at it is to replace the word threat with promise. Global
competitiveness implies improved global technical and economic
advancement. This, in turn, should become a tide which raises all ships.
The U. S. has historically shared in global technical advances. It has
had to adjust, sometimes painfully, to do so. Still, the social and
economic benefits have outweighed the pain.

*The second policy thrust should be to improve scientific and technical
innovation output.* Again, this paper is limited to addressing limited
aspects of the larger issue. The essence of this thrust lie in policies
to improve the utility and efficiency of data in science and innovation,
given its affordable accessibility. Many improvements involve
enhancements to data storage, communication, and computational methods
which are currently addressed in Federal and non-governmental programs
(5).

Some international data sharing initiatives involve a combination of
diplomatic and scientific negotiations, along with participation in
international agencies and projects. All of these initiatives have
involved the emergence of new methods of data handling and analysis,
such as data mining, intelligent search agents, or bioinformatics. Such
advances, to be relevant to specific disciplines, depend also on
advances in understanding of the underlying theory and phenomena in the
specific disciplines involved, refinement of new questions to be
studied, new methods of observation and experimentation, and new ways of
organizing and communicating these advances to peers, laypersons, and
students. Current policies may need to be strengthened or broadened, but
much useful activity is already underway. That is not the case regarding
the role of statistical theory and methods.

**How statistics and statistical ethics can help**

Statistics can help the research and innovation enterprise more than
seems to be common now. At least as much as in other fields, there is an
essential synergy between the sound use of statistical methods and the
adherence to statistical ethics. It is not always clear that the
methods, the ethics, or the synergy are widely understood by scientists
generally. A new statement of statistical ethics seeks to fill that void
(6). Federal policies which would strongly encourage the appropriate and
efficient use of sound statistical methods, ethically applied, are
capable of contributing importantly to the efficiency of technological
innovation. Even where federal funding is not involved, the federal
policies may serve as role models encouraging appropriate use of
statistical methods in the private sector.

Proper use of statistical techniques and analyses such as statistical
sampling,[^1] and exploratory data analysis (EDA[^2]) can leverage
scarce resources for conducting and evaluating research and innovation.
Federal policy should also discourage misuse of statistics whether based
in errant philosophy, lack of current statistical knowledge, or
deliberate efforts to mislead ones peers.

There are also several common problems specifically dealing with
statistics that deserve greater attention than they now receive,
including, but not limited to:

The fashionable assumption in many fields that a probability of error of
less than five percent establishes scientific validity.

The assumption that a statistically significant result from a computer
is meaningful absent a thorough understanding of both the statistical
theory and the software used.

Data selection or manipulation which is designed to produce a predefined
outcome.

Rejection of outlier data points excessively or invalidity.

Selection of analytic methods after seeing the specific data to which
the methods will be applied (8).

None of these is necessarily fatal to valid research, but all of them
introduce significant avoidable dangers. Furthermore, these problems can
be deliberately introduced into research at low risk without engaging in
fabrication, falsification, or plagiarism (9).

It is not necessarily true with todays federal policies that a research
contract offer or a research grant proposal involving statistical
methods will contain adequate detail about a proposed statistical
approach to allow sound determination of its strength. Furthermore, the
page length tolerance in journals will be too slight to allow reporting
of all of the detail about data selection and statistical methodology
which would be necessary to support a definitive review. The standards
for reporting in reputable, peer-reviewed journals can be expected to
carry over into reporting on government grants and contracts.

*New federal policies should encourage wider consideration of
statistical methods to boost research efficiency. Other policies should
require sufficient description of relevant data selection, manipulation,
and analysis techniques to protect against avoidably fruitless research,
careless analysis, and misleading reporting of research results.* All of
these missteps may induce waste not only in the research at hand, but
also in subsequent efforts to replicate apparent results which cannot be
reproduced under suitable statistical controls.

**References**

1\. National Research Council (1997), *Bits of Power: Issues in Global
Access to Scientific Data*. Washington, DC: National Academy Press.

2\. A proposed treaty to impose *sui generis* database intellectual
property protection was tabled in 1996 by the World Intellectual
Property Organization (WIPO). Various \[similar\] proposed U. S. laws
have posed similar risks, the most recent being H.R. 354 of the 106^th^
Congress.

3\. Federal Register: August 11, 1999, 64(154), 43786‑43791

4\. National Research Council, *Ibid.,* "Pricing Publicly Funded
Scientific Data," 124-126.

5\. *Ibid.*, "Trends and Issues in Information Technology," 24-46.

6\. American Statistical Association Online (forthcoming, 1999),
*Ethical Guidelines for Statistical Practice*,
[http://www.amstat.org](http://www.amstat.org/)

7\. Tukey, John W. (1977), *Exploratory Data Analysis,* New York:
Addison-Wesley

8\. Although this principle is mentioned in many statistical textbooks,
the author is indebted to John C. Bailar, III, M.D., Ph.D., for
emphasizing the relevance of this point to science generally in personal
discussions.

9\. Feigenbaum, Susan and Levy, David M. (1996), "The Technological
Obsolescence of Scientific Fraud," *Rationality and Society* 8(3):
261-276.

[^1]: If the entirety of a data set is too massive to be efficiently
    handled by the computers available, and if the phenomena of interest
    are not vanishingly rare, then a series of properly drawn and
    statistically valid samples may provide a reasonable workaround, at
    least for initial investigations

[^2]: EDA (7) consists of a number of processes that allow an
    investigator to explore the distributional characteristics of data
    sets without necessarily deriving any firm conclusions from them.
    EDA allows a researcher to understand the applicability of
    assumptions underlying various definitive analytic methods to the
    data set at hand.