Christopher T. Hill
The University of Tokyo
March 1, 2012

Institutions for policy-making and policy advice
in the realm of science, technology andinnovation
 Analysis and research in support of S,T&I
policy-making
 Academic research and education in S,T&I
policy-making and advice
 Characteristics of the U.S. S,T&I system Contemporary challenges in policy-making forS,T&I
Pre-European era: Shaman advises the chief
 18th century and the “Founding Fathers”
 Benjamin Franklin – America’s first scientist andfounder of the American Philosophical Society Thomas Jefferson – scientist and inventor Alexander Hamilton – 1791 Report on Manufactures

19th century and the age of invention
 Abraham Lincoln – the only President with a patent 1863 – National Academy of Sciences is chartered 1884 – The Allison Commission on the organizationand objectives of Federal scientific agencies

20th century foundations for S,T&I institutions
 1914 – National Research Council of the NAS
 1938 – National Resources Committee writes“Research: A National Resource”
 1940 – National Defense Research Committee
 1941 – Office of Scientific Research andDevelopment
 1945 – “Science-the Endless Frontier” (Bush report)
 1950 – NSF Act creates the National Science Board To share governance of NSF with the NSF director
 To coordinate the entire Federal government’s scientificprograms (generally not done)

1957-2001 The Golden Age of Science Advice





1957 – Presidential science and technology advisor, President’s Scientific Advisory
Committee (PSAC), and Federal Coordinating Council for Science and Technology
1962 – White House Office of Science and Technology
1961-1981 – Growth of R&D and of policy research and analysis units across theFederal government, with decline thereafter
1964 – National Academy of Engineering
1963 – Commerce Technical Advisory Board (first Federal report on innovation 1966)1972 – National R&D Assessment Program at NSF (after Magruder exercise)1974 – Elimination of OST and of President’s Science Advisor by President Nixon
1976 – National Science and Technology Policy and Priorities Act
 Office of Science and Technology Policy OSTP
 Science and Technology Advisor to the President Federal Coordinating Council on Science, Engineering and Technology






1989 – White House Council on Competitiveness1992 – Critical Technologies Institute created to serve OSTP
1993 – National Science and Technology Council (formerly FCCSET)
1994 – OMB/OSTP joint memorandum on S&T budget priorities
2001 – President Bush’s statement on Federal funding for stem cell research

Developments 2002-2012
 2005 – NSF Science of Science and Engineering

Policy Program 2003 – Critical Technologies Institute at RAND
becomes Science and Technology Policy Institute atIDA
 2009 – Involvement of National Economic Council inNational Innovation Strategy
 2009 – Appointment of the first Chief Technology
Officer in the White House



1959 – House Committee on Science and Astronautics(later the Committee on Science or the Committee onScience and Technology)
1970 – Legislative Reorganization Act creates
Congressional Research Service from old LRS
Science Policy Research Division Senior Specialists in Science and Technology (Policy)1972 – Technology Assessment Act creates OTA(de-funded 1995)
1988 – Competitiveness Policy Council mandate(implemented in 1991)
NOTE: GAO and CBO also do some S,T&I analysis
NOTE: Many other congressional committees influence
S,T&I policyAcademic institutions (more below)
Private foundations

Scientific and technical societies











Industrial Research InstituteNational Association of Manufacturers
Semiconductor Industries Association

Think tanksBusiness-Higher Education ForumCouncil on Competitiveness
Science Coalition

Industry associations
American Association for the Advancement of Science (AAAS)
American Physical Society, etc.

Coalitions

Alfred P. SloanIBMCarnegie

American Enterprise InstituteBrookings Institution
Woodrow Wilson International Center for Scholars

National AcademiesCommittee on Science, Engineering and Public Policy
Policy Division of National Research Council




1964 – Harvard Program in Technology and Society (IBMfunding)1968 – NSF “Interdisciplinary Research on Problems of OurSociety” and 1974 -- “Research Applied to National Needs”
supports many academic study groups
1971 – Technology and Human Affairs Program atWashington University, St. Louis1972 – Sloan Foundation program to strengthen the social
science component of engineering education
1970s – Many new “science policy” programs at universities
1977 – National Bureau of Economic Research program on
productivity, innovation and entrepreneurship1986 – AAAS survey of graduate education and careers inscience, engineering and public policy

Masters degree programs in S&T policy
Some based on an applied social science approach Others based on broadening scope of engineering Some focused on policy-making processes Others focused on specific societal problems






Doctoral programs in S&T policy (not common)
Topical focus of dissertations in economics, politicalscience, sociology, business, engineering, physics…
Related but somewhat different fields
Science, technology and society (STS) Systems engineering Management of technology (MOT) EntrepreneurshipHistorical studies of institutions and events
 The politics of science, technology and innovation
 Program and policy evaluation and assessment
 Modeling the dynamics of scientific research andtechnological innovation
 Anticipating the consequences of alternative policy
actions, including investments in new technologies
 Measuring the contributions of R&D investments tohuman welfare Normative and ethical issues in the conduct ofR&D and innovation





The U.S. S,T&I policy-making landscape is highly
decentralized
Many and diverse institutions in and out of government
play significant roles in S,T&I policy-making
Coordination of S,T&I policies is a constant challenge
Decentralization, diversity and coordination problems
are typical of every important arena of public policy
making in the United States
Driven by particular American traditions, including:
Separation of powers (horizontal and vertical)
 Limits on governmental authorities Constitutional right to “petition for redress of grievances”
 Constitutional emphasis on “due process of law”
 Systems of accountabilityNot a top-priority issue for senior policymakers
 Policymakers are dependent on experts but
skeptical of their judgment and authority
 Open and participatory Ad-hoc and incremental
 Redundant, with checks and balances
 Characterized by rapid turn-over of seniorpolicymakers in Congress and the ExecutiveBranchThe outcome of nearly every policy decision isuncertain
 But, decisions about S,T&I policy—especiallyabout funding for R&D—are beset by a profounduncertainty regarding whether anything ofsignificance will result and what it will be worth
 Research is designed to produce knowledge that
we don’t now have. The more ambitious theresearch project, the less we can know in advanceabout its outcomes.
 In fact, if we knew or could calculate the outcomesof a research project in advance, we would notneed to do the project!
Criteria for decisionPlausibility of the proposed project Consistency with known science Presentation of a coherent plan Analogies to previous successful work“Track record” of the investigators
Adequacy of the available resources
Anticipated value of the results, if obtained

Mechanisms for decision

Merit review or peer review
All depend on expert judgment



There has long been dissatisfaction with expert
merit and peer review
 Elitist Privileges established investigators with records
 Is not reliable—reviewer judgments are diverse Is subject to prejudices of all sorts; e.g.,

 Disciplinary
 Schools of thought Gender, race and ethnicity of investigators
 Location of the project

There ought to be a better way!



Proposed “reforms” of the expert merit review system
Support investigators or institutions, not projects
 Require proposers to pay for part of the project, thereby
winnowing out weak ideas
 Give preferences to inexperienced or disadvantaged
investigators
 Use the DARPA “strong program manager” approach Support portfolios of projects and/or redundant projects
 Pay for results, not for effort (the “innovation prize” concept)
 Select winning proposals by lotteryAll but the last two simply relocate the locus of
judgmental merit review, but do not eliminate it
So, we still need a “better way!”If we only had a deep understanding of therelationship between the inputs to scientific andtechnical effort and the resulting outputs we could
use that understanding to make decisions about
what to do and what to support.
 That is, we need to understand “cause and effect”
in the conduct of scientific and innovative activity
 We could call the understanding of the relationshipof cause and effect “the science of science andinnovation policy”






We have no way to build models of processes whose
outcomes are profoundly uncertain and particularistic
Hypothesis: the explanatory power of models of thescientific discovery and innovation processes is limited
by our understanding of the science itself. That is, if
the science is strong enough to be able to predict theoutcome of a new study, then we MIGHT be able to saysomething relatively secure about the likely outcome ofthe study, if we decide to do it. But, it is under thesecircumstances that we are least likely to want to fundthe study.
Where science won’t enable us to predict theoutcomes, we must still fall back on expert judgment

Even if we can’t develop models to predict theoutcomes of particular S,T&I investments, we canstill do very useful work; e.g.,
 Empirical studies of factors that tend to influence
aggregates of outcomes, such as:Funding mechanisms
 Investigator preparation, experience, attitudes, etc.
 Size and disciplinary make-up of research teams
 Incentives and rewards to researchers and institutions
 The social and spatial organization of researchorganizations
 Linkages of researchers to ultimate users of resultsDescription of the incredible complexity of theU.S. system for S,T&I policy-making
 Identification of key periods of change andreform Role of profound uncertainty in making
important S,T&I policy decisions
 How that uncertainty has been addressed
through judgmental merit review
 The promise and pitfalls of the “science ofscience and innovation policy”
Christopher T. Hill
Professor Emeritus of Public Policy and Technology
George Mason University
Senior Fellow
SRI InternationalPrincipal
Technology Policy Internationalchill2@gmu.edu
Tel 865 951 0767