# Artificial intelligence in science: An emerging general method of invention

Stefano Bianchini, Moritz Müller, Pierre Pelletier

► To cite this version:

Stefano Bianchini, Moritz Müller, Pierre Pelletier. Artificial intelligence in science: An emerging general method of invention. Research Policy, 2022, 51 (10), pp.104604. 10.1016/j.respol.2022.104604. hal-03958025

HAL Id: hal-03958025

https://hal.univ-lorraine.fr/hal-03958025

Submitted on 6 Mar 2023

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>
<figure><img src="image_2.png" /></figure>

This article forms part of the Special Issue on The Governance of AI

# Artificial intelligence in science: An emerging general method of invention☆

Stefano Bianchini*, Moritz Müller, Pierre Pelletier

BETA - Université de Strasbourg, France

<figure><img src="image_3.png" /></figure>

## ARTICLE INFO

**Keywords:**

Artificial intelligence

Emerging technologies

Method of invention

Scientific discovery

Novelty

## ABSTRACT

This paper offers insights into the diffusion and impact of artificial intelligence in science. More specifically, we show that neural network-based technology meets the essential properties of emerging technologies in the scientific realm. It is novel, because it shows discontinuous innovations in the originating domain and is put to new uses in many application domains; it is quick growing, its dimensions being subject to rapid change; it is coherent, because it detaches from its technological parents, and integrates and is accepted in different scientific communities; and it has a prominent impact on scientific discovery, but a high degree of uncertainty and ambiguity associated with this impact. Our findings suggest that intelligent machines diffuse in the sciences, reshape the nature of the discovery process and affect the organization of science. We propose a new conceptual framework that considers artificial intelligence as an *emerging general method of invention* and, on this basis, derive its policy implications.

> "In today's world, the magic of AI is everywhere – maybe it's not full AI but there are significant parts."

Nils Nilsson (The Quest for Artificial Intelligence, 2009)

## 1. Introduction

Measurable research outputs such as papers, patents, and innovations have been subject to high enduring growth rates over the last century. Yet, recent empirical evidence suggests that research productivity is ever falling and new ideas are becoming increasingly harder to find (Gordon, 2016; Bloom et al., 2020). A common narrative for this decline in productivity rests on the so-called ‘knowledge burden’. Over the past few decades, data and information have begun to grow and accumulate on an unprecedented scale, and searching through an increasingly vast and complex knowledge space has become prohibitively expensive (Weitzman, 1998; Fleming, 2001; Jones, 2009).

Recent advances in artificial intelligence (AI) – in particular the rapid improvements in prediction achieved by (multi-layer) neural networks (NN) – have brought a wave of optimism that these technologies will speed up scientific discovery (Hey et al., 2009; Agrawal et al., 2018; Cockburn et al., 2018). NN-based models have been found to be particularly good for discovering representations, invariances, and laws, that is, unusual and interesting patterns that are hidden in high-dimensional data (LeCun et al., 2015; Schmidhuber, 2015; Goodfellow et al., 2016). In other words, NNs have shown themselves to be particularly suited to addressing scientific problems.

The first question we raise in this article is whether NNs are, in fact, diffusing into the sciences and, if so, what the mechanics of this diffusion process might be. In so doing, we consider five key attributes that allow a technology to be defined as ‘emerging’ – namely: (i) radical novelty, (ii) fast growth, (iii) coherence, (iv) prominent impact, and (v) uncertainty and ambiguity (Rotolo et al., 2015) – and show that NNs conform to these properties.<fnref n="1" />

<footnote>* The research leading to the results of this paper has received financial support from the CNRS through the MITI interdisciplinary programs [reference: Artificial intelligence in the science system (ARISE)] and the French National Research Agency [reference: DInnAMICS-ANR-18-CE26-0017-01].</footnote>

<footnote>* Corresponding author.</footnote>

<footnote>E-mail addresses: s.bianchini@unistra.fr (S. Bianchini), mueller@unistra.fr (M. Müller), p.pelletier@unistra.fr (P. Pelletier).</footnote>

<footnote n="1">¹ Rotolo et al. (2015) conceive of an emerging technology as “[a] radically novel and relatively fast growing technology characterized by a certain degree of coherence persisting over time and with the potential to exert a considerable impact on the socio-economic domain(s) which is observed in terms of the composition of actors, institutions and patterns of interactions among those, along with the associated knowledge production processes. Its most prominent impact, however, lies in the future and so in the emergence phase is still somewhat uncertain and ambiguous” (p.1828).</footnote>
The second question we address is how NNs influence scientific discovery. Machines are becoming much more than mere scientific instruments, and might even be described as teammates. Today, intelligent machines can engage in various stages of a (complex) problem-solving process. They can, for example, define the problem(s), identify root causes, propose and evaluate solutions, choose between different options, make plans, take actions, and learn from interactions (Seeber et al., 2020). AI and, in particular, multi-layer NNs have been qualified as a *general-purpose invention in the method of invention* (Cockburn et al., 2018), a conceptual framework that blends the concepts of the method of invention (MI) (Griliches, 1957) and general-purpose technology (GPT) (Bresnahan and Trajtenberg, 1995). Building on this idea, Agrawal et al. (2018) suggest that NN-based prediction machines can alter the knowledge production function in combinatorial-type research problems by affecting two dimensions: those of ‘search’ and ‘discovery’. NN ‘search’ methods would support knowledge access by making existing relevant knowledge available to the researcher, whereas NN ‘discovery’ methods would help identify valuable combinations among elements of that available knowledge. Thus, in a needle-in-a-haystack problem, the ‘search’ dimension would arrange the haystack and the ‘discovery’ dimension would find the needle.

This distinction between ‘search’ and ‘discovery’ is conceptually interesting. Yet, it tells us little about how AI influences the direction of knowledge development, because it only deals with one body (or one haystack to stick with the analogy) of pre-existing elements of knowledge. However, there are two sides to the knowledge explosion: increasing knowledge within each domain (i.e., larger haystacks) and an increasing number of domains (i.e., more haystacks). A priori, AI can either help scientists explore familiar conceptual spaces – structured styles of thought – in depth or transform the space by making unfamiliar combinations of distant knowledge elements (Boden, 2004, 2009). The fundamental question, then, is whether AI is currently being used to cope with the knowledge explosion within a domain or to facilitate knowledge creation *across* domains – that is, in-depth exploration of a known domain *vis-à-vis* the transformation of the domain through knowledge recombination across other domains.

Hence, we are interested in investigating empirically how NN methods contribute to science in terms of recombinatorial novelty and *impact*, an analysis confined here to the *health sciences*. In this study, the concept of recombinatorial novelty refers to novel recombinations across domains, as proxied by scientific journals, whereas the concept of impact refers to the relative importance of a study in the scientific community, as proxied by citation indices. We find that NN adoption is negatively associated with recombinatorial novelty, suggesting that researchers are using NNs as a research tool primarily to cope with the knowledge explosion within domains rather than across domains. Interestingly, our results also reveal a considerable degree of uncertainty as regards impact, reflected by a high variation in citation performance. We suggest that this outcome is consistent with the intrinsic nature of emerging technologies, but also with a sort of ‘mode effect’ whereby ‘everyone wants to be AI and data savvy, but few are ready’.

The rest of this paper is structured as follows. Section 2 discusses the emergence of the new data-intensive scientific paradigm; Section 3 presents the method for identifying NN-related research and our sample construction; Section 4 documents aspects of the NN diffusion process in the sciences; Section 5 presents our analysis of the contribution of NN methods to the health sciences; and, the final section concludes by identifying a number of areas that might benefit from policy considerations.

## 2. Data-intensive scientific discovery

*“Few fields are untouched by the machine-learning revolution, from materials science to drug exploration; quantum physics to medicine.”*

Nature Editorial (2019)

Historically, the process of scientific inquiry has evolved through paradigms, seen as symbolic generalizations, metaphysical commitments, values and exemplars that are shared by a community of scientists and that guide the research of that community (Kuhn, 1962).

For most of human history, scientists have been observing phenomena, postulating laws or principles to generalize the complexity of their observations into simpler concepts – i.e., compressed, elegant mathematical representations that offer insights into the functioning of the universe. Originally there were just two sciences, the *experimental* and the *theoretical*. Indeed, Hey et al. (2009) identify empirical observation and logical (theory) formulation as the first and second scientific paradigms, respectively. Towards the middle of the last century, however, many problems proved too complicated to be solved analytically and researchers had to start simulating. Science entered a third paradigm, one characterized by the development of *computational models* and *simulations* to understand complex phenomena. As the knowledge frontier expands and the landscape gets more complex, it is becoming harder and harder for researchers to know enough to find (useful) combinations of knowledge that produce new (valuable) ideas.

Ongoing developments in AI, especially the impressive achievements made using NN techniques, have led to mounting pressure to shift from hypothesis-driven to *data-driven scientific discovery*. The emerging scientific paradigm is being built on data-intensive computing with the massive deployment of intelligent machines capable of finding representations, rules, and patterns in an ever-increasing volume of structured and unstructured data (King et al., 2009; Hey et al., 2009; Nature Editorial, 2019). Even today, Francis Bacon's basic insight continues to hold: the scientists' job is to search for regularities in the empirical data. Bacon probably could not have foreseen that this search is best achieved today with the support of AI.

What makes NNs particularly powerful is the learning process, that is, they learn from past experience and understand the world in terms of a hierarchy of concepts, where each concept is defined by the way it relates to simpler concepts (Schmidhuber, 2015; Goodfellow et al., 2016). It is clear that the term ‘artificial neural networks’ has been coined by analogy with biological neural networks, complete with their neurons, connections and firings. In a general NN model, the variables observed in the data are presented to an input or visible layer composed of several nodes; then a series of hidden layers (also composed of nodes) extracts increasingly abstract features from the data. The term ‘hidden’ stresses the idea that there is no predetermined structure; rather, it is the model itself that learns which concepts are useful to explain the relationships observed in the data. The nodes in the input, the hidden and output layers are all vaguely similar to biological neurons, and the connections between these nodes can be thought of as reflecting the connections between neurons (Hassabis et al., 2017).

NN-based methods can be applied in scientific settings in a variety of ways (see, e.g., Raghu and Schmidt, 2020). The most common application is to use NNs to tackle complex *prediction problems* – i.e., mapping inputs to predicted outputs. By way of example, the input might be an MRI image and the machine has to output a prediction of whether there are any signs of cancer. A second common application is to obtain interpretable insights into which property of the data led to the observed prediction – that is, *from prediction to understanding*. For example, some tools can be used to analyse the hidden representations of a neural network and detect which features of the input are most critical. A third application is to perform complex *transformations of input data*, such as image super-resolution and data compression, which in turn make data analysis easier and save space. Other recent tools, although in their infancy, would help scientists write better papers and co-write codes.

It is clear that intelligent machines can help shoulder the ‘knowledge burden’ within a scientific domain, act as a fertilizer of knowledge recombination across domains, and thus enrich and transform the knowledge space. In short, intelligent machines can influence both ‘search’ and ‘discovery’ processes.

In the case of the ‘search’ process, NNs can support access to
knowledge by predicting which elements of knowledge and information are most relevant to the researcher. Three examples will serve to illustrate this function. First, NN-based recommender systems can offer high quality cross-domain recommendations by exploiting numeric measurements, images, text and interactions in a unified joint framework (Zhang et al., 2019). Second, transformational learning can improve learning tasks in one domain by using knowledge transferred from other (related) domains, and in turn capture generalizations and differences across domains (Olier et al., 2021). And, third, AI can be used for fact-checking, that is, assessing the veracity of scientific claims in sensitive areas such as climate change or Covid-19 pandemic (Wadden et al., 2020).

In the case of the 'discovery' process, NNs provide a better prediction of which elements of knowledge can be combined to produce new knowledge and of the value of that knowledge. Literature-based discovery, for example, is a way to understand implicit (hidden) associations from existing studies, which can result in interesting, surprising, non-trivial hypotheses that are worth studying. Other NN-based tools, such as machine reading comprehension systems, can propose variations on an experiment after having identified gaps in the literature (Badaradan et al., 2020). Highly efficient forms of deep active learning have also been developed that can reduce the uncertainty associated with those regions of the experiment space that are sparsely populated with results (Daugherty and Wilson, 2018).

A major consequence of considering AI as a research tool – indeed, as a teammate – is that its impact is not limited to its ability to reduce the costs of specific scientific activities, but that it can facilitate a new approach to science itself, by modifying the scientific paradigm in the domains where the new research tool is deployed. Exploring the emergence of NN-based technology in science and its impact on scientific discovery is at the core of our study.

### 3. Identifying neural network research

Our empirical analysis of scientific publications exploits two databases: arXiv.org and Web of Science (WoS). First, we use arXiv.org to draw up an appropriate list of search terms referring to NNs based on the natural language processing of scientific abstracts from publications in the subject areas of 'Computer Science', 'Mathematics', and 'Statistics'. Second, these search terms are used to query the WoS database and to extract a sample of NN papers across all scientific fields.

Reliance on a list of search terms for document retrieval is a common practice in research on emerging technologies and science in general. Unfortunately, extant studies do not provide us with an authoritative 'ready-to-use' list of search terms. Here, we train the word embedding model Word2Vec (Mikolov et al., 2013a, 2013b) with scientific abstracts from arXiv.org's documents in order to learn NN-related terms.

Our training sample consists of scientific abstracts from arXiv.org. AI research tends to be a blend of statistics and informatics, but is developed in the main within the computer sciences. Informatics is a fast-developing field in which conference proceedings traditionally play an important role. More recently, however, the rapid dissemination of research is (best) achieved via open access journals and platforms. Of these, arXiv.org is the most prominent and provides us with a rich corpus for the identification of NN-related terms. We downloaded a total of 197,439 abstracts of papers from the subject areas of 'Computer Science', 'Mathematics' and 'Statistics', for the period 1990–2018. The three areas represented roughly 50 % of all arXiv.org documents in 2018, and just 10 % in the early 2000s.

Once pre-processed (details in Supplementary material), the corpus was used to train the Word2Vec model in its skip-gram with negative sampling version. The main outcome of this model is one vector representation for each term in the vocabulary. Hence, we were able to identify the terms that appear in the same cluster as the term 'neural network'. The resulting list of potential search terms included individual words (uni-grams) as well as technical terms consisting of multiple

words (n-grams). We opted to retain only those terms consisting of multiple words – i.e., we removed all uni-grams – in order to err on the side of conservativism and to ensure only the inclusion of terms that relate unambiguously to NNs. Moreover, we retained only the 30 most frequent n-grams after eliminating terms considered as being too generic (e.g., 'short term' or 'supervised learning'). The final list of search terms used in our study is shown in Table 1.

Our sample for subsequent analysis included all publications in the WoS Core Collection published between 1990 and 2018, and having at least one of the search terms (Table 1) in their title, keywords, or abstract. In total, we identified 260,459 documents (144,095 articles; 39,925 conference proceedings; 76,439 others).

### 4. Technology diffusion in the sciences

This Section documents the diffusion of NN-based methods in the sciences. We show that the diffusion dynamics and the characteristics of the technology largely conform to properties of emerging technologies.

#### 4.1. (Relative) fast growth

One of the defining attributes of an emerging technology is the speed of its growth, which is evident in such dimensions as the number of actors involved, the funding made available, and the knowledge output produced.

Our data confirm a 'burst of research activity' in all scientific areas (Fig. 1), although the volume (blue line) varied markedly. 'Technology' (Panel A) is the dominant field, which can be explained in part by the fact that it includes 'Computer Science', the main field of origin. It is followed, with about five times fewer papers, by 'Physical Sciences' (Panel B), which in turn is closely followed by 'Life Sciences &

<table><caption>**Table 1**

NN-related search terms from word embedding.</caption><thead><tr><td>n-gram</td><td>Count</td></tr></thead><tbody><tr><td>neural network</td><td>402,996</td></tr><tr><td>neural networks</td><td>173,470</td></tr><tr><td>artificial neural</td><td>100,749</td></tr><tr><td>artificial neural network</td><td>99,794</td></tr><tr><td>deep learning</td><td>24,104</td></tr><tr><td>convolutional neural</td><td>20,742</td></tr><tr><td>convolutional neural network</td><td>20,595</td></tr><tr><td>recurrent neural</td><td>14,355</td></tr><tr><td>recurrent neural network</td><td>13,965</td></tr><tr><td>deep neural</td><td>9418</td></tr><tr><td>multilayer perceptron</td><td>9352</td></tr><tr><td>deep neural network</td><td>9181</td></tr><tr><td>hidden layer</td><td>7810</td></tr><tr><td>deep convolutional</td><td>4263</td></tr><tr><td>deep convolutional neural network</td><td>3384</td></tr><tr><td>long short term memory</td><td>3122</td></tr><tr><td>hidden layers</td><td>2080</td></tr><tr><td>restricted boltzmann</td><td>1635</td></tr><tr><td>auto encoder</td><td>1444</td></tr><tr><td>generative adversarial</td><td>1242</td></tr><tr><td>encoder decoder</td><td>1198</td></tr><tr><td>adversarial network</td><td>1192</td></tr><tr><td>generative adversarial network</td><td>1085</td></tr><tr><td>fully convolutional network</td><td>688</td></tr><tr><td>convolutional layers</td><td>568</td></tr><tr><td>variational autoencoder</td><td>216</td></tr><tr><td>adversarial attacks</td><td>197</td></tr><tr><td>adversarial examples</td><td>92</td></tr><tr><td>variational autoencoders</td><td>75</td></tr><tr><td>adversarial perturbations</td><td>24</td></tr></tbody></table>

Notes: The count refers to how many times a given term occurs in the Web of Science corpus. A document may (and very often does) include several terms. Adding more terms would only slightly change the number of documents retrieved from WoS, as can be seen from the counts of the last few terms.
<figure><img src="image_4.png" /><figcaption>**Fig. 1.** Trends in NN publication activity by scientific area.</figcaption></figure>

Notes: The blue lines show the number of publications and the orange lines plot the growth rates in each scientific area. Growth rates are calculated as three-year moving averages and omitted publications before 2001. Scientific areas correspond to WoS research areas. Panel H refers to research published on arXiv.org, based on the sample discussed in Section 3.

Biomedicine' (Panel C). Publications in 'Health Sciences' (Panel D) – defined as a subset of 'Life Sciences & Biomedicine' and the focus of the analysis conducted in the next Section – largely parallel those of 'Life Sciences'. Publication counts in 'Social Sciences' (Panel E) are relatively low, becoming negligible for 'Arts & Humanities' (Panel F). Panel G, which combines all WoS documents into one, shows that the (three-year average) growth rate (orange line) in NN publication activity around 2005 was high (at about 10 %), it then suffered something of a decline in the years around 2010, before recovering and experiencing steady growth to the end of the observation period (reaching 20 %). Indeed, the individual areas exhibited very similar growth patterns.<fnref n="2" />

Publication activity on arXiv.org (Panel H) follows essentially the same dynamics. Growth rates mimic the shape described above but are about five times higher than those in the WoS panels. The comparatively higher rates are attributable it would seem to the fact that open platforms are increasingly popular, given their efficiency and speed, as a channel of communication between researchers, particularly within the machine learning and computer science communities (Sutton and Gong,

<figure><img src="image_5.png" /></figure>

2017).

Research output increased not only in absolute numbers but also relative to the overall number of papers in a given scientific area, albeit at a lower level. In 2018, NN documents represented 2.6 % of all papers in ‘Technology’, 1.02 % in ‘Physical Sciences’, and 0.3 % in ‘Life Sciences & Biomedicine’. This means NN publications still account for only a tiny fraction of the whole research volume, in particular in application domains. However, recent growth rates in these shares are remarkable. NN-related research presents the highest growth rates in the ‘Life Sciences & Biomedicine’ (47.3 % in the period 2017–2018), ranks second in ‘Physical Sciences’ (42 %), and in ‘Technology’ presents a growth rate of roughly 18 %.

## 4.2. Spatial diffusion and actor re-configuration

Another of the defining attributes of an emerging technology is the speed of change in the configuration of actors – e.g., countries, users, and scientists.

Fig. 2 shows the dynamics of science at the country level. Each document is attributed to a given country when the affiliation of at least one of its authors is in that country. During the first period, 1990–1999, most of the documents (about 5000) were published by scientists in the United States. Publishing activity was relatively low in absolute numbers in the European countries, Australia and China, and negligible or non-existent in most other countries. In the following decade, 2000–2009, China became the most prolific country with about 20,000 documents. The US ranked second with around 14,000 articles, whereas

<footnote n="2">² The overall number of NN-related documents varies according to the sub-disciplines within each scientific area (not shown here). The general trend in ‘Technology’ is driven mainly by ‘Computer Science’ (103,729 documents), ‘Engineering’ (95,638) and ‘Automation & Control Systems’ (24,721). In the case of ‘Physical Sciences’, it is driven by ‘Physics’ (7239), ‘Mathematics’ (5123) and ‘Chemistry’ (3702), while in ‘Life Sciences & Biomedicine’, it is driven by ‘Environmental Sciences & Ecology’ (2632), ‘Neurosciences & Neurology’ (2032), and ‘Biochemistry & Molecular Biology’ (1728).</footnote>
<figure><img src="image_6.png" /><figcaption>**Fig. 2.** Global diffusion of NN in science across countries.</figcaption></figure>

Notes: The intensity of colour reflects a country's relative number of NN publications in a given period, with no observed NN publication activity in hatched countries [WoS sample].

European countries, Australia, Canada, and India grew sufficiently to maintain their relative strength in the field. Interestingly, in this decade, NN research activity took off in an increasing number of countries. These trends were further reinforced in the last period, 2010–2018. Compared to the previous decade, China doubled its research output, widening the gap with the US and, to a lesser extent, with the EU.

In summary, our data seem to suggest that NN research has diffused rapidly at the global scale, and that since the early stages of development there has been a re-configuration of global actors. We consistently observed high volatility in the rankings, with some countries climbing the ladder and others lagging behind.

### 4.3. Radical novelty and ‘double-boom’ cycle

NNs have experienced a discontinuous wave of major innovations, which points to the radical nature of this technology. (Artificial intelligence has a long, rich history dating back to the 1950s, when researchers from different domains began to explore various paths towards mechanizing intelligence – interested readers may consult Nilsson (2009) and Russell and Norvig (2021 – Ch.1 and 23).)

Novelty can also arise from putting the technology to a new use – that is, applying it from one domain to another (Adner and Levinthal, 2002). The originating domain of NN research is predominately computer science; thus, it seems appropriate to follow Cockburn et al. (2018) and assume that NN publications in all areas other than computer science represent applications of NN methods to address field-specific research problems.

The diffusion of emerging technologies from the originating domain to the application domains typically follows a ‘double-boom’ cycle (Schmoch, 2007). Initially, the new technology seems to be of high potential, and high expectations trigger considerable development efforts, especially theoretical – the first boom. However, during these early development activities, several actors discover the difficulties of translating theory into practice. Most fail and cease their innovation activities, putting an end to the first boom. But some continue and, as time passes, they overcome some of the more important practical hurdles and are able to demonstrate genuine advances – starting the second boom. Interestingly, this pattern is largely consistent with the growth patterns recorded in Fig. 1 (orange lines), where the first boom, subsequent decline, and second boom are clearly evident.

We also find that the second boom is marked by a shift in emphasis from theoretical principles to practical applications. In support of this evidence, we considered the top five cited references in each year of the observation period (i.e., those documents with the highest annual shares of all cited references in our publications), which gave us a list of 18 unique articles and their corresponding citation counts, as shown in Table 2. Using dynamic time warping (DTW) to measure dissimilarity between time series, we then clustered these temporal sequences by means of k-medoids (Berndt and Clifford, 1994). As shown in Fig. 3, we obtained two clusters. In the first period, the most cited articles in our sample were theoretical contributions, including a discussion of the possibility of using multilayer feedforward networks as universal
<table><caption>**Table 2**
Influential NN publications.</caption><thead><tr><th>Title | journal</th><th>Cluster</th><th># Citations</th><th>Share [%]</th></tr></thead><tbody><tr><td>Multilayer feedforward networks are universal approximators | NN</td><td>1</td><td>5904</td><td>0.14</td></tr><tr><td>Neural networks and physical systems with emergent ... | PNAS</td><td>1</td><td>4658</td><td>0.11</td></tr><tr><td>Learning representations by back-propagating errors | Nature</td><td>1</td><td>4645</td><td>0.11</td></tr><tr><td>Learning internal representations by error propagation | MIT Press</td><td>1</td><td>3921</td><td>0.09</td></tr><tr><td>Approximation by superpositions of a sigmoidal function | MCSS</td><td>1</td><td>3657</td><td>0.09</td></tr><tr><td>Training feedforward networks with the Marquardt algorithm | IEEE TNNLS</td><td>1</td><td>3128</td><td>0.07</td></tr><tr><td>ANFIS: adaptive-network-based fuzzy inference system | IEEE SMC</td><td>1</td><td>2909</td><td>0.07</td></tr><tr><td>Identification and control of dynamical systems using ... | IEEE TNNLS</td><td>1</td><td>2551</td><td>0.06</td></tr><tr><td>Cellular neural networks: theory | IEEE CAS</td><td>1</td><td>2267</td><td>0.05</td></tr><tr><td>ImageNet classification with deep convolutional neural networks | NeurIPS</td><td>2</td><td>7177</td><td>0.17</td></tr><tr><td>Gradient-based learning applied to document recognition | IEEE Proceedings</td><td>2</td><td>3590</td><td>0.09</td></tr><tr><td>Deep learning | Nature</td><td>2</td><td>3542</td><td>0.08</td></tr><tr><td>Long short-term memory | NC</td><td>2</td><td>3074</td><td>0.07</td></tr><tr><td>A fast learning algorithm for deep belief nets | NC</td><td>2</td><td>2710</td><td>0.06</td></tr><tr><td>Reducing the dimensionality of data with neural networks | Science</td><td>2</td><td>2621</td><td>0.06</td></tr><tr><td>Very deep convolutional networks for large-scale image recognition | arXiv</td><td>2</td><td>2582</td><td>0.06</td></tr><tr><td>Particle swarm optimization | IEEE Proceedings ICNN</td><td>2</td><td>2568</td><td>0.06</td></tr><tr><td>Deep residual learning for image recognition | IEEE Proceedings CVPR</td><td>2</td><td>2160</td><td>0.05</td></tr></tbody></table>

Notes: This table reports the references (title and journal) of the most cited articles from the WoS publication sample over the period 2000–2018. From a total of 4,190,306 references (1,618,836 unique) cited by the documents in our sample, we selected the five most used references for each year. This gives us 18 time series that were clustered. Clustering is obtained via *k*-medoid and dynamic time warping. References within clusters ranked by total number of citations.

function approximators, training algorithms (backprop), and parallel computing theories (cellular NN). In the second period, the most influential articles were no longer theoretical contributions, but rather articles that show how to put theoretical principles into practice. These contributions included inventions that have brought enormous performance gains to real-world tasks, especially for image and text analyses (e.g., deep convolutional neural networks and long short-term memory (LSTM) architectures).

### 4.4. Coherence

Another defining attribute of an emerging technology is its coherence, understood as the shared interpretation and acceptance of the technology within a community. Signals of coherence can include the creation of dedicated conference sessions, new specialist journals and new categories in established classification systems. Here, we consider the transition from cross-disciplinary to disciplinary research effort as a sign of coherence, as this would mean that the technology has moved beyond its conceptual stage requiring close interaction between users and developers, and has become ‘common practice’ in application domains.

Each document is labelled by WoS as belonging to at least one subject category on the basis of the journal in which it was published. In most instances, a document falls into more than one category. The extent to which publications in a given scientific area are cross-classified as computer science contributions can therefore proxy cross-disciplinarity with respect to computer science. Thus, for each broad scientific area and year, we calculated the fraction of NN documents that are (also) labelled as ‘Computer Science’.

Fig. 4 shows the corresponding time trends. Each point of the plot for ‘Technology’ (Panel A) represents the average number of ‘Technology’ NN documents cross-classified as ‘Computer Science’ in a given year. For example, in 1990 about 60 % of ‘Technology’ publications also fell into the ‘Computer Science’ category (first dot). The overall trend (blue line) follows a flat U-shape that reaches around 70 % in 2005, before falling to less than 50 % by the end of the observation period. Indeed, in 2018, a large proportion of papers in ‘Technology’ are no longer labelled as computer science contributions. ‘Physical Sciences’ (Panel B) also

<figure><img src="image_7.png" /><figcaption>**Fig. 3.** Trends in annual citations of influential NN publications.</figcaption></figure>

Notes: This figure shows the annual share of all citations in the Web of Science sample for the two clusters of most cited NN articles. The shaded areas are time series intervals defined by minimum and maximum citation shares. In the main, the orange profile represents ‘theoretical’ contributions and the blue profile represents ‘applications’. Due to the limited number of articles that could be cited in the initial period, we clustered the time series from 2000 onwards.
<figure><img src="image_8.png" /><figcaption>**Fig. 4.** NN publications cross-classified as ‘Computer Science’. Notes: The figures show the fraction of NN documents cross-classified as ‘Computer Science’. Orange dots represent the share of cross-classified papers in each year. The blue curve corresponds to a simple local regression, with the surrounding shaded area representing the 95 % confidence interval around the mean.</figcaption></figure>

presents an inverse U-shape, with an increase in cross-classified computer science documents that reached 20 % in 2000, before falling to 10 % by the end of the period. No increase in computer science cross-classification was observed in ‘Life Sciences & Biomedicine’ (Panel C). From the very high share of 70 % at the beginning of the period, a continuous decline was subsequently recorded (with significant drops around 2000 and again in 2010), finishing the period at around 20 %. ‘Health Sciences’ (Panel D) presents the same evolution. ‘Social Sciences’ (Panel E) increased their share of computer science documents to 40 % around 2010, but this was followed by a sharp downturn, while in ‘Arts & Humanities’ (Panel F), the share of computer science documents is very noisy, and no particular trend can be deciphered.

Taken together, these dynamics suggest that NNs diffuse from computer science, the originating discipline, into other application-oriented scientific disciplines. Thus, over time, we see a greater propensity of different communities to integrate the technology into their discipline, which is a good signal of coherence.

In short, it is more than apparent that NN technology fulfils many of the conditions to be classified as an emerging technology. It exhibits rapid growth in all domains; it has experienced a turbulent shift and reconfiguration of the actors involved in its development and adoption; and it presents a degree of coherence that persists over time. However, the picture arrived at in the first part of this analysis is incomplete. How does the technology influence scientific discovery in its domains of application? What can be learned about its undoubted impact yet, at the same time, the uncertainty that is often associated with its adoption? We address these questions in the next Section.

# 5. Neural networks in the health sciences

Here, we specifically address the impact of NN-based methods in the ‘Health Sciences’, one of the application domains with the highest short-term societal impacts (Raghupathi and Raghupathi, 2014; Miotto et al., 2018). AI, in general, and deep learning, in particular, have already contributed to a variety of data-driven innovations in the health domain – improving healthcare systems, supporting clinicians, and monitoring patient diseases, among others. A review of the literature enabled us to identify applications in virtually all sub-disciplines: health informatics and biomedical research (Marx, 2013; Ravi et al., 2017), computational biology (Angermueller et al., 2016), genomic medicine (Leung et al., 2015), medical imaging (Litjens et al., 2017; Shen et al., 2017; Savadjiev et al., 2019), drug discovery and pharmacogenomics (Ma et al., 2015), real-time patient monitoring (Rajkomar et al., 2018), public health (Miotto et al., 2018; Zhang et al., 2018), and neuroscience and the cognitive sciences (Marblestone et al., 2016; Hassabis et al., 2017; Lake et al., 2017).<fnref n="3" />

## 5.1. Novelty and impact in science

A ‘scientific contribution’ is typically considered as comprising two elements: *novelty* and *impact*. Different terms for essentially this same

<footnote n="3">³ We define the ‘Health Sciences’ as comprising 83 Web of Science subject categories within the ‘Life Sciences & Biomedicine’ research area. The complete list of categories included can be consulted in the Supplementary material.</footnote>
idea were used in earlier studies of science, so that debates centred on discussions of the notions of originality, discovery and breakthrough and contributions to scientific progress (de Solla Price, 1963; Merton, 1957; Bourdieu, 1975). It was Kuhn (1962) who coined the term 'novelty' to describe a more radical contribution that does not simply make an incremental advance in the 'normal science' in place, but rather breaks the current paradigm. More recently, the term novelty has partly lost this radical connotation, but it still carries the idea of a high degree of originality, while the concept of 'recombinatorial novelty' has emerged to highlight the idea that new knowledge arises out of the recombination of previously generated bits of knowledge (Fleming, 2001; Arthur, 2009; Uzzi et al., 2013; Wang et al., 2017).

Only a very small percentage of the potential for useful re-
combinations in the knowledge space is currently exploited. NNs can
change the way science develops by helping to overcome our human
limitations (Agrawal et al., 2018; Cockburn et al., 2018; Furman and
Teodoridis, 2020). Yet, how exactly does NN adoption correlate with
novelty? The answer to this question depends very much on how the
technology is used in the scientific complex. Indeed, scientists can adopt
new methods either to advance well-established research trajectories
within a conceptual space or to explore new avenues by altering the
conceptual space with knowledge from other domains, leading to low
and high recombinatorial novelty, respectively.

The second element of a 'scientific contribution' concerns its impact, a key attribute of emerging technologies. Impact is related to, but different from, novelty; if research provides novelty, that novelty must be adopted by the scientific community in order for its impact to be felt. And, moreover, research can have an impact on subsequent research for reasons other than (recombinatorial) novelty, especially when providing new insights within established knowledge structures.

Yet, nor should impact be considered fully independent of novelty.
Evidence suggests that a high degree of novelty is likely to increase the
risk of delays and failures (Azoulay et al., 2011). Moreover, novel
research often requires more complex and risky collaborative social
structures (Fleming et al., 2007; Foster et al., 2015). Thus, highly novel
research can be subject to considerable variations in 'quality' (Fleming,
2001; Wang et al., 2017) and, hence, to greater variations in impact.
Uncertainty and ambiguity are common features of the research process,
especially because the potential applications of the technology have yet
to be explored and understood. Social inertia can further reinforce the
uncertainty associated with impact. Emerging technologies typically
encounter resistance in society precisely because they cause structural
changes in roles and norms (Merton, 1957; Bourdieu, 1975). This is
particularly true of AI which operates at the intersection of ethical and
legal considerations and, as such, is shaping the future of both in-
dividuals and society as a whole (Lanier, 2010; O'Neil, 2016; Zuboff,
2019).

5.2. *Empirical analysis*

We measure scientific knowledge creation in scientific papers pub-
lished in peer-reviewed journals and conference proceedings in the
'Health Sciences'. Henceforth, the term 'journal' is used to refer inter-
changeably to both peer-reviewed scientific journals and conference
proceedings. We restrict our focus to journals that are not cross-
classified as 'Computer Science' journals, ensuring that publications
include NN methods as a research tool.

Our approach is to compare publications that involve NNs with those
that do not involve NNs, while controlling for a set of confounding
factors. Comparisons are made in terms of their recombinatorial novelty
and scientific impact. For the main analysis, we operationalize the
concept of 'recombinatorial novelty' as the first appearance of a
knowledge combination, very much in line with Wang et al. (2017), the
details of which we describe below. Novelty à la Wang et al. complies
with the idea of NNs as a method of invention – i.e., a method for
creating something new and valuable. In the case of 'scientific impact',

we operationalize this concept as the subsequent use made of a paper,
measured by the number of citations received.

5.2.1. Sample

We include all the articles for the whole observation period (2000–2018) published in those health journals where research involving NNs has been most prominent. This provides us with a relatively coherent knowledge base against which we can examine the concepts of novelty and impact. In total, we identified 26,461 NN health papers in about 5000 health journals and proceedings. Roughly 45 % (11,520) of these documents are published in the top 100 health journals in the sample. Hence, we downloaded the entirety of these journals for the period 1990–2018. Our final sample, combining NN and non-NN publications, contains 1,081,223 articles.

5.2.2. *Variables*

Our main explanatory variable is a binary indicator of a paper's NN
content: 1 if the paper involves the use of NN methods, 0 otherwise. Our
main dependent variables are (various measures of) recombinatorial
novelty and scientific impact based on citation counts.

Recombinatorial novelty is measured in relation to the journals referenced by a paper. Thus, each paper is examined to determine whether it makes ‘first-time-ever’ combinations of referenced journals – i.e., its list of references contains journal pairs that have never previously appeared jointly in any list of references. In order to exclude journal pairs that simply formed once by happenstance, we further impose the condition that journal pairs be observed again within the next three years. A paper with at least one journal pair in the reference list that is both novel and that has been re-used, is considered as providing some novelty. Thus, we construct a binary indicator of novelty, henceforth referred to as *Novelty Dummy*. A further consideration is that a novel journal pair may span domains that vary in their distance one from another (i.e., more or less distant). This subtlety is captured through the co-citation profiles of the two journals forming a novel pair. The idea is that if both journals are often (rarely) cited with the same third journal(s), they are likely to span less (more) distant domains. In this way, we are able to construct a distance-weighted (continuous) measure of novelty, henceforth referred to as *Novelty*.

Calculations of the binary and weighted novelty measures follow
Wang et al. (2017). However, our procedure differs in two major re-
spects. First, we judge novelty and co-citation distance only on journal
pairs that are observed in the reference lists of our sampled papers. Thus,
we do not measure novelty per se but rather with respect to a knowledge
base covered by the sampled health journals.

Second, we calculate different measures of novelty by considering
different sets of journals in the references. In this way, we are able to
capture the *source* of novelty – i.e., where does this novelty come from?
ICT, health, or other domains? While it is true that all the articles in our
sample are published in outlets of the ‘Health Sciences’, they can
reference journals in various domains. For instance, a health science
paper involving NNs is likely to cite computer science journals where the
NN methods were first published. This translates into a recombinatorial
novelty ‘simply’ because of the adoption of the method, but it does not
necessarily reflect the recombinatorial potential of NNs to connect and
recombine knowledge in complex knowledge landscapes. In other
words, we seek to measure whether NN adoption fosters novel re-
combinations *within* the health sciences and/or *between* the health sci-
ences and disciplines other than the computer sciences. Thus, we
calculate novelty not only in journal pairs, as indicated by ‘All Sciences’,
but also limited to journal pairs where (i) no referenced journal is
classified as a computer science journal, indicated by ‘No CS’; and (ii)
both referenced journals are uniquely classified as health sciences,
indicated by ‘Only HS’. By way of example, the combination of ‘Biology
& Biochemistry’ and ‘Computer Science’ journals can be regarded as an
‘All Sciences’ combination; ‘Engineering’ and ‘Molecular Biology & Genetics’ as a ‘No CS’ combination; and ‘Neuroscience & Behaviour’ and ‘Psychiatry/Psychology’ as an intra-domain ‘Only HS’ combination.

Combining these three recombinatorial options with the possibility of calculating novelty as either a binary indicator or a continuous score, we obtain six different novelty measures, namely: *Novelty Dummy (All Sciences)*, *Novelty Dummy (No CS)*, *Novelty Dummy (Only HS)*, *Novelty (All Sciences)*, *Novelty (No CS)*, and *Novelty (Only HS)*.

Impact is measured by the number of citations (# Citations) received by a paper from its year of publication up to 2019, the time of data extraction. Furthermore, we code dummy indicators for so-called ‘big hit’ contributions – i.e., highly cited papers. Whether a paper is among the top 5 or 10 % cited papers (*Top 5 % Cited* and *Top 10 % Cited*) is calculated with reference to other papers published in the same year and falling in the same WoS subject category.

We consider a set of control variables to capture various characteristics of a focal paper. We control for the number of references made by a paper (# References) as this might automatically increase the likelihood of its having new combinations. In prior research, the number of authors has been shown to be positively associated with both novelty and impact, hence we control for that (# Authors). The adoption of AI in scientific settings can indeed have an ambiguous effect on team size. Size may increase as new members are needed to manage the technology (at least in the early stages), but the technology may also automatize some tasks, thereby generating a replacement effect in the scientific workforce. International collaborations may also be a source of novelty and impact, and may be instrumental in the adoption of the technology. We proxy international collaboration by a dummy (International Collab.) taking a value of 1 if there are at least two different countries in the authors' affiliations, 0 otherwise. For the same reason, we construct a dummy for private sector participation (Private Partic.) taking a value of 1 if the paper has at least one non-university affiliation in the list. We consider the journal impact factor (JIF), since, on the one hand, high impact journals may be biased against novelty, but, on the other, increase visibility and hence citations. We additionally control for the journal age (Journal Age). Finally, we include a dummy indicating whether the paper provides a review or survey of extant literature (Survey). A survey may in fact cover separate streams of research without really connecting them.<fnref n="4" /> Descriptive statistics of the variables are reported in Appendix A.

### 5.2.3. Estimation methods

We model three different types of outcome: (i) binary indicators of novelty and impact, (ii) positive continuous measures of novelty, and (iii) positive discrete measures of impact (number of received citations). Each type of outcome requires a specific econometric setting.

All binary indicators are modelled with a Probit. Our continuous novelty measure is censored at zero, hence we use a Tobit model. Citations are count data for which the Poisson and negative binomial models are natural candidates. Over-dispersion and the conditional mean of the outcome variable being much lower than its variance are the most common arguments for favouring the negative binomial over the Poisson model. In our case, both empirical arguments hold; therefore, we opted for the negative binomial to model mean and dispersion separately, each with a linear predictor incorporating our main left-hand side variables and controls.

In all estimations, we include the control variables discussed above and a set of dummies to control for scientific field and cohort effects. We proxy scientific field using WoS categories (field WC). As a paper may fall into several categories, we code dummy variables taking a value of 1 for each category. Throughout the analysis, robust standard errors clustered at the journal-level are obtained via bootstrapping all journals.

### 5.2.4. Results

Table 3, Columns 1–3, shows the Tobit regressions of the continuous measures of novelty, *Novelty*. Columns 4–6 report the Probit estimates of the binary novelty indicators, *Novelty Dummy*.

When considering recombinatorial novelty across all sciences (Column 1), the estimated coefficient is positive but non-significant, but when we exclude computer science references (Column 2) the coefficient becomes negative yet remains non-significant. Restricting references to health sciences only (Column 3) increases the negative coefficient, which is now significant below the 1 % significance level. The same pattern is observed when we consider the results of the Probit regression of the novelty dummy.

To what extent does the adoption of NN methods change our expectations of recombinatorial novelty in the health sciences? To a considerable degree, given that adopting NN decreases the degree of novelty by 18.6 %. In addition, the marginal effects of Probit (Column 6) tell us that, for the median observation, NN decreases by 0.031 the probability of an article being novel (0.037 for the average observation).

In sum, NN adoption is not significantly correlated with novel recombinations across the entire knowledge landscape, nor with novel recombinations involving anything other than computer sciences. Yet, it is significantly and negatively correlated with novel recombinations within the health sciences. These findings suggest that NN methods tend to be adopted as part of a ‘balancing strategy’ in which the risk associated with the (emerging) technology is counterbalanced by keeping the knowledge landscape stable. Another way of interpreting this outcome is that NNs are employed mainly as a research tool to support already formalized and well-defined research trajectories in the health sciences community. This evidence is consistent with the idea of extending science while maintaining the advantages of conventional domain-level thinking (Boden, 2004; Uzzi et al., 2013).

Our estimates of the control variables echo previous research. Larger teams are associated with more novelty (Fleming et al., 2007; Lee et al., 2015); international collaborations are negatively associated with novelty (Wagner et al., 2019); the chances of providing a new combination of journal references increase with the number of references (Wang et al., 2017); and, literature reviews also tend to draw from a wider range of sources leading to novel combinations of references. We find a negative effect of private involvement and, finally, a journal's age and impact factor seem to play no role.

How does NN adoption correlate with impact? Table 4, Column 1, shows the results of the negative binomial regression of citation counts. Here, the mean and dispersion parameters may vary with various right-hand side factors.<fnref n="5" /> We find that NN adoption positively and significantly affects the number of citations received, both in terms of expectation and variance. Compared to non-NN papers, ceteris paribus, NN papers receive on average 10.32 % more citations. The expectation of citation count increases by a median of 6.01 for NN research. The dispersion of the citation distribution is 19.57 % higher for NN papers than for non-NN papers.

The Probit regressions used to model the probability of a paper falling in the right tail (top 5 % or 10 %) of the year-field citation

<footnote n="4">⁴ Private Partic. takes a value of 1 if we detect in the authors' affiliation at least one of the acronyms present in the Wikipedia page: 'List of legal entity types by country'. We use the SCImago Journal Rank to obtain the impact factor (JIF) for each journal in each year. *Journal Age* is calculated as the time elapsed from the date of the journal's creation to the year of publication. *Survey* takes a value of 1 if we detect in the title of the paper the terms 'Survey', 'Overview' or 'Review'.</footnote>

<footnote n="5">⁵ We excluded dummy variables other than NN to model the dispersion of citations because these variables caused problems with the convergence of the maximum likelihood estimator. In modelling the dispersion, we also tried simpler specifications by progressively incorporating a few variables at a time.</footnote>
<table><caption>**Table 3**
Novelty profile of NN publications.</caption><thead><tr><th rowspan="2"></th><th colspan="3">Tobit: Novelty</th><th colspan="3">Probit: Novelty Dummy</th></tr><tr><th>All Sciences<br>(1)</th><th>No CS<br>(2)</th><th>Only HS<br>(3)</th><th>All Sciences<br>(4)</th><th>No CS<br>(5)</th><th>Only HS<br>(6)</th></tr></thead><tbody><tr><td>NN</td><td>0.044<br>(0.038)</td><td>-0.031<br>(0.034)</td><td>-0.186***<br>(0.040)</td><td>0.053<br>(0.037)</td><td>-0.008<br>(0.033)</td><td>-0.150***<br>(0.037)</td></tr><tr><td># References (log)</td><td>1.046***<br>(0.033)</td><td>1.050***<br>(0.033)</td><td>1.029***<br>(0.033)</td><td>0.878***<br>(0.026)</td><td>0.879***<br>(0.026)</td><td>0.843***<br>(0.023)</td></tr><tr><td># Authors (log)</td><td>0.177***<br>(0.021)</td><td>0.184***<br>(0.022)</td><td>0.227***<br>(0.024)</td><td>0.184***<br>(0.020)</td><td>0.189***<br>(0.020)</td><td>0.223***<br>(0.022)</td></tr><tr><td>International Collab.</td><td>-0.053***<br>(0.010)</td><td>-0.058***<br>(0.010)</td><td>-0.084***<br>(0.010)</td><td>-0.050***<br>(0.009)</td><td>-0.054***<br>(0.010)</td><td>-0.076***<br>(0.009)</td></tr><tr><td>Private Partic.</td><td>-0.004<br>(0.012)</td><td>-0.004<br>(0.012)</td><td>-0.027*<br>(0.014)</td><td>-0.007<br>(0.012)</td><td>-0.008<br>(0.013)</td><td>-0.026**<br>(0.013)</td></tr><tr><td>JIF</td><td>-0.026<br>(0.019)</td><td>-0.024<br>(0.019)</td><td>-0.017<br>(0.021)</td><td>-0.025<br>(0.017)</td><td>-0.024<br>(0.017)</td><td>-0.017<br>(0.018)</td></tr><tr><td>Journal Age (log)</td><td>-0.098<br>(0.099)</td><td>-0.082<br>(0.100)</td><td>-0.044<br>(0.108)</td><td>-0.074<br>(0.090)</td><td>-0.061<br>(0.090)</td><td>-0.030<br>(0.095)</td></tr><tr><td>Survey</td><td>0.225***<br>(0.049)</td><td>0.216***<br>(0.047)</td><td>0.181***<br>(0.050)</td><td>0.206***<br>(0.049)</td><td>0.199***<br>(0.047)</td><td>0.163***<br>(0.046)</td></tr><tr><td>Log likelihood</td><td>-263,098</td><td>-258,255</td><td>-221,241</td><td>-180,701</td><td>-178,639</td><td>-161,710</td></tr><tr><td>χ² [null model]</td><td>96,074***</td><td>94,950***</td><td>77,374.6***</td><td>75,936***</td><td>75,187***</td><td>64,730***</td></tr><tr><td>χ² [w/o NN model]</td><td>4.90*</td><td>2.20</td><td>60.90***</td><td>6.70**</td><td>0.10</td><td>44.60***</td></tr><tr><td># obs</td><td>356,037</td><td>356,037</td><td>356,037</td><td>356,037</td><td>356,037</td><td>356,037</td></tr></tbody></table>

Notes: This table reports coefficients of the effect of NN methods (NN, dummy) on recombinatorial novelty built by considering different knowledge landscapes. Bootstrapped (500 replications) standard errors clustered at the journal-level in parentheses: ***, ** and * indicate significance at the 1 %, 5 % and 10 % levels, respectively. The effect of NN on the positive continuous novelty measure is estimated using a Tobit regression (Columns 1–3). The effect on the novelty dummy is estimated using a Probit (Columns 4–6). Each novelty measure is calculated on three different sets of journal references: ‘All Sciences’ – All cited journals, ‘No CS’ – All cited journals except for computer science journals, and ‘Only HS’ – Only citations to health science journals. Constant term, scientific field (WoS subject category) and time fixed effects are incorporated in all model specifications. Likelihood-ratio tests are used to compare the goodness-of-fit of two statistical models: (i) null model against complete model; (ii) model without the NN variable against the complete model.

distribution corroborate the results. The marginal effects suggest that research involving NN has a 0.019 (median value) higher probability of being in the top 10 % of the most influential contributions (0.027 mean value), and a 0.009 higher probability of being in the top 5 % (0.014 mean value).

As for the controls, the number of authors is positively related to impact (Lee et al., 2015) and reduces impact variation; international collaborations increase citation expectations (Glänzel and Schubert, 2001); publishing in a high impact factor journal further increases the average number of citations; surveys and other papers with many references tend to attract more citations; and, finally, a negative effect is found between private participation and scientific impact, albeit not particularly significant.

In sum, the econometric analysis shows that research using NN has a high potential for greater impact, on the one hand, but that it is also associated with greater uncertainty of having an impact, on the other. There are several (complementary) explanations for this uncertainty: the ‘high-risk/high-gain’ that characterizes the adoption of emerging technologies and breakthrough research (Rotolo et al., 2015; Wang et al., 2017); the challenge of integrating the scientific instrument into existing scientific practices (Rosenberg, 1992); the ability to extract the true potential from the instrument and not to adopt it simply because ‘everybody does’; and the possible social resistance, especially in sensitive domains, as some areas of the health sciences are known to be.

Based on these results, we propose that AI – here, specifically, NN methods – be regarded as an *emerging general method of invention*: ‘emerging’ because it shares the key attributes of emerging technologies; ‘general’ because it is increasingly integrated as a research tool in many scientific domains; and, a ‘method of invention’ because it has great potential for impact in application domains. We consider it more appropriate to consider AI an emerging general method of invention as opposed to a general-purpose method of invention (as in Cockburn et al., 2018) for two reasons. First, as we have seen in Section 4, although growing, the proportion of scientific contributions related to NNs remains marginal compared to the whole body of scientific activity.

Second, whether or not AI can be classified as a general-purpose technology remains open to debate and we find more arguments to support the contention that AI is better considered, for example, as a large technical system with infrastructural properties (Vannuccini and Prytkova, 2021).

### 5.3. Robustness analysis

Our results are robust across a wide range of additional tests. Tables and further material can be found in Appendix A and Supplementary material.

First, we excluded all articles that fall into the WoS ‘Neurosciences’ category. This domain can be potentially problematic in that some terms (neural network, first and foremost) may not necessarily refer to artificial intelligence but rather to human intelligence and the biological brain. The sample falls by about 30 % and the number of NN articles almost halves. However, our results are consistent when replicating the analysis on the sub-sample.

Second, we excluded all articles that contain the terms ‘neural network’ and ‘neural networks’ exclusively in their title, keywords, or abstract. Bear in mind that an article may still contain a term such as ‘artificial neural network’ or ‘convolutional neural network’ which should now refer to artificial intelligence stricto sensu. In this case, neuroscience papers may form part of the sample. This restriction is severe insofar as the number of NN articles falls by more than 70 %. Yet our results are robust to this constraint.

The third exercise consists of a different econometric approach. Instead of regression analysis, we compared each NN paper with a ‘twin’ non-NN paper. More precisely, the empirical strategy considers the adoption of NN as a ‘treatment’; hence, we employ exact matching and 1:1 nearest neighbour matching on propensity scores (PSM) to select an appropriate control group of untreated papers. Exact matching is performed considering Web of Science categories, publication year, and journal – that is, we compare a NN article in terms of novelty and impact with an article belonging to the same domain(s), published in the same
<table><caption>**Table 4**

Impact profile of NN publications.</caption><thead><tr><th></th><th></th><th>NegBin: # Citations<br>(1)</th><th>Probit: Top 5 % Cited<br>(2)</th><th>Probit: Top 10 % Cited<br>(3)</th></tr></thead><tbody><tr><td rowspan="10">Panel A: Mean</td><td>NN</td><td>0.101**<br>(0.040)</td><td>0.147***<br>(0.041)</td><td>0.155***<br>(0.043)</td></tr><tr><td>Novelty (All Sciences)</td><td>0.153***<br>(0.023)</td><td>0.200***<br>(0.016)</td><td>0.191***<br>(0.015)</td></tr><tr><td># References (log)</td><td>0.491***<br>(0.064)</td><td>0.429***<br>(0.075)</td><td>0.477***<br>(0.062)</td></tr><tr><td># Authors (log)</td><td>0.237***<br>(0.026)</td><td>0.166***<br>(0.039)</td><td>0.194***<br>(0.036)</td></tr><tr><td>International Collab.</td><td>0.064***<br>(0.013)</td><td>0.083***<br>(0.014)</td><td>0.085***<br>(0.013)</td></tr><tr><td>Private Partic.</td><td>-0.029*<br>(0.015)</td><td>-0.027<br>(0.018)</td><td>-0.034**<br>(0.015)</td></tr><tr><td>JIF</td><td>0.205***<br>(0.022)</td><td>0.167***<br>(0.017)</td><td>0.179***<br>(0.018)</td></tr><tr><td>Journal Age (log)</td><td>0.050<br>(0.036)</td><td>-0.066<br>(0.086)</td><td>-0.048<br>(0.079)</td></tr><tr><td>Survey</td><td>0.541***<br>(0.060)</td><td>0.667***<br>(0.054)</td><td>0.627***<br>(0.049)</td></tr><tr><td rowspan="6">Panel B: Dispersion</td><td>NN</td><td>0.136***<br>(0.051)</td><td></td><td></td></tr><tr><td>Novelty (All Sciences)</td><td>0.093***<br>(0.017)</td><td></td><td></td></tr><tr><td># References (log)</td><td>-0.496***<br>(0.038)</td><td></td><td></td></tr><tr><td># Authors (log)</td><td>-0.213***<br>(0.044)</td><td></td><td></td></tr><tr><td>JIF</td><td>0.040<br>(0.031)</td><td></td><td></td></tr><tr><td>Journal Age (log)</td><td>-0.118***<br>(0.029)</td><td></td><td></td></tr><tr><td colspan="2">Log likelihood</td><td>-1,519,720</td><td>-69,222</td><td>-110,788</td></tr><tr><td colspan="2">χ² [null model]</td><td>318,463***</td><td>19,317***</td><td>31,564***</td></tr><tr><td colspan="2">χ² [w/o NN model]</td><td>8.70***</td><td>24.80***</td><td>40.00***</td></tr><tr><td colspan="2"># obs</td><td>356,037</td><td>356,037</td><td>356,037</td></tr></tbody></table>

Notes: This table reports coefficients of the effect of NN methods (NN, dummy) on scientific impact proxied by the number of citations received (Column 1) and ‘big hits’ (Columns 2 and 3). Bootstrapped (500 replications) standard errors clustered at the journal-level in parentheses: ***, ** and * indicate significance at the 1%, 5% and 10% level, respectively. The effect of NN on the citation count is estimated using a negative binomial regression. Estimates for the expectation and variance are reported in Panels A and B, respectively. Effects on the binary indicators are estimated using a Probit. Constant term, scientific field (WoS subject category) and time fixed effects are incorporated in all model specifications. Likelihood-ratio tests are used to compare the goodness-of-fit of two statistical models: (i) null model against complete model; (ii) model without the NN variable against the complete model.

year and in the same journal. We obtain the propensity scores associated with the binary treatment via the estimation of the Probit model containing the original set of variables. The average treatment effects (ATT) for the selected variables lend further support to our results.

A final test concerns the way novelty is measured. Indeed, some research shows that different novelty indicators are often inconsistent with each other and may return different sets of novel contributions (Fontana et al., 2020). Thus, we implemented the indicator developed in Uzzi et al. (2013) to define an ‘atypical’ (novelty/conventionality) quadrant: high-conventionality/high-novelty (HC-HN); high-conventionality/low-novelty (HC-LN); low-conventionality/high-novelty (LC-HN); and low-conventionality/low-novelty (LC-LN). The four categories are employed in a multinomial logistic regression. We find that, within the knowledge landscape of the health sciences, NN articles are more likely to draw on highly conventional combinations of knowledge. Ceteris paribus, our estimates suggest that when NN methods inject some highly (field-specific) unusual combinations, they do so primarily in an exceptionally conventional knowledge space.

## 6. Concluding remarks

Most socio-economic analyses of AI have looked at the effects of technology on economic growth (Brynjolfsson and McAfee, 2014; Aghion et al., 2017), labour market and productivity dynamics (Furman and Seamans, 2019; Acemoglu and Restrepo, 2020; Van Roy et al., 2020), changes in skills (Graetz and Michaels, 2018; Brynjolfsson and

Mitchell, 2017), and inequality and discrimination (O'Neil, 2016; Zuboff, 2019). Our contribution, here, provides insights into the diffusion and impact of AI methods in the scientific system.

In this paper, we first examined the diffusion of NN research in the sciences in an effort to verify whether NNs conform to certain characteristics of emerging technologies. We found that NN research activity has grown exponentially in almost all sciences and all over the world, and the diffusion process has followed a double-boom cycle with a strong re-configuration of global actors. The diffusion of NN methods into application domains began in a cross-disciplinary fashion involving the computer sciences, breaking their way into ‘pure’ field-specific research within the various application domains. We then examined the impact of technology adoption on scientific discovery, with a particular focus on the health sciences. We found the adoption of NN methods to be negatively correlated with recombinatorial novelty; however, a positive correlation was found with the expectation and dispersion of citations received, increasing a contribution’s likelihood of becoming a ‘big hit’.

Conceptually, we considered scientific discovery to be a recombinatorial process in which existing knowledge is recombined to create new knowledge, a process that continues perpetually in a dynamic knowledge landscape. A traditional image of science is one in which the knowledge landscape is made up of islands – i.e., (sub)-disciplines or scientific fields – where most of this recombination takes place. The islands reflect the structure of nature but also the need for a scientific mind to organize the complexity of the world. Seen this way, scientists
are sailors whose goal is to navigate from island to island, figure out their structure, and explore the surrounding landscape. Sailors can opt to stay in the ‘comfort zone’ and further their knowledge of one (or neighbouring) island(s), or they can sail to more distant islands and connect new areas of the landscape. Both actions enrich the knowledge space, one exploring well-formalized knowledge structures, the other reshaping and rearranging the landscape. Our findings suggest that, at least as it is used today, AI – the boat or the compass, to stick with the analogy – seems to be more in line with the first action. However, the possibilities of discovering new and valuable things about the known islands are far from obvious, as confirmed by our results on scientific impact.

A general-purpose invention in the method of invention? Or a passing fad in science? We think not. Our findings lead us to take up a more moderate stance in the recent debate on how AI affects the development of knowledge. NN methods do not (yet) serve as an autopilot for navigating the sea of knowledge and connecting ideas, but they are, nevertheless, an extremely powerful and versatile research tool that impacts knowledge creation in measurable ways. Thus, we propose that AI should be considered an *emerging general method of invention*. But do not be fooled, we are not simply seeking to win the race to coin the most attractive designation; rather, as we discuss below, thinking of this technology as ‘general’ and ‘emerging’ has policy implications that differ substantially from those that might result from thinking of it as a general-purpose technology (for more on the latter, see, e.g., *Trajtenberg, 2018; Klinger et al., 2021*).

First, the diffusion of intelligent machines as input in the research production process calls into question the organization and management of science. AI may trigger a short-term substitution towards capital and away from highly skilled labour in the knowledge production process. Whether such a substitution effect is occurring is doubtful and clearly requires further empirical investigation. In parallel, the arrival of automation technologies in science puts a wide range of research tasks under threat, either by reducing the cost of performing those tasks or by outperforming human scientists in the performance of them. Some tasks within the occupation may be suitable for automation while others may not, and the overall effects on employment in science are very complex. Therefore, research-oriented organizations need a better understanding of the set of tasks performed by their scientists, the coordination of these tasks, and the respective strengths and weaknesses of humans (H) and machines (M), before they can hope to unleash the benefits of H + M cooperation.

Machines are set to become more than tools; they have the potential to become another teammate. As such, H–M interactions will require the coordination of complex activities, including communication, joint actions and human-aware execution. As these machine teammates will operate in different collaborative environments, they need to be designed with different collaborative capabilities. This design area will require considering such aspects as appearance (what machines should look like); learning and knowledge processing (how they should learn); conversation (how they should interact and socialize with their peers); architecture (what their main components should be); reliability, responsibility and liability. (For a more in-depth discussion on design areas for human-machine collaboration, see *Seeber et al., 2020*).

It seems that NN methods are being adopted in different scientific fields but that existing knowledge structures are remaining relatively stable. This suggests the full potential of the technology (and its future development) might be better achieved by further spanning the boundaries between scientific areas. The bringing together of expertise and knowledge from various domains could help in the identification of blind spots and opportunities in the knowledge landscape. The concepts of ‘knowledge communities’ and ‘communities of practice’ seem particularly apt in this context. Although communities often self-organize and self-sustain themselves, they can also benefit from policy endorsement. It seems crucial to us that institutions and a policy environment be developed that are conducive to enhancing dialogue and

cross-fertilization between communities. This could be achieved, for instance, by reinforcing both horizontal (intra-field) and vertical (inter-field) knowledge management. Digital platforms and knowledge hubs could be complemented by physical ‘collaborative spaces’ where the tacit knowledge of different communities might be transferred face-to-face, documented and made accessible for later use. Another standard instrument is obviously research funding, which should not target individual areas but rather research ‘priorities’ (e.g., fighting a given disease) involving different communities that can frame their research questions together.

However, promoting collaboration between communities can pose certain challenges in terms of governance and data ownership. Data is a polymorphous category, which means standards, principles and rules governing the various types of data are not homogeneous across communities, let alone across countries. This opens up the question of how data should be generated/used in compliance with different regulations, and also how the value of data should be distributed (Savona, 2019).

The diffusion of AI, as a research instrument, can be self-sustaining only if there is social acceptance – i.e., if the crew trusts the captain and the equipment. Several AI applications represent innovations that can bring about far-reaching changes in all aspects of our daily lives. These social innovations can have unintended yet negative consequences in terms of security, privacy and social equity (O'Neil, 2016). The public will no longer tolerate being excluded from the debate and it is here that the scientific and policy community have a key role to play. Both parties can improve the channelling of scientific evidence into the public arena and fight the risks posed by fake news. Policy can promote communication by setting the right, often intrinsic, incentives to encourage as many scientists as possible to engage with different segments of the public. However, communicating science to non-scientific audiences can be difficult since it requires a different approach from that of communicating science to scientific audiences. This means scientists need to be able to detach the layers of scientific complexity that characterize their research so as to deliver a clear message to the public, a message, moreover, that should include both potential impacts and ethical issues. ‘Listening mechanisms’ can also be used to inform citizens' knowledge, expectations, and imaginaries about intelligent machines and, why not, about their role in science. There are a variety of means available for achieving these goals, ranging from in-depth interviews and material deliberations to citizen science. We believe that citizen science has the potential to bring the greatest benefits to both the public and the scientific system. The nonprofessional involvement of volunteers in the scientific process, whether in more mundane tasks such as data collection or in other phases of the research, offers great opportunities for the public to become familiar with the technology but also provides researchers with great opportunities to improve their results (Bonney et al., 2014; Sullivan et al., 2018). However, fully accountable institutional mechanisms are a precondition for guaranteeing trust between scientists and the public and for ensuring continuity in their relationship. For instance, all results and the process used in reaching these results should be open to scrutiny. Policy should promote feedback activities so as to maintain citizen involvement and explain how their inputs were used in meeting research aims; reconcile conflicting values and objectives; and, put in place collective intelligence mechanisms that can help them develop a systemic understanding of the future implications of technological progress and make better consensus decision-making – all very much in line with the notion of ‘Decisions 2.0’ (Bonabeau, 2009). Finally, we fully embrace the concept of ‘boundary organizations’ specifically designed to deal with socioeconomic transformations in the digital age. These organizations would sit at the intersection of scientific and political spheres and allow scientists and policy-makers to maintain a constant dialogue with each other.

Although the AI revolution has been the subject under scrutiny here, ironically this revolution offers the tools with the greatest potential for bringing about a radical transformation in the interactions between the
public, the scientific community and the policy environment. These interactions, if exploited carefully, should serve to give a boost to human efforts to better understand the greatest mystery of all: the origin and function of the world and our place in it, that is, the tasks of science itself.

## Declaration of competing interest

No conflict of interest to declare.

## Acknowledgement

Earlier versions of this paper were presented at the INNOVA MEA-SURE III Expert Workshop 'Brainstorming in Ispra'; 7th European

Conference on Corporate R&D and Innovation, Seville (Spain); EMAEE 2019 'Economics, Governance and Management of AI, Robots and Digital Transformations', Brighton (UK); OLKC 2019 'The Human Side of Innovation – Understanding the Role of Interpersonal Relations in an Increasingly Digitised Workplace'; CAGE-NESTA Workshop on 'Data Science for the Economics of Science, Technology and Innovation', London (UK); and the Workshop Series on 'The Economics and Management of AI Technologies', Copenhagen (Denmark) and Strasbourg (France). The authors thank seminar participants and in particular Tommaso Ciarli, Giacomo Damioli, Mirko Draca, Daniel S. Hain, Björn Jindra, Bertrand Koebel, Roman Jurowetzki, Patrick Llerena, Juan Mateos-Garcia, Maria Savona, Simone Vannuccini, Daniel Vertesy, and Marco Vivarelli. We also are very grateful to two anonymous referees for their constructive criticism.

## Appendix A

<table><caption>**Table 5**

Descriptive statistics of the variables.</caption><thead><tr><th></th><th>NN papers</th><th>Non-NN papers</th><th>Total</th></tr></thead><tbody><tr><td>Re-combinatorial novelty</td><td></td><td></td><td></td></tr><tr><td>Novelty Dummy (All Sciences)</td><td>36.43</td><td>30.32</td><td>30.40</td></tr><tr><td>Novelty Dummy (No CS)</td><td>32.39</td><td>29.55</td><td>29.59</td></tr><tr><td>Novelty Dummy (Only HS)</td><td>20.96</td><td>23.52</td><td>23.49</td></tr><tr><td>Novelty (All Sciences)</td><td>0/0.81 (2.39)</td><td>0/0.74 (3.10)</td><td>0/0.74 (3.09)</td></tr><tr><td>Novelty (No CS)</td><td>0/0.65 (2.12)</td><td>0/0.71 (3.07)</td><td>0/0.71 (3.06)</td></tr><tr><td>Novelty (Only HS)</td><td>0/0.37 (1.62)</td><td>0/0.50 (2.40)</td><td>0/0.5 (2.39)</td></tr><tr><td>Scientific impact</td><td></td><td></td><td></td></tr><tr><td>Top 5 % Cited</td><td>8.33</td><td>5.77</td><td>5.80</td></tr><tr><td>Top 10 % Cited</td><td>15.68</td><td>11.38</td><td>11.43</td></tr><tr><td># Citations (raw count)</td><td>17/38.34 (114.43)</td><td>18/35.48 (82.67)</td><td>18/35.51 (83.15)</td></tr><tr><td>Citations (yearly normalized)</td><td>2.06/4.06 (8.16)</td><td>2.08/3.75 (8.02)</td><td>2.08/3.75 (8.02)</td></tr><tr><td>Controls</td><td></td><td></td><td></td></tr><tr><td># References</td><td>40/45.92 (29.59)</td><td>33/37.12 (25.66)</td><td>33/37.23 (25.73)</td></tr><tr><td># Authors</td><td>4/4.07 (2.37)</td><td>4/4.90 (3.50)</td><td>4/4.89 (3.49)</td></tr><tr><td>International Collab.</td><td>26.21</td><td>23.02</td><td>23.06</td></tr><tr><td>Private Partic.</td><td>6.80</td><td>7.09</td><td>7.09</td></tr><tr><td>JIF</td><td>1.39/2.12 (2.06)</td><td>1.73/2.42 (2.13)</td><td>1.73/2.41 (2.13)</td></tr><tr><td>Journal Age</td><td>22/28.57 (26.07)</td><td>33/38.47 (29.08)</td><td>32/38.35 (29.06)</td></tr><tr><td>Survey</td><td>0.72</td><td>0.78</td><td>0.77</td></tr><tr><td>Time period</td><td>[2001–2015]</td><td>[2001–2015]</td><td>[2001–2015]</td></tr><tr><td># scientific fields</td><td>46</td><td>48</td><td>48</td></tr><tr><td># journals</td><td>92</td><td>92</td><td>92</td></tr><tr><td># papers</td><td>4560 (1.28 %)</td><td>351,477 (98.72 %)</td><td>356,037 (100 %)</td></tr></tbody></table>

Notes: Binary indicators in [%], for continuous measures [median/mean (s.d.)]. The statistics refer to the period used for the econometric analysis.

<table><caption>**Table 6**

Novelty and impact profile – Matching.</caption><thead><tr><th rowspan="2"></th><th colspan="2">Exact matching</th><th colspan="2">Propensity score matching</th></tr><tr><th>(1)</th><th>(2)</th><th>(3)</th><th>(4)</th></tr></thead><tbody><tr><td>Novelty (All Sciences)</td><td>0.054***</td><td>0.053***</td><td>0.035***</td><td>0.023*</td></tr><tr><td>Novelty (No CS)</td><td>0.026**</td><td>0.026**</td><td>0.008</td><td>-0.001</td></tr><tr><td>Novelty (Only HS)</td><td>-0.005</td><td>-0.005</td><td>-0.025**</td><td>-0.033***</td></tr><tr><td># Citations</td><td>0.192***</td><td>0.195***</td><td>0.102***</td><td>0.063**</td></tr></tbody></table>

Notes: This table reports Average Treatment Effect on the Treated (ATT) for novelty and impact variables. The set of variables used for each matching is composed as follows: (1) Journal/WoS categories/publication year; (2) all dummy variables in our set of control variables/journal/WoS categories/publication year; (3) number of authors (log)/number of references (log)/journal/WoS categories/publication year; (4) all variables. ***, ** and * indicate significance at the 1 %, 5 % and 10 % levels, respectively.
<table><caption>**Table 7**
Atypical profile of NN publications.</caption><thead><tr><th rowspan="2">Category</th><th>All Sciences</th><th>No CS</th><th>Only HS</th></tr><tr><th>(1)</th><th>(2)</th><th>(3)</th></tr></thead><tbody><tr><td>HC-HN</td><td>0.008<br>(0.130)</td><td>0.208<br>(0.133)</td><td>0.308**<br>(0.136)</td></tr><tr><td>HC-LN</td><td>-0.041<br>(0.157)</td><td>0.090<br>(0.152)</td><td>-0.049<br>(0.154)</td></tr><tr><td>LC-LN</td><td>-0.043<br>(0.162)</td><td>-0.086<br>(0.163)</td><td>0.021<br>(0.155)</td></tr><tr><td>Other variables</td><td>Yes</td><td>Yes</td><td>Yes</td></tr><tr><td>Log likelihood</td><td>-374,002</td><td>-374,000</td><td>-363,855</td></tr><tr><td>χ² [null model]</td><td>95,913***</td><td>95,488***</td><td>115,891***</td></tr><tr><td>χ² [w/o NN model]</td><td>259***</td><td>158.20***</td><td>144***</td></tr><tr><td># obs</td><td>320,587</td><td>320,587</td><td>320,587</td></tr></tbody></table>

Notes: This table reports coefficients of the effect of NN methods (NN, dummy) on atypical profiles. Category LC-HN is the reference category for all models. Bootstrapped (500 replications) standard errors clustered at the journal-level in parentheses: ***, ** and * indicate significance at the 1 %, 5 % and 10 % levels, respectively. All variables are incorporated in all model specifications, details in Supplementary material. Likelihood-ratio tests are used to compare the goodness-of-fit of two statistical models: (i) null model against complete model; (ii) model without the NN variable against the complete model.

## Appendix B. Supplementary data

Supplementary data to this article can be found online at https://doi.org/10.1016/j.respol.2022.104604.

## References

Acemoglu, D., Restrepo, P., 2020. Robots and jobs: Evidence from US labor markets. J. Polit. Econ. 128 (6), 2188-2244.

Adner, R., Levinthal, D.A., 2002. The emergence of emerging technologies. Calif. Manag. Rev. 45 (1), 50-66.

Aghion, P., Jones, B.F., Jones, C.I., 2017. Artificial Intelligence and Economic Growth (No. w23928). National Bureau of Economic Research.

Agrawal, A., McHale, J., Oettl, A., 2018. Finding Needles in Haystacks: Artificial Intelligence and Recombinant Growth (No. w24541). National Bureau of Economic Research.

Angermueller, C., PaPärnamaa, T., Parts, L., Stegle, O., 2016. Deep learning for computational biology. Mol. Syst. Biol. 12 (7).

Arthur, W.B., 2009. The Nature of Technology: What it is and How it Evolves. Simon and Schuster.

Azoulay, P., Graff Zivin, J.S., Manso, G., 2011. Incentives and creativity: evidence from the academic life sciences. RAND J. Econ. 42 (3), 527-554.

Baradaran, R., Ghiasi, R., Amirkhani, H., 2020. A Survey on Machine Reading Comprehension Systems. arXiv preprint arXiv:2001.01582.

Berndt, D.J., Clifford, J., 1994. Using dynamic time warping to find patterns in time series. In: KDD Workshop, Vol. 10, pp. 359-370. No. 16.

Bloom, N., Jones, C.I., Van Reenen, J., Webb, M., 2020. Are ideas getting harder to find? Am. Econ. Rev. 110 (4), 1104-1144.

Boden, M.A., 2004. The Creative Mind: Myths and Mechanisms. Routledge.

Boden, M.A., 2009. Computer models of creativity. AI Mag. 30 (3), 23-23.

Bonabeau, E., 2009. Decisions 2.0: the power of collective intelligence. MIT Sloan Manag. Rev. 50 (2), 45.

Bonney, R., Shirk, J.L., Phillips, T.B., Wiggins, A., Ballard, H.L., Miller-Rushing, A.J., Parrish, J.K., 2014. Next steps for citizen science. Science 343 (6178), 1436-1437.

Bourdieu, P., 1975. The specificity of the scientific field and the social conditions of the progress of reason. Soc. Sci. Inf. 14 (6), 19-47.

Bresnahan, T.F., Trajtenberg, M., 1995. General purpose technologies 'Engines of growth'? J. Econ. 65 (1), 83-108.

Brynjolfsson, E., McAfee, A., 2014. The Second Machine Age: Work, Progress, and Prosperity in a Time of Brilliant Technologies. WW Norton & Company.

Brynjolfsson, E., Mitchell, T., 2017. What can machine learning do? Workforce implications. Science 358 (6370), 1530-1534.

Cockburn, I.M., Henderson, R., Stern, S., 2018. The Impact of Artificial Intelligence on Innovation (No. w24449). National Bureau of Economic Research.

Daugherty, P.R., Wilson, H.J., 2018. Human + Machine: Reimagining Work in the Age of AI. Harvard Business Press.

Fleming, L., 2001. Recombinant uncertainty in technological search. Manag. Sci. 47 (1), 117-132.

Fleming, L., Mingo, S., Chen, D., 2007. Collaborative brokerage, generative creativity, and creative success. Adm. Sci. Q. 52 (3), 443-475.

Fontana, M., Iori, M., Montobbio, F., Sinatra, R., 2020. New and atypical combinations: an assessment of novelty and interdisciplinarity. Res. Policy 49 (7), 104063.

Foster, J.G., Rzhetsky, A., Evans, J.A., 2015. Tradition and innovation in scientists' research strategies. Am. Sociol. Rev. 80 (5), 875-908.

Furman, J., Seamans, R., 2019. AI and the economy. Innov. Policy Econ. 19 (1), 161-191.

Furman, J.L., Teodoridis, F., 2020. Automation, research technology, and researchers' trajectories: evidence from computer science and electrical engineering. Organ. Sci. 31 (2), 330-354.

Glänzel, W., Schubert, A., 2001. Double effort = double impact? A critical view at international co-authorship in chemistry. Scientometrics 50 (2), 199-214.

Goodfellow, I., Bengio, Y., Courville, A., 2016. Deep Learning. MIT press.

Gordon, R.J., 2016. The Rise and Fall of American Growth. Princeton University Press.

Graetz, G., Michaels, G., 2018. Robots at work. Rev. Econ. Stat. 100 (5), 753-768.

Griliches, Z., 1957. Hybrid corn: an exploration in the economics of technological change. Econometrica 501-522.

Hassabis, D., Kumaran, D., Summerfield, C., Botvinick, M., 2017. Neuroscience-inspired artificial intelligence. Neuron 95 (2), 245-258.

Hey, A.J., Tansley, S., Tolle, K.M., 2009. The Fourth Paradigm: Data-intensive Scientific Discovery, Vol. 1. Microsoft research, Redmond, WA.

Jones, B.F., 2009. The burden of knowledge and the “death of the renaissance man”: is innovation getting harder? Rev. Econ. Stud. 76 (1), 283-317.

King, R.D., Rowland, J., Oliver, S.G., Young, M., Aubrey, W., Byrne, E., Sparkes, A., 2009. The automation of science. Science 324 (5923), 85-89.

Klinger, J., Mateos-Garcia, J., Stathoulopoulos, K., 2021. Deep learning, deep change? Mapping the evolution and geography of a general purpose technology. Scientometrics 1-33.

Kuhn, T., 1962. The Structure of Scientific Revolution. University of Chicago, Chicago, USA.

Lake, B.M., Ullman, T.D., Tenenbaum, J.B., Gershman, S.J., 2017. Building machines that learn and think like people. Behav. Brain Sci. 40.

Lanier, J., 2010. You are not a Gadget: A Manifesto. Vintage.

LeCun, Y., Bengio, Y., Hinton, G., 2015. Deep learning. Nature 521 (7553), 436-444.

Lee, Y.N., Walsh, J.P., Wang, J., 2015. Creativity in scientific teams: unpacking novelty and impact. Res. Policy 44 (3), 684-697.

Leung, M.K., Delong, A., Alipanahi, B., Frey, B.J., 2015. Machine learning in genomic medicine: a review of computational problems and data sets. Proc. IEEE 104 (1), 176-197.

Litjens, G., Kooi, T., Bejnordi, B.E., Setio, A.A.A., Ciompi, F., Ghafoorian, M., Sánchez, C. I., 2017. A survey on deep learning in medical image analysis. Med. Image Anal. 42, 60-88.

Ma, J., Sheridan, R.P., Liaw, A., Dahl, G.E., Svetnik, V., 2015. Deep neural nets as a method for quantitative structure-activity relationships. J. Chem. Inf. Model. 55 (2), 263-274.

Marblestone, A.H., Wayne, G., Kording, K.P., 2016. Toward an integration of deep learning and neuroscience. Front. Comput. Neurosci. 10, 94.

Marx, V., 2013. Biology: the big challenges of big data. Nature 498, 255-260.

Merton, R.K., 1957. Priorities in scientific discovery: a chapter in the sociology of science. Am. Sociol. Rev. 22 (6), 635-659.

Mikolov, T., Chen, K., Corrado, G., Dean, J., 2013a. Efficient Estimation of Word Representations in Vector Space. arXiv preprint arXiv:1301.3781.

Mikolov, T., Sutskever, I., Chen, K., Corrado, G.S., Dean, J., 2013b. Distributed representations of words and phrases and their compositionality. In: Advances in Neural Information Processing Systems, pp. 3111-3119.

Miotto, R., Wang, F., Wang, S., Jiang, X., Dudley, J.T., 2018. Deep learning for healthcare: review, opportunities and challenges. Brief. Bioinform. 19 (6), 1236-1246.
Nature Editorial, 2019. The scientific events that shaped the decade. Nature 576 (2019), 337-338.

Nilsson, N.J., 2009. The Quest for Artificial Intelligence. Cambridge University Press.

O'Neil, C., 2016. Weapons of Math Destruction: How Big Data Increases Inequality and Threatens Democracy. Broadway Books.

Olier, I., Orhobor, O.I., Dash, T., Davis, A.M., Soldatova, L.N., Vanschoren, J., King, R.D., 2021. Transformational machine learning: learning how to learn from many related scientific problems. Proc. Natl. Acad. Sci. 118 (49).

Raghu, M., Schmidt, E., 2020. A Survey of Deep Learning for Scientific Discovery. arXiv preprint arXiv:2003.11755.

Raghupathi, W., Raghupathi, V., 2014. Big data analytics in healthcare: promise and potential. Health Inf. Sci. Syst. 2 (1), 3.

Rajkomar, A., Oren, E., Chen, K., Dai, A.M., Hajaj, N., Hardt, M., Sundberg, P., 2018. Scalable and accurate deep learning with electronic health records. NPJ Digit. Med. 1 (1), 18.

Ravi, D., Wong, C., Deligianni, F., Berthelot, M., Andreu-Perez, J., Lo, B., Yang, G.Z., 2017. Deep learning for health informatics. IEEE J. Biomed. Health Inform. 21 (1), 4-21.

Rosenberg, N., 1992. Scientific instrumentation and university research. Res. Policy 21 (4), 381-390.

Rotolo, D., Hicks, D., Martin, B.R., 2015. What is an emerging technology? Res. Policy 44 (10), 18271843.

Russell, S., Norvig, P., 2021. In: Artificial Intelligence: A Modern Approach, Global Edition 4th. Foundations, 19, p. 23.

Savadjiev, P., Chong, J., Dohan, A., Vakalopoulou, M., Reinhold, C., Paragios, N., Gallix, B., 2019. Demystification of AI-driven medical image interpretation: past, present and future. Eur. Radiol. 29 (3), 1616-1624.

Savona, M., 2019. The value of data: Towards a framework to redistribute it. In: SPRU Working Paper Series 2019-21.

Schmidhuber, J., 2015. Deep learning in neural networks: an overview. Neural Netw. 61, 85-117.

Schmoch, U., 2007. Double-boom cycles and the comeback of science-push and market-pull. Res. Policy 36 (7), 1000-1015.

Seeber, I., Bittner, E., Briggs, R.O., De Vreede, T., De Vreede, G.J., Elkins, A., Sollner, M., 2020. Machines as teammates: a research agenda on AI in team collaboration. Inf. Manag. 57 (2), 103174.

Shen, D., Wu, G., Suk, H.I., 2017. Deep learning in medical image analysis. Annu. Rev. Biomed. Eng. 19, 221-248.

de Solla Price, D.J., 1963. Little Science, Big Science. Columbia University Press, New York.

Sullivan, D.P., Winsnes, C.F., Akesson, L., Hjelmare, M., Wiking, M., Schutten, R., Smith, K., 2018. Deep learning is combined with massive-scale citizen science to improve large-scale image classification. Nat. Biotechnol. 36 (9), 820-828.

Sutton, C., Gong, L., 2017. Popularity of arXiv.org Within Computer Science. arXiv preprint arXiv:1710.05225.

Trajtenberg, M., 2018. AI as the Next GPT: A Political-Economy Perspective (No. w24245). National Bureau of Economic Research.

Uzzi, B., Mukherjee, S., Stringer, M., Jones, B., 2013. Atypical combinations and scientific impact. Science 342 (6157), 468-472.

Van Roy, V., Vertesy, D., Damioli, G., 2020. AI and robotics innovation. In: Handbook of Labor, Human Resources and Population Economics, pp. 1-35.

Vannuccini, S., Prytkova, E., 2021. Artificial Intelligence's New Clothes? From General Purpose Technology to Large Technical System, SWPS 2021-02. No. 16.

Wadden, D., Lin, S., Lo, K., Wang, L.L., van Zuylen, M., Cohan, A., Hajishirzi, H., 2020. Fact or Fiction: Verifying Scientific Claims. arXiv preprint arXiv:2004.14974.

Wagner, C.S., Whetsell, T.A., Mukherjee, S., 2019. International research collaboration: novelty, conventionality, and atypicality in knowledge recombination. Res. Policy 48 (5), 1260-1270.

Wang, J., Veugelers, R., Stephan, P., 2017. Bias against novelty in science: a cautionary tale for users of bibliometric indicators. Res. Policy 46 (8), 1416-1436.

Weitzman, M., 1998. Recombinant growth. Q. J. Econ. 113 (2), 331-360.

Zhang, Q., Yang, L.T., Chen, Z., Li, P., 2018. A survey on deep learning for big data. Information Fusion 42, 146-157.

Zhang, S., Yao, L., Sun, A., Tay, Y., 2019. Deep learning based recommender system: a survey and new perspectives. ACM Comput. Surv. 52 (1), 1-38.

Zuboff, S., 2019. The age of surveillance capitalism: the fight for a human future at the new frontier of power. In: Profile Books.