Will This Idea Spread Beyond Academia? Understanding Knowledge Transfer of Scientific Concepts across Text Corpora (2020)

Abstract: What kind of basic research ideas are more likely to get applied in practice? There is a long line of research investigating patterns of knowledge transfer, but it generally focuses on documents as the unit of analysis and follow their transfer into practice for a specific scientific domain. Here we study translational research at the level of scientific concepts for all scientific fields. We do this through text mining and predictive modeling using three corpora: 38.6 million paper abstracts, 4 million patent documents, and 0.28 million clinical trials. We extract scientific concepts (i.e., phrases) from corpora as instantiations of “research ideas”, create concept-level features as motivated by literature, and then follow the trajectories of over 450,000 new concepts (emerged from 1995-2014) to identify factors that lead only a small proportion of these ideas to be used in inventions and drug trials. Results from our analysis suggest several mechanisms that distinguish which scientific concept will be adopted in practice, and which will not. We also demonstrate that our derived features can be used to explain and predict knowledge transfer with high accuracy. Our work provides greater understanding of knowledge transfer for researchers, practitioners, and government agencies interested in encouraging translational research.

## introduction science generates a myriad of new ideas, only some of which find value in practical uses @xcite . large government agencies (e.g., nsf, nih) pour billions of dollars into basic research in the hopes that it will span the research-practice divide so as to generate private sector advances in technologies (narin and * equal contribution noma, 1985), social policies @xcite , and pharmaceuticals @xcite . to this end, these agencies increasingly seek to nurture "translational research" that succeeds at extending, bridging and transforming basic research so it finds greater applied value @xcite . surrounding this effort has arisen a line of research that tries to identify when, where, and how academic research influences science and technological invention @xcite . however, prior research efforts are limited in their ability to understand and facilitate the translation of research ideas. this is partially due to a shortage of data, a biased focus on successful examples, and specialized modeling paradigms. in practice, only a small proportion of knowledge outputs are successfully translated into inventive outputs (∼ 2.7% concepts from wos to patent, and ∼ 11.3% concepts from wos to clinical trials, according to our data analysis). previous studies conduct post-hoc analyses of successful scientifictechnological linkages, but are unable to explain why the majority of scientific innovations do not transfer into technological inventions. additionally, prior work mostly look at document-level linkages across research and applied domains, i.e., citations from patents into research papers or shared inventors across them, rather than diving into the document content where ideas are discussed @xcite . documents entail many ideas, and linkages across them loosely capture which intellectual innovation is in focus and being transferred. by contrast, we conceptualize knowledge transfer in terms of scientific concepts, rather than documents associated to particular desirable outcomes, and demonstrate the importance of our derived features in knowledge transfer through machine learning model in a large-scale original dataset. in this paper, we focus on studying patterns behind knowledge transfer from academia to research. we use "knowledge transfer from academia to research" in our study to mean a "concept's transfer from research papers to patent documents/clinical trials", or a concept that first appear in academia later get used a non-trivial frequency (decided by a pre-defined threshold) in practical outlets (patents, clinical trials). 1 in scientific writing, a scientific concept is a term or set of terms that have semantically coherent usage and reflect scientific entities -e.g., curricula, tools, programs, ideas, theories, substances, methods, processes, and propositions, which are argued to be the basic units of scientific discovery and advance @xcite . we use the titles and abstracts of 38.6 million academic publications from the web of science (wos) to identify 0.45 million new scientific concepts emerging between 1995 to 2014 through state of the art phrase mining techniques (autophrase), and follow their trajectories in 4 million patent documents of the united states patent and trademark office (uspto), and 0.28 million clinical trials from u.s. national library of medicine. in our analysis, we compare the properties of 1 we use knowledge transfer, concept transfer and idea transfer interchangeably throughout the paper new scientific concepts that successfully transfer into patents with those that did not. we find that (a) the intrinsic properties of ideas and their temporal behavior, and (b) relative position of the ideas are the two mechanisms that determine whether an idea could transfer successfully. in particular, we find new engineering-focused scientific concepts situated in emotionally positive contexts are more likely to transfer than other concepts. furthermore, increased scientific hype and adoption across scientists, as well as usage in interdisciplinary venues over time, are early signs of impending knowledge transfer into technological inventions. finally, we find that new concepts positioned close to concepts that already transferred into patents are far more likely to transfer than their counterparts. based on the derived features, we further built model to predict the likelihood of knowledge transfer from papers to patents/clinical trials at individual concept level, and demonstrated our derived feature can achieve great performance, indicating that our proposed features can explain majority of the knowledge transfer cases. contributions our main contributions are summarized as follows: (1) to the best of our knowledge, we present the first ever research that aims at understanding knowledge transfer at a large scale, using multiple corpora. (2) we are the first to leverage text mining techniques to understand transfer on scientific concept level, rather than document level. ## data preparation and processing in this section we introduce the dataset used in our study (sec. 2.1), and present the concept extraction process (sec. 2.2). ## feature creation and analysis based on the concepts extracted from research papers, patent and clinical trial documents, we first create concept level features as motivated by prior literature on knowledge diffusion, and present a large-scale data analysis on transferred and nontransferred concepts to better understand properties facilitating the knowledge transfer process. here we present transfer patterns from research paper to patent and omit clinical trial due to page limit foot_1 . ## predictive analysis of features so far we have systematically analyzed the potential factors that reflect the process of knowledge transfer from research to practice. but how well can these features explain and predict knowledge transfer in practice? in this section, we seek to shed light on this question through predictive analysis. ## experiments experiment set-up. we apply z-score normalization on time-series attributes and divide dataset into training/test sets as fig. 9 shows. given test cutoff year t, we first ensure the prediction intervals (red line in fig. 9 ) of training and test set have no overlap to avoid data leakage, and then use the latest three cutoff years as train cutoff years. for @xcite as training cutoff years. as concept transfer status is irreversible, we exclude all transferred concepts from test set but still use them to train. details of model training and hyperparameter settings are discussed in appendix j. here we primarily report experimental results on knowledge transfer prediction from wos to uspto, while using clinical trial as a robustness check. evaluation metric. we adopt area-under-curve (auc) as evaluation metric, which is not affected by data imbalance in test set. ## conclusion in this paper, we systematically studied the process and properties of knowledge transfer from research to practice. specifically, we used a sample of 38.6 million research papers, 4 million patents and 280 thousand clinical trials, where we leveraged au-tophrase to extract concepts from text and focus on the applicable career of nearly 450,000 new scientific concepts that emerged from 1995-2014. through extensive analysis, we found that 'transferable' ideas distinguish themselves from 'nontransferable' ideas by their (a) intrinsic properties and their temporal behavior, and (b) their relative position to other concepts. through predictive analysis, we showed our proposed features can explain majority of transfer cases. our research not only provides significant implications for researchers, practitioners, and government agencies as a whole, but also introduces a novel research question of real world impact for computer scientists.