Creation of a highly detailed, dynamic, global model and map of science

The majority of the effort in metrics research has addressed research evaluation. Far less research has been done to address the unique problems of research planning. Models and maps of science that can address the detailed problems associated with research planning are needed. This article reports on the creation of an article-level model and map of science covering 16 years and nearly 20 million articles using co-citation-based techniques. The map is then used to define disciplinelike structures consisting of natural groupings of articles and clusters of articles. This combination of detail and high-level structure can be used to address planning-related problems such as identification of emerging topics, and the identification of which areas of science and technology are innovative and which are simply persisting. In addition to presenting the model and map, several process improvements that result in higher accuracy structures are detailed, including a bibliographic coupling approach for assigning current papers to co-citation clusters, and a sequentially hybrid approach to producing visual maps from models. Introduction The majority of the effort in metrics (sciento-, biblio-, infor-, alt-) studies has been aimed at research evaluation (B. R. Martin, Nightingale, & Yegros-Yegros, 2012). Examples of evaluation-related topics include impact factors, the h-index and other related indices, university rankings and national level science indicators. The 40+ year history of the use of documentbased indicators for research evaluation includes publication of handbooks (cf., Moed, Glänzel, & Schmoch, 2004), the introduction of new journals (e.g., Scientometrics, Journal of Informetrics) and several annual or biannual conferences (e.g., ISSI, Collnet, STI/ENID) specifically aimed at reporting on these activities. The evaluation of research using scientific and technical documents is a well-established area of research. Far less effort in metrics research has been aimed at the unique problems of research planning. Planning-related questions (Börner, Boyack, Milojević, & Morris, 2012) that are asked by funders, administrators and researchers are different from evaluation-related questions. As such, they require different models than those that are used for evaluation. For example, planning requires a model that can predict emerging topics in science and technology. Funders need models that can help them identify the most innovative and promising proposals and researchers. Administrators, particularly those in industry, need models to help them best allocate their internal research funds, including knowing which existing areas to cut. To enable detailed planning, document-based models of science and technology need to be highly granular, and while based on retrospective data, must be robust enough to enable forecasting. Overall, the This is a preprint of an article accepted for publication in Journal of the American Society for Information Science and Technology copyright © 2013 (American Society for Information Science and Technology) 2 technical requirements of a model that can be used for planning are unique. Research and development of such models is an under-developed area in metrics research. To that end, this article reports on a co-citation-based model and map of science comprised of nearly 20 million articles over 16 years that has the potential to be used to answer planningrelated questions. Although the model is similar in concept to one that has been previously reported (Klavans & Boyack, 2011), it differs in several significant aspects: it uses an improved current paper assignment process, it uses an improved map layout process, and it has been used to create article-level discipline-like structures to provide context for its detailed structure. In the balance of the article, we first review related work to provide context for the work reported here. We then detail the experiments that led to the improvements in our map creation process. This is followed by introduction and characterization of the model and map, along with a description of how the map was divided into discipline-like groupings. The paper concludes with a brief discussion of how the map will be used in the future for a variety of analyses. Background Science mapping, when reduced to its most basic components, is a combination of classification and visualization. We assume there is a structure to science, and then we seek to create a representation of that structure by partitioning sets of documents (or journals, authors, grants, etc.) into different groups. This act of partitioning is the classification part of science mapping, and typically takes the majority of the effort. The resulting classification system, along with some representation of the relationships between the partitions, can be thought of as a model of science inasmuch as it specifies structure. The visualization part of science mapping uses the classification and relationships as input, and creates a visual representation (or map) of that model as output. Mapping of scientific structure using data on scientific publications began not long after the introduction of ISI’s citation indexes in the 1950s. Since then, science mapping has been done at a variety of scales and with a variety of data types. Science mapping studies have been roughly evenly split between document, journal, and author-based maps. Both text-based and citationbased methods have been used. Many of these different types of studies have been reviewed at intervals in the past (Börner, Chen, & Boyack, 2003; Morris & Martens, 2008; White & McCain, 1997). Each type of study is aimed at answering certain types of questions. For example, authorbased maps are most often used to characterize the major topics within an area of science, and to show the key researchers in those topic areas. Journal-based models and maps are often used to characterize discipline-level structures in science. Overlay maps based on journals can be used to answer high level policy questions (Rafols, Porter, & Leydesdorff, 2010). However, more detailed questions, such as questions related to planning, require the use of document-level models and maps of science. The balance of this section thus focuses on document-level models and maps. When it comes to mapping of document sets, most studies have been done using local datasets. The term ‘local’ is used here to denote a small set of topics or a small subset of the whole of science. While these local studies have successfully been used to improve mapping techniques, and to provide detailed information about the areas they study, we prefer global mapping This is a preprint of an article accepted for publication in Journal of the American Society for Information Science and Technology copyright © 2013 (American Society for Information Science and Technology) 3 because of the increased context and accuracy that are enabled by mapping of all of science (Klavans & Boyack, 2011). The context for work presented here lies in the efforts undertaken since the 1970s to map all of science at the document level using citation-based techniques. The first map of worldwide science based on documents was created by Griffith, Small, Stonehill & Dey (1974). Their map, based on co-citation analysis, contained 1,310 highly cited references in 115 clusters, showing the most highly cited areas in biomedicine, physics, and chemistry. Henry Small continued generating document-level maps using co-citation analysis (Small, Sweeney, & Greenlee, 1985), typically using thresholds based on fractional citation counting that ended up keeping roughly (but not strictly) the top 1% of highly cited references by discipline. The mapping process and software created by Small at the Institute for Scientific Information (ISI) evolved to generate hierarchically nested maps with four levels. Small (1999) presents a four level map based on nearly 130,000 highly cited references from papers published in 1995, which contained nearly 19,000 clusters at its lowest level. At roughly the same time, the Center for Research Planning (CRP) was creating similar maps for the private sector using similar thresholds and methods (Franklin & Johnston, 1988). One major difference is that CRP’s maps only used one level of clustering rather than multiple levels. The next major step in mapping all of science at the document level took place in the mid-2000’s when Klavans & Boyack (2006) created co-citation models of over 700,000 reference papers and bibliographic coupling models of over 700,000 current papers from the 2002 fileyear of the combined Science and Social Science Citation Indexes. Later, Boyack (2009) used bibliographic coupling to create a model and map of nearly 1,000,000 documents in 117,000 clusters from the 2003 citation indexes. Through 2004, the citation indexes from ISI were the only comprehensive data source that could be used for such maps. The introduction of the Scopus database in late 2004 provided another data source that could be used for comprehensive models and maps of science. Klavans & Boyack (2010) used Scopus data from 2007 to create a co-citation model of science comprised of over 2,000,000 reference papers assigned to 84,000 clusters. Over 5,600,000 citing papers from 2003-2007 were assigned to these co-citation clusters based on reference patterns. All of the models and maps mentioned to this point have been static maps – that is they were all created using data from a single year, and were snapshot pictures of science at a single point in time. It is only recently that researchers have created maps of all of science that are longitudinal in nature. In the first of these, Klavans & Boyack (2011) extended their co-citation mapping approach by linking together nine annual models of science to generate a nine-year global map of science comprised of 10,360,000 papers from 2000-2008. The clusters in this model have been found