Research impact analysis of international funding agencies in the realm of allergy and immunology
Authors: Takeya Adachi, Yasushi Ogawa, Tamami Fukushi, Kei Ito, Amane Koizumi, Masashi Shirabe, Masako Toriya, Jun Hirako, Takenori Inomata, Katsunori Masaki, Ryohei Sasano, Sakura Sato, Keigo Kainuma, Masaki Futamura, Keiko Kan‐o, Yosuke Kurashima, Saeko Nakajima, Masafumi Sakashita, Hideaki Morita, Aikichi Iwamoto, Sankei Nishima, Mayumi Tamari, Hajime Iizuka
Journal: Allergy (2022)

A longitudinal approach should be employed for research and development (R&D) on allergic and immunological diseases across all life stages. To strategically use limited public funds in promoting such R&D, their characteristics of long-term research support and societal implementation should be considered.1 However, outcomes of the funding research evaluation have focused on conventional, shortsighted indicators. To determine the kind of indicators needed for the funding strategy, we compared the research impact of funding agencies (FAs) in the UK, US, and Japan, utilizing indices related to research substantiality2 and analyzing index words/abstracts connected with the national strategy for allergy and immunology.3 We used AMEDfind—an open database of top-down R&D projects funded by AMED—and selected 53 awards for a Practical Research Project for Allergic Diseases and Immunology (AMED-PPAI) (Figure S1). 1053 papers with verified PubMed IDs were included. As the controls, we selected the Hypersensitivity, Autoimmune, and Immune-mediated Diseases Study Section (NIH-HAI), an immunology-focused project in the Americas, and Human Immunology Unit (MRC-HIU), that in Europe, extracting 373 US papers and 118 UK papers, published in 2015–2019, respectively (see Appendix S1 for all methods). The Field-Weighted Citation Impact (FWCI)—evaluating research paper quality—was highest for MRC-HIU following NIH-HAI and AMED-PPAI (Table 1, Figure 1A). Although the international co-authorship rate was lowest in the AMED-PPAI, the annual trend showed a gradual increase (Figure 1B, Table 1). The number of top 10% most cited papers2/value, evaluating funding efficiency, was highest for MRC-HIU (Table 1). To characterize these outputs, we performed natural language analyses of the top 50 FWCI papers from three FAs and top 100 papers on this topic during 2015–2019 (Figure 1C–E).4 Although all FAs produced mainly basic allergy/immunology study papers (e.g., clusters 0, 1, 2, and 9 in Figure 1D), AMED-PPAI produced relatively more clinically relevant outputs (clusters 3 and 8). Further, we analyzed MeSH headings for relevance to national unique Strategic Outlook toward 2030 formulated by the Ministry of Health, Labour and Welfare of Japan (Figure 1F, Table S1).3 While most projects were pathogenic studies, AMED-PPAI funded research published more papers on precision medicine and host-extrinsic factor relations (Action I-2, I-3). The Action II and III groups were not strongly represented. To assess societal impact, we utilized ResearchFish—a widely used database of the FAs in Europe5—to randomly select 1376 UK-funding awards and their reports, trend-matched for the AMED-PPAI (UK comparator) (Table S2). The number of intellectual properties and publications per value was higher for AMED-PPAI, while the open access rate and new funding amount obtained were higher in the UK comparator (Table 1). Despite the similar number of engagement activities, their outreach targets differed (AMED toward public and media; the UK towards students, patients, and industry; Figure 1G). The purpose of this study is not to compete for superiority among countries or FAs but to expand the possibilities for multidimensional interpretation of trends and characteristics of funded outcomes using multiple indicators rather than uniform one. Due to the limitation of open databases, we focused on three countries for the funding impact analysis, whereas scholarly output in this realm is also increasing in other countries and jurisdictions, including the EU (Figure S2). China's growth is particularly remarkable, and additional analysis is desirable with public funding status.6 Furthermore, the indices used have different trends among countries, and their balance should be carefully considered to reflect each country's science and technology policies. In conclusion, we conducted impact analyses from multiple perspectives, including indicators related to substantiality2 and index words profiling/clustering based on the national strategy.3 These findings may inform international collaborative long-term research that strategically leverages each research funding institution's strengths. We extend our special gratitude to Prof. Warwick Anderson (Former Secretary-General, International Human Frontier Science Program Organization) for his valuable suggestions regarding the manuscript. We also acknowledge Dr. Gavin Reddick at Interfolio UK for his support in the societal impact analysis. TF and AI are employees of the AMED. YO and KA are scientific advisors of AMED. MTa is the Program Officer, and HI is the Program Supervisor of the AMED-PPAI. TA (2015–2020) and YO (2018–2020) were employees of the AMED. SNi was the former Program Supervisor of the AMED-PPAI (2015–2018). The other details about competing interests are provided separately. Please note: The publisher is not responsible for the content or functionality of any supporting information supplied by the authors. Any queries (other than missing content) should be directed to the corresponding author for the article.