GraSPP-DP-E-14-002 and STIG-DP-E-14-002

Hidden Rationales behind Scientiﬁc Publication:a Case Study of Japanese Life Sciences
Sotaro Shibayama, Yasunori Baba

January 2014

GraSPP Discussion Paper E-14-002

STIG Discussion Paper E-14-002

GraSPP-DP-E-14-002 and STIG-DP-E-14-002

Hidden Rationales behind Scientific Publication:a Case Study of Japanese Life Sciences

Sotaro Shibayamaa,b,*, Yasunori Babaa
January 2014a

The University of Tokyo, Research Center for Advanced Science and Technology
4-6-1 Komaba, Meguro-ku, Tokyo 153-8904, Japan.
Phone/Fax: +81-3-5452-5371.
e-mail: shibayama@00.alumni.u-tokyo.ac.jpb
The University of Tokyo, Department of Technology Management for Innovation
7-3-1 Hongo, Bunkyo-ku, Tokyo 113-8656,JapanGraSPP Discussion Papers can be downloaded without charge from:http://www.pp.u-tokyo.ac.jp/research/dp/index.htm

Graduate School of Public Policy
The University of Tokyo
7-3-1, Hongo, Bunkyo-ku, Tokyo, 113-0033, Japan
Phone:+81-3-5841-1349
Discussion Papers are a series of manuscripts in their draft form. They are not intended forcirculation or distribution except as indicated by the author. For that reason Discussion Papers may
not be reproduced or distributed without the written consent of the author.

Hidden Rationales behind Scientific Publication: a Case Study of Japanese Life Sciences

Sotaro SHIBAYAMA a,b,*
Yasunori BABA aa

The University of Tokyo. Research Center for Advanced Science and Technology.
4-6-1 Komaba, Meguro-ku,Tokyo 153-8904, Japan
Tel/Fax: +81-3-5452-5371
shibayama@00.alumni.u-tokyo.ac.jpb

The University of Tokyo. Department of Technology Management for Innovation.
7-3-1 Hongo, Bunky-ku,Tokyo 113-8656, Japan

Abstract
Modern science relies on intense evaluation of scientific publication, often described as “publish or perish.”
However, the current science system seems to overlook the multifaceted nature of scientific publication.
Though papers published in scientific journals are considered simply the output of science, they are, in fact, acollection of differently motivated products. Drawing on in-depth survey of professors in the Japanese biology
sector, this study identifies the determinants of high and low-quality publications in an attempt to disentangle
the complex rationales behind scientific publication. Results show that even productive academics withexcellent publication records often produce low-quality papers and that such papers tend to be a by-product ofstudent training and industry collaboration. In addition, results suggest that the practice of publication isstrongly governed by certain norms formed through career. Inbred academics tend to avoid challenging
projects while returnees from research experience abroad avoid publishing papers in low-impact journals. This
study also investigates the impact of recent policies that emphasize competition and merit-based resource
allocation, finding that the overconcentration of budget and human resources on selected academics actuallyfacilitates low-quality publication.

Keywords
Scientific production; publication; motivation; science policy; evaluation*

Corresponding author.

11. Introduction
Recent policies have increasingly emphasized the role of academic science in technologicalinnovation and economic growth to boost scientific productivity (Etzkowitz and Leydesdorff, 2000; Stephan,2012). Since scientific productivity is commonly measured on the basis of scientific publication, academics
and research organizations are under intense pressure to publish (Hagstrom, 1974), often described as thedoctrine of “publish or perish” (e.g., Anderson et al., 2007; Fanelli, 2010).
Despite the strong emphasis on productivity, it is debatable whether the current regime actuallyfacilitates the progress of science. It is not difficult to find examples in which academics resort to rent-seeking
behavior such as misconduct, abuse of authorship, and inadequate citation practices to inflate superficial
productivity (e.g., Martin, 2013). Such misbehaviors set aside, there is a concern that recent policies may notcontribute to the quality of science while probably increasing the quantity of publications. Recent years haveseen a substantial increase in commercial journals, including countless online-access journals, not a few ofwhich are criticized for dubious scientific legitimacy (Bohannon, 2013). In addition, a significant proportion of
publications attract little scholarly attention in terms of citation. Laband and Tollison (2003) showed that the
proportion of uncited papers in economics had been constant over two decades. Wallace et al. (2009) also
report that uncited papers are common in many other disciplines. Though citation does not directly translate
into quality (Macroberts and Macroberts, 1996), roughly speaking, the scientific contribution of rarely cited
papers and unread journals should be limited (Cole and Cole, 1972). With this regard, current policies might be
wasting resources on trivial research and creating burden for science. While rarely cited papers are more often
produced in emerging countries, which are in the process of improving their scientific standards, such papers
are universally observed (Figure 1). Importantly, low-quality publications can result from ill-designed policy
interventions. An infamous example was reported on the Australian funding system, where the system wasreformed so that research money is now awarded based on publication count, and academics started to produce
more papers but of lower quality (Butler, 2003). “Salami slicing” is also an infamous unintended consequence,
where academics fragment a single study into multiple papers to inflate publication count (Broad, 1981).
Intriguingly, focusing on the publication pattern at the individual level, we notice that productive
academics do not always publish excellent papers, nor do unproductive academics always publish mediocre
papers. As later shown, many academics produce both high and low-quality papers. This appears paradoxical
especially when the qualitative aspect is highly regarded in every sort of evaluation system, which led us toquestion rationales behind such a publication pattern. The results of this study suggest that, though papers inscientific journals tend to be simply considered output of science, they are in fact a collection of differently2

motivated products, and that the motivation is affected differently by contexts. The goal of this study is to
explore such multifaceted rationales behind scientific publication with micro-organizational contexts taken into
consideration. In so doing, this study attempts to offer implications for science policy in better designing andimplementing the incentive system of science.
To this end, the current study offers a case study of the Japanese biology sector with in-depth
interviews and econometrical analyses based on a questionnaire survey and bibliometric data. We suppose thatprior literature based on aggregate productivity measures (e.g., publication count, citation count) obscures thesubtlety in publication. Our approach, however, disentangles the determinants of high and low-quality
publication in an attempt to better understand the subtle nature of scientific production. The field of biology inJapan offers an interesting case in that Japan is highly ranked in life sciences (Adams et al., 2010) and yet
commonly produces low-quality publications (Figure 1). Importantly, the Japanese science system is fairly freefrom extrinsic incentives (e.g., pay raise) for publication, which allows us to see some rationales for
publication possibly hidden in other contexts. The implication from the case study is threefold. First, ourresults imply that scientific papers are produced for various reasons. Some publications are produced not forscientific progress but as a byproduct of other missions. Specifically in the Japanese context, results show that
low-quality publications tend to be a by-product of student training and perhaps industry collaboration. Second,
this study suggests that the choice of publication quality is influenced by the norms on quality. Academics haveheterogeneous attitudes on engaging in low-quality research, on publishing in low-impact journals, and soforth. Results indicate that the norms are formed during the course of academics’ career, particularly affectedby inbreeding and mobility. Third, this study presents empirical evidence about the intended or unintended
impacts of policy interventions in Japan on publication quality. Results suggest that Japanese policies fostering
competition and concentration of resources (e.g., money and workforce) on productive academics actually
lower publication quality.
This paper is structured as follows. Chapter 2 presents an overview of the literature on the reward
system and publication. Chapter 3 outlines our data. Chapter 4 explores rationales behind scientific publication
based on the literature review and our interviews, which are further explored by a survey and bibliometric data
in Chapter 5. Finally, Chapter 6 discusses the results and summarizes policy implications.

2. Reward system and publication
A traditional view of Sociology of Science supposes that academics are obliged to publish theirfindings because they owe their discoveries to bodies of prior work published by predecessors (David, 2004).
3

While academics have to forgo exclusive rights to their discoveries through publication, they are rewarded byrecognition and esteem from their peer academics (Merton, 1973). This reward system behind the regime ofopen science is self-sustaining inside the scientific community. While this remains indispensable, modern
science has become increasingly interlinked with society and has been under stricter external control(Dasgupta and David, 1994; Etzkowitz and Leydesdorff, 2000). Policymakers and research organizations
evaluate academics and allocate resources based on objective evidence, typically on publication record (Geunaand Martin, 2003; Hicks, 2012). With the aid of enriched bibliometric data and computational power, various
measures of scientific productivity have been invented, such as the citation index and H-index (e.g., Hirsch,
2005; Narin and Hamilton, 1996), and incorporated into evaluation systems. Countries and researchorganizations are ranked with these measures, which has further reinforced political pressure for evaluation.
Consequently, academics are forced to strive in the publish-or-perish regime (Hagstrom, 1974).
In modern science, most evaluation systems rely somewhat on bibliometric indices, among whichcitation plays a critical role with the straightforward premise that highly cited papers offer a good foundation
for subsequent research (Cole and Cole, 1972). Thus, academics and research organizations are keen on
receiving citations, though aware of the potential problems of citations (Macroberts and Macroberts, 1996).
Their obsession with citation is easily observed in some anecdotes; for example, some journal editors were
found to coerce authors to cite their papers in peer review (Wilhite and Fong, 2012), and some universities are
known to offer part-time employment to highly cited academics with the condition that the university name be
added in their publications (Bhattacharjee, 2011). These examples might be exceptional, but the current regime
undoubtedly affects the practice of scientific publication. It is only natural that academics want to publish theirpapers in journals that are likely to invite many citations, so-called high-impact or prestigious journals,
characterized by a high quality standard. Such journals attract more submissions, and this raises the quality of
papers accepted by those journals. In this positive spiral, journal prestige has been substantiated and come torepresent the value of papers published in them (Gordon, 1984; Ravetz, 1973).1 Though the quality of papers
in prestigious journals certainly varies (Oswald, 2007), publishing in prestigious journals of itself is regarded
as a great scientific achievement. Evaluation systems explicitly or implicitly take into account the journal
impact, and academics make a tremendous effort to publish in journals of as high an impact as possible (Frank,
1994; Gordon, 1984). These features of modern scientific publication are well captured by Holub’s criticism
that “especially where a scientist publishes has become much more important than what he is publishing”

1

Although journal prestige may have weak consensus in social sciences, it attains a reasonable consensus in life sciences(Mcallister et al., 1980), which is the focal subject of this study.

4

(Holub et al., 1991).
While scientific merit is always emphasized, modern science is becoming heterogeneous. That is, notonly research and education but also contribution to society is strongly demanded (Dasgupta and David, 1994;
Etzkowitz and Leydesdorff, 2000). Among others, a recent trend called academic entrepreneurship encourages
academics and research organizations to engage in commercial activities and university-industry relations(Etzkowitz and Leydesdorff, 2000). This diversifies the motivation of academics (Frickel and Moore, 2005;
Shibayama, 2012), and importantly, it can crowd out the motivation to achieve scientific excellence by keeping
academics busy on non-research missions.2 In fact, some studies suggest inconsistency between commercial
activities and scientific production while others show their compatibility (Baldini, 2008; Fabrizio and Di Minin,2008).

3. Data
Overall, prior literature has identified a few mechanisms of scientific publication and their historicaltransition. On top of this general picture, the current study attempts to explore multifaceted rationales behindpublication with micro-organizational contexts taken into consideration. To this end, we draw on a case studyof the Japanese biology sector. We do not claim that the science system in Japan is representative, but webelieve that it offers an interesting case for a few reasons. First, the Japanese science system is fairly free fromextrinsic incentives for publication. Once academics obtain a full professorship, their income is basicallydetermined by seniority with very limited influence of scientific productivity, and their status is secure tillretirement even with no publication. This enables us to observe some rationales behind publication that might
exist in general but have been hidden. Second, even with weak incentives, Japan is highly ranked in life
sciences with a reasonable proportion of high-quality publications (Adams et al., 2010), while it produces anapparently higher rate of low-quality publications than other developed nations (Figure 1). Third, the Japanesescience system is in a transition for recent reforms, and resulting heterogeneous policy contexts allow us to see
variation in publication behavior. The modern science system in Japan started in the early 20th century,modeled on the German system (Kneller, 2007). However, since the 1990s, the government has been reforming
various aspects of the system including funding, education, employment, promotion, and so forth. These
reforms are intended to increase global competitiveness and have placed greater emphasis on merit-based
evaluation and resource allocation, basically modelling the American system (Kneller, 2007).

2

In such cases, it is not easy to evaluate the holistic impact to society; for example, even if quality of research is compromised, it
might be compensated by commercial application.

5

We first conducted interviews of 30 principal investigators (PIs) of biology laboratories. We chose
our interviewees from universities of a diverse range of organizational prestige to examine institutional
differences. Each interview took 1-2 hours. We investigated interviewees’ publication record based on
bibliometric data and asked a series of questions about their specific publications. We further inquired into
their publication strategies, the impact of recent policies, the evaluation criteria in their departments, and soforth.
To confirm and further explore the findings from the interviews, we conducted a questionnaire
survey of PIs in 2010. We restricted our survey sample to professors who are the PI of a laboratory since they
are the primary decision-makers in publication.3 We selected PIs who received national grants in the field of
biology at least once from 2007 to 2009 to make sure that they were active researchers. We prepared a list of
1,378 PIs drawing on a database of national research funds.4 After re-examining their research fields andaffiliations with public information, we chose 900 PIs in 56 universities as a final sample.5 We designed oursurvey instrument based on the interviews. We mailed the survey and collected 400 responses (response rate =44%).6 For the 400 respondents, we collected publication data of approximately 12,000 articles that the
respondents authored from 2006 to 2010 from Web of Science (WoS). We also obtained the Impact Factor (IF)data from Journal Citation Report (JCR).

4. Scientific Publication in Context
Drawing on interview results and literature review, this chapter explores how academics behave inscientific publication under specific organizational and policy contexts and how this affects the quality of
publication.
In terms of a general trend of publication quality in Japan, most of our interviewees acknowledgedthat the Japanese system has been becoming quality-oriented, consistent with the policy intention.
Simultaneously, informed about the higher rate of low-IF publication compared to other developed countries(Figure 1), they admitted that the Japanese system traditionally rewards for the volume of publication. On top

3

In the Japanese chair system, associate and assistant professors may not be a PI but be supervised by another full professor.
We used a database of Grants-in-Aid for Scientific Research, which is the primary competitive funding source for Japanese
university researchers (http://www.jsps.go.jp/j-grantsinaid/index.html).
5
The major player in academic science in Japan is national universities, and more recently private and other public (city andprefecture) schools joined the system though they are rather education oriented. As of 2010, Japan has 86 national universities(School Basic Survey by Ministry of Education, Culture, Sports, Science and Technology: http://www.e-stat.go.jp/), of which
about 50 are engaged in some life science research. Our sample covers most of the relevant national universities and several
private and other public schools.
6
To examine the non-response bias in the survey, we randomly selected 50 non-respondents and found no significant difference
between the response and non-response groups in publication productivity, organizational rank, and gender (p > 0.1).
46

of this general trend, our interviewees suggested that publication quality is affected at the micro-organizational
level in three stages of academics’ decision making (Table 1). First, since the roles of academic organizations
have become diverse (Dasgupta and David, 1994; Hackett, 1990), academics have different priorities in thoseroles. Non-research missions can crowd out motivation aiming at scientific excellence or result in by-productpublications primarily intended for non-research missions, where academics do not really care about their
quality. Second, academics arrange a portfolio of projects to achieve their goals, with some academics
concentrating only on high-quality research while others not. Such a choice of portfolio is importantly affectedby research capacity, which substantially differs between laboratories under biased resource allocation. Third,
academics can adopt different strategies in publication; for example, the choice of journals or the amount ofeffort in publication per se may differ even when identical research results are obtained. Such strategies arestrongly governed by the norms on quality, which are formed through academics’ career. In what follows, we
discuss three key determinants of publication quality: 1) priority in multiple missions, 2) norms on quality, and3) capacity for research.

4.1. Priority in multiple missions
The mission of academic organization has been becoming diverse in history (Dasgupta and David,
1994; Hackett, 1990). In addition to research and education, recent policies worldwide have emphasized directcontribution of academic organizations to society in the forms of industry-university collaboration, technology
transfer, commercial activities, and so forth (Etzkowitz and Leydesdorff, 2000; Geuna and Rossi, 2011). If
these non-research activities are independent of research, effort for non-research missions should simply
decrease output from research. However, these multiple missions are actually interrelated. For example,
industry collaboration often leads to both scientific papers and patents (Baldini, 2008). Governmental
consulting projects can result in both policy reports and scientific papers (Ynalvez and Shrum, 2011). Trainingof students may yield scientific papers along with a trained workforce (Shibayama et al., 2013). In theseexamples where academics simultaneously engage in multiple missions, it may be up to each academic’s
preference which mission comes as their first priority, but research is more likely to come secondary under the
institutional emphasis on non-research missions. Our interviewees implied that this can affect publication
quality in two ways. First, academics often produce scientific papers as a by-product of non-research missions,and they do not really care about the quality of such papers. Interestingly, our interviewees consistently
suggested that the educational mission is an important source of by-product publications with mediocre quality
in the Japanese context. Illustrating the design of typical PhD programs in Japan, a professor made thefollowing comment:

7

PhD students are usually allowed to graduate in three years [in addition to a two-year masterprogram] even if their dissertation leaves some room for improvement. Thus, students have an
incentive to wrap up their research even in a premature stage. In addition, since low-ranked
universities often require a certain number of publications as a condition for graduation, PhD
students tend to publish mediocre papers to earn a degree. PIs accept this practice from an
educational point of view.
Another PI, referring to the competition in employment, said,
PhD students have to prepare a compelling vita during the three years of the PhD program so thatthey can earn a job or fellowship after graduation. To this end, having a good publication record isvital. However, sticking to high-impact journals may be risky because it takes time and can end up
with no publication. Thus, producing mediocre-quality but low-risk papers is a reasonable strategy
for students.
Thus, in the context of the Japanese education system, while young academics are the indispensable workforce
and possibly a source of serendipity (Shimizu et al., 2012; Stephan and Levin, 1992), they could undermine thepotential value of publication.
Second, the emphasis on non-research missions can crowd out motivation to achieve scientificexcellence. When the primary reward for academics is the recognition from peers conditioned on publication,
publishing in prestigious journals should be academics’ primary concern (Merton, 1973). However, as different
types of activities are appreciated and incorporated into evaluation systems, the relative importance of
scientific contribution should decline (Ynalvez and Shrum, 2011). In fact, literature finds heterogeneity in the
motives of contemporary academics (Roach and Sauermann, 2010).
An obvious policy trend in Japan, as in other many countries, is academic entrepreneurship(Etzkowitz and Leydesdorff, 2000). Japanese universities have implemented infrastructure for patenting,licensing, and commercial activities (e.g., Nagaoka et al., 2009). Consequently, many academics haveparticipated in these activities, and their focus of research has shifted from basic toward applied areas(Shibayama et al., 2012). Prior literature has shown mixed findings about the impact of this trend on scientificproduction (Baldini, 2008). On the one hand, an entrepreneurial and use-oriented approach can potentiallyinduce fundamental breakthroughs also for pure science, as exemplified by the French scientist Louis Pasteur(Baba et al., 2009; Stokes, 1997; Subramanian et al., 2013). One PI referred to the downside of narrowly
focusing on basic science:
Typical PIs emphasizing pure science tend to shut themselves inside their labs, unwilling to engage
in public relations. Though this may be efficient, it can limit the input of resources and information
8

from outside. Successful PIs who are practically motivated seem to attract input from the public
and the industry, which allows them to dynamically adjust their research direction and stay on the
cutting edge.
In fact, Baba et al. (2012) find that a sizable proportion of Japanese material scientists produce both practicallyuseful and scientifically important discoveries.
On the other hand, the trend can also cause both crowding-out and by-product effects. Concerning
motivational crowding out, one interviewee mentioned:
My perception on applied research is that not only scientific excellence but also other societal
factors play an important role in employment, promotion, and funding. To be practically useful,academics cannot always stay inside their lab but they must engage in social interaction to transfer
their knowledge to society. They simply spend more time on public relations or non-research
activities to achieve societal contribution. This can weaken the incentive to pursue high-impact
research.
Furthermore, it is well known that academics engaging in entrepreneurial activities tend to be secretive, thus
compromising scientific quality in their publications (Walsh et al., 2007). Walsh and Huang (2007) find thatJapanese academics are clearly secretive in publications coauthored with industry, suggesting that the goal of
publication is more advertisement for industry partners than advancement of science. In addition, Hicks et al.
(1996) reported that university-industry coauthored papers in Japan often resulted from the secondment ofcorporate researchers to university laboratories, where the coauthorship is intended to signal social
commitment rather than indicate genuine scientific contribution. These observations imply that the emphasison entrepreneurship can facilitate by-product publication possibly of mediocre quality.

4.2. Norms on quality
Though the country-level overall quality standard appears to be improving in Japan (Morichika andShibayama, 2011), some interviewees, particularly junior professors, suggested that there remains a gap in
quality standards between older and younger generations, and that younger generations under intense pressure
for competition are keener on quality. This implies that the norms on quality might persist once establishedunder a certain context in early career stages.
The Japanese career system used to be, and still is partly, characterized by permanent employment anda high rate of inbreeding (Horta et al., 2011), while the reform has facilitated fixed-term employment andmobility (Morichika and Shibayama, 2011). The effect of inbreeding on productivity has been studied with9

inconclusive findings (e.g., Hargens and Farr, 1973; Inanc and Tuncer, 2011). One clear effect of inbreeding is
to eliminate the pressure from external competition, which can affect productivity either positively bystabilizing the research conditions or negatively by giving academics room for slack. In terms of its effect on
quality, our interview suggests that stability as well as slack affects quality more often negatively thanpositively. In general, PIs want their inbred subordinates to further the PIs’ own research agenda because thiswill raise their reputation. In this scenario, subordinates would be obedient since they want to secure their
future job. Even if PIs are generous enough to allow autonomy in subordinates’ choice of research topics,
subordinates have a reason to remain in established topics and avoid novel but risky ones. Furthermore, since
the practice of lab work, including publication strategies, is usually passed down from PIs to their students(Delamont et al., 1997), inbred PIs would follow conventions formed in a volume-oriented regime.
Consistently, one professor with a highly mobile career mentioned from his observation of inbred colleagues:

In my impression, inbred academics tend to emphasize quantity of production more than quality.
When junior academics earn a position of PI in a new affiliation, they have to bear a start-up costand endure a temporary stall of production. Since this initial cost is substantial and cannot becompensated by publishing mediocre papers, mobile academics need to bet on risky but
high-impact subjects. On the other hand, inbred PIs can maintain productivity, when their previous
boss retires, by keeping the ex-boss’s lab setup and maintaining the same research agenda eventhough such an approach might lack originality.

Thus, the practice of inbreeding seems to sustain traditional norms and deter academics from adopting
quality-orientated publication strategies.
Such negative effects of inbreeding may be addressed by facilitating mobility. Among different typesof mobility, our interviewees suggested that international mobility has a particularly strong effect. Japanesescience policies have consistently encouraged international mobility to Western countries (mostly to the US),
where the rate of high-quality publication is higher than in Japan (Figure 1). As prior literature suggested,mobility to those countries can produce a learning effect and a network effect, which directly improves
productivity (Hoisl, 2007; Scellato et al., 2012). However, we assume that an equally or more important effectis that international mobility can change academics’ quality standard. With a sample of economists,
Hamermesh and Pfann (2012) suggest that publication count could have a negative effect on reputation in theUS, implying that publishing many mediocre papers is looked down upon in the American academic culture.
This is clearly different from the norms on quality, at least in the pre-reform Japanese system. Thus, the norms
10

on quality seem to be shaped under the influence of geographically or culturally constrained contexts, andinternational mobility can help transform them. Our interviewees with foreign experience tended to emphasize
quality and hold a negative view of mass production. Two interviewees who used to have professorships in the
US made the following comments:

I have a policy not to publish in journals with an Impact Factor (IF) less than 3.5. I would rather
scrap my paper than to publish it in lower-impact journals. From my experience in the US, junk
papers are not only unappreciated but also could harm my reputation. Besides, publishing injournals of very low IF (e.g., IF < 1) is pointless since it would not contribute to science.

My perception is that academics in my field consider an IF around 4 a fair quality. Without having
a publication of that class, it would be impossible to find a job in the US. I advise my lab members
to keep this standard. Journals with an IF of 2 are minimally acceptable. Publishing in journals
with an IF less than 1 is meaningless since nobody would read such journals.

Thus, in the Japanese context, international mobility seems to affect the norms on quality and accordingly
encourage research and publication strategies with a stronger orientation for quality.

4.3. Capacity for researchPrior literature has shown that scientific production heavily relies on resource input (e.g., Jacob andLefgren, 2011; Johnston, 1994). This should be the case particularly in biological research, which is highly
resource-intensive as a result of its strong empiricism and dependence on experimental work (Bertalanffy et al.,1962). As is the global trend (Geuna and Martin, 2003; Hicks, 2012), the Japanese science policy hasemphasized merit-based evaluation and resource allocation. This changes the distribution of research money
and human resources, both strongly affecting the research capacity of biology laboratories.
The Japanese funding system consist of roughly two parts; non-competitive block grants foruniversities, part of which is redistributed to faculty members, and competitive grants for individual academics.
In the reform, non-competitive block grants have been gradually replaced by competitive grants, and researchmoney has been increasingly concentrated on selected universities and individuals. Shibayama (2011)
investigated the distribution of national funds at the individual and university levels and founddisproportionately large grants for limited academics and the practice of multiple awarding (i.e., one academic

11

receives many grants simultaneously).7 From policymakers’ viewpoint, it may be reasonable to concentrate
the budget on productive researchers rather than distributing a small amount of money to everyone. In fact,
literature suggests that high-impact publication tends to be produced with the support of large funding(Shimizu et al., 2012). However, this does not necessarily mean that all large budgets are efficiently used.
From the interviews, we found that large budgets could cause some undesirable consequences. The
most straightforward is that academics start carelessly spending money, though this may not affect research
quality. Another wasteful but natural reaction of PIs who receive more budget than necessary is that theyattempt to use up all the money rather than to return it to the funders. Since a PI has a limited number of
research ideas, PIs with excessive budgets often try their second and third best ideas, knowing that these maylead to only mediocre results. Furthermore, PIs can exploit a large budget more strategically. Scientificresearch is highly unpredictable, and the risk may be lowered by running many projects. Such a strategy istotally legitimate though it is debatable whether the portfolio should be arranged inside each laboratory orcollectively among the community. Problematically, our interviewees suggested that some PIs recklessly carryout numerous projects without carefully designing them, hoping that a few of them by chance turn out
innovative at the cost of many mediocre results. Two interviewees critically made the following comments:

One typical strategy for mass production is to slightly modify someone else’s prior studies so thatthey can produce some results with a minimum effort. Though such an approach is unlikely toyield high-impact results, it sometimes accidentally gives interesting results. If a budget issufficiently large, this approach could produce high-impact publications constantly.
Pursuing originality is psychologically tough because of the high risk involved. Compared to that,a mass production strategy mimicking or following others’ research is much easier. What PIs haveto do is to raise funds and arrange facilities. Then, as many publications as the number of students
would come out almost automatically though their quality is questionable.

The payoff of these strategies would be acceptable when efficiency is disregarded in evaluation. In addition,
once PIs are accustomed to large budgets, they have difficulty in downsizing their laboratory. Laying off
research staff is stressful, and the cost of operating large devices is high. In such cases, maintaining revenue
size can become a PI’s priority over pursuing scientific excellence (Sousa, 2008); i.e., they act more like arevenue-maximizer than a profit-maximizer. One senior professor about to retire made the following comment:
7

The government has attempted to mitigate these issues after criticism from science communities.

12

I was awarded a large grant in my early career. Once I got used to it, I kind of wanted to have thesame amount of budget continuously. Having a lot of money is not bad, but writing funding
proposals and reporting is time-consuming. Looking back on my career, I regret that I should haveused my time more on research itself than on funding.

In sum, though we agree that the productivity of biology research should depend on budgetary input (e.g.,
Jacob and Lefgren, 2011), too large a budget could compromise quality.
In addition to research budgets, the reform has increased the bias in the distribution of humanresources. The primary research workforce in Japanese universities is PhD students rather than postdocs. A
reform, in an attempt to reinforce graduate education, doubled the overall quota of graduate students during the
1990s. However, this has yielded numerous PhDs and postdocs without stable employment, and the apparently
bleak prospect of an academic career has significantly decreased PhD enrolment especially in low-ranked
universities during the last decade. Consequently, the reform has created an unequal distribution of PhD
students among universities. A director in a local university said,
In the past, we used to attract 200 applicants for 50 seats of the PhD course. The situation haslargely changed after the reform, especially after the University of Tokyo [one of the top
universities] was allowed to increase its PhD quota, which took all the good students away. We
have barely filled our quota in recent years.
The effect of lab size on scientific production, though well studied, remains inconclusive. Cohen (1981)
reported that publication count per researcher is constant regardless of lab size. Likewise, Johnston (1994)
suggested that productivity does not improve with size, but that below a critical mass of lab size, typically 5-8
researchers, productivity declines. Focusing on PhD education, Salonius (2008) observed that the quality ofstudent training worsens when lab size increases and PIs cannot spend sufficient time on each student. Thisimplies that the quality of students’ projects may be sacrificed in laboratories with too many students. In fact, asenior PI of a medium-sized laboratory said,

I am not interested in expanding my laboratory. I want to concentrate on thoroughly-planned andwell-selected projects in a modest sized lab. Considering my own capacity, I could not come up
with a greater number of promising projects than I currently do. If I accepted more students, I
would have to assign them improvised or mediocre projects. Currently, my students publish
reasonably good papers in good journals, but mediocre papers would be inevitable in a larger lab.
13

In line with this opinion, another professor criticized PIs in top universities:

Some PIs in top universities accept many students and produce numerous papers. A problem is thatstudents in such big labs have to compete with one another inside the small world of labs, andsome of them inevitably end up losers and become demotivated. I believe that even those losers
would be stars and much more productive in a less competitive environment. The excessive
concentration of students in top universities has compromised holistic productivity.

In sum, excessive concentration of students can be counterproductive, while we agree that a certain lab size isessential in labor-intensive biological research (Johnston, 1994).

5. Regression Analysis of the Determinants of Publication Quality
5.1. Analytical approachThis chapter further explores the findings in the previous chapter with a survey and bibliometric data.
Our analytical approach draws on the quality-based portfolio of scientific publication at the individual level;that is, we collected the data of scientific papers published by an individual academic in a certain period andanalyzed the distribution of quality among the collection of papers. A typical measure of publication quality iscitation count (e.g., Garfield, 1972), but for the following reasons, we draw on the distribution of Impact
Factors (IFs) of the journals where our respondents published their papers. IF is a measure of quality orprestige at the journal level.8 Though IF is a controversial index (e.g., Denrell and Liu, 2012), it has attained ahigh consensus particularly among life scientists (Mcallister et al., 1980) and is incorporated in sciencesystems. In fact, our interviewees suggested that their perceived quality of journals reasonably agree with IFs,
and that IFs are, explicitly or implicitly, in common use in formal evaluations, though they express a concern
in excessive emphasis on IFs. As a result, the decision of academics and research organizations is strongly
influenced by IFs in some fields where academics’ perceptions of journal quality and IFs coincide well (Frank,
1994; Gordon, 1984). Thus, we believe that academics’ choice of journals in terms of IFs well reflects theirmotives and is probably more directly affected by policy and organizational contexts than citation count is. Of
course, journal choice can be beyond academics’ control (i.e., submitted papers may be rejected), but Calcagno
et al. (2012) show that authors usually know where their papers will be accepted, and approximately 70% of
8

IF is defined as “the number of current citations to articles published in a specific journal in a two year period divided by the
total number of articles published in the same journal in the corresponding two year period”(http://ip-science.thomsonreuters.com/support/patents/patinf/terms/).

14

published papers are accepted by the journals to which they are first submitted. That is, authors recognize thequality of their own papers and submit them to journals of suitable IFs.
To identify the determinants of high and low-quality publication, we categorize journal classes andinvestigate the explanatory factors for publication in each class. Based on our interviews, we categorize fourjournal classes; very low quality (IF<1), low quality (IF=1-2), middle quality (IF=2-8), and high quality (IF>8).
Our interviewees suggested an intuitive threshold, saying that “IF<1 means that papers in the journal would beread [i.e., cited] only once [in two years], and publishing a paper in such journals makes no impact onscience.”9 We set another less stringent threshold of IF<2. Interviewees also suggested that IF=7-8 or above isregarded as high quality, so we consider IF>8 as high quality. The 12,000 articles that our respondents
published from 2006 to 2010 comprise 7% very-low, 14% low, 69% middle, and 10% high-quality
publications.10

5.2. Measurement
5.2.1.

Dependent variables
We run regressions at the individual level, and supplementarily, at the article level. As article-levelmeasures, we prepared dummy variables of IF<1, IF<2, and IF>8. To make our analyses comparable to thosein prior literature, we also use adjusted citation count of each paper.11 As individual-level measures, we
computed the percentage of each IF class: %IF<1, %IF<2, and %IF>8. We also computed the publication
count and the summation of the adjusted citation count for all papers published by each respondent during thefive-year period. On average, our respondents published 6.6 papers per year. Figure 3 illustrates thequality-based portfolio of publication by plotting %IF<2 and %IF>8, suggesting that most PIs publish both
high and low-quality papers. One concern regarding the use of IFs is that IFs are field-dependent (Seglen,1997). Though our sample is restricted to biology, it includes multiple subfields. To address this issue, we also
computed field-adjusted IF classes.12 Since regression results with non-adjusted and adjusted measures are
9This threshold may appear too strict for social scientists since IF<1 is not uncommon in many social sciences fields. This is not
the case in life sciences as later explained.
10
The majority of the 12,000 articles were published in the subject category of “Biochemistry & Molecular Biology (BMB),”
which is the second largest in WoS. In 2010, about 64,000 articles were published in BMB worldwide, and each IF classaccounted for 5%, 14%, 74%, and 8% from very low to high quality (Appendix 1).
11
Since recent papers should have fewer citations than old papers, we divided the citation count of each paper by the number ofyears elapsed since its publication (Shibayama et al., 2013).
12
In each Subject Category of WoS, all journals registered to JCR2010 are sorted with respect to IFs. With BMB used as a base,
we defined high-quality and low-quality journals; i.e., a journal is defined as high-quality if its ranking is comparable to IF>8journals in BMB, and a journal is defined as low-quality if its ranking is comparable to IF<2 journals in BMB. This adjusted
measure is not perfect, either. When IFs in a certain field are generally low (or high), it may be due to field-specific citation
practices but it might be due to truly low (or high) quality of science in the field. We suppose that the true quality is somewhere
between the adjusted and non-adjusted measures. As later shown, results from both measures are qualitative similar, which lends

15

qualitatively similar (Appendix 2), we primarily report results with non-adjusted measures.

5.2.2.

Independent variables
To measure research orientation, we asked: “Which describes your research goal, basic (aiming at

advancement of theory and knowledge) or applied (aiming at solving problems in the real society)?” using afive-point scale, 1 = mostly basic, 2 = more basic than applied, 3 = both to a similar extent, 4 = more applied
than basic, and 5 = mostly applied (applied research). Furthermore, with the publication data, we identified
industry partners in the author list of each paper. We prepared a dummy variable at the article level, assigning
one if at least one author was from industry (industry coauthor), and another dummy variable at the individuallevel, assigning one if at least one paper was coauthored with industry in the five-year period (industry
collaboration). To examine the possibility of by-product publication due to student training, we identified PhD
students included in the author list of each paper.13 Since the first author tends to be the primary contributor in
biology papers, we prepared a dummy variable, assigning one if a paper’s first author is a PhD student (student
first author), which we use for article-level regressions. In addition, we inquired into time allocation. We
measured time for research, teaching, and administration based on the number of hours per week spent oneach activity with a six-point scale: 1 = less than 10 hours, 2 = 10-20 hours, 3 = 20-30 hours, 4 = 30-40 hours,
5 = 40-50 hours, and 6 = 50 hours or more.
As for variables on career, we measured PIs’ generation by the number of years since the first degree.
In addition, we prepared two measures for the mobility of academics. We measured international mobility witha 6-point scale: 1 = none, 2 = less than half a year, 3 = one year, 4 = 2 years, 5 = 3 years, and 6 = 4 years ormore (foreign experience). Examining curriculum vitae, we found that the majority of foreign experienceoccurred in the US. We also measured inbreeding with a dummy variable, assigning one if respondents earned
their PhD in the current laboratory and zero otherwise (inbred).
As for research capacity, we asked the number of PhD holders (i.e., senior staff and postdocs) and
PhD students, respectively, in each respondent’s laboratory. We control for the number of PhD holders (#staff)
as a lab-size measure. To measure the extent of workforce concentration, we divided the number of PhD
students by the number of PhD holders (#PhD/#staff) with the assumption that students are supervised andtrained by experienced researchers. We then asked about the total research budget in the year of 2010. To
some confidence in the interpretation of our results.
13
The Japanese National Library provides the database of PhD dissertations (http://opac.ndl.go.jp/), where we downloaded thelist of PhD graduates during 2006-2013. Considering that the PhD course in Japan generally takes three years, the list should
cover most PhD graduates who were students during 2006-2010. With the publication year, full author name, and affiliation from
WoS, we matched the publication data with the dissertation data and identified PhD students in the author list of each paper.

16

measure the extent of budget intensity, we divided the total budget by the number of all lab members(budget/#member) (in million JPY; roughly 1USD = 100 JPY).
We attempt to control for the PIs’ innate ability by computing the adjusted citation count before theyearned tenure positions (#adjusted cite before tenure). We also control for institutional capital by university
rank based on the amount of competitive funding with a four-point scale: 4 = 1st or 2nd, 3 = 3rd-7th, 2 = 8th-20th,and 1 = 21st or below (univ rank).14 We also control for gender (female). At the article level, we control for thenumber of authors (#author) and the number of organizations (#organization) listed in the author information.
Table 2 presents the descriptive statistics at the individual level.

5.3. Determinants of publication quality
5.3.1.

Multiple missions
Table 2 presents the regression results with five dependent variables at the individual level: thepercentage of papers in IF<1, IF<2, and in IF>8 journals, publication count, and adjusted citation count.
First, we examined the effect of entrepreneurial orientation. Engagement in applied research shows anegative effect on high-quality publication (Model 3: b = -1.60, p < .01) and a positive effect on low-quality
publication (Model 2: b = 3.06, p < .01). However, industry collaboration shows a tendency to decrease
low-quality publication (Model 1: b = -4.76, p < .01; Model 2: b = -4.17, p < .05). Thus, if research is not only
applied but also actually applicable to industrial needs, it could increase the quality of research. This agrees
with the argument of Pasteur scientists (Stokes, 1997). However, reverse causality is possible; academics
producing high-quality research attract industry partners.
To further examine the effect of industrial collaboration, we also ran the article-level regressions inTable 3. Results in Table 3A are mostly consistent with those in Table 2, but the dummy variable of industry
coauthorship does not show a significant effect. For closer inspection, we include in Table 3B two dummy
variables assigning the value of one: 1) if the particular article is coauthored with industry, and 2) if theparticular article is not coauthored with industry but the respondent had at least one industry-coauthored paperin the past five years. Results show that the latter variable has a significantly negative effect on low-quality
publication (Model 1: b = -.383, p < .05; Model 2: b = -.214, p < .1), consistent to the individual-level analysis.
However, the former variable has an insignificant effect with a smaller magnitude. Thus, the positive effect due
to industry collaboration decreases if a paper is coauthored with industry partners. This supports the argumentthat industry collaboration might compromise quality due to secrecy or that such papers are published more for
14

This categorization is based on the fact that top seven universities in Japan enjoy prestigious status as pre-imperial universities,
and that Universities of Tokyo and Kyoto are exceptional among others.

17

advertisement than for scientific progress. Overall, a possible cynical scenario is that industries tend to be
attracted by productive academics with the intention to exploit the collaboration for non-academic objectives
and publish papers of mediocre quality.
Second, concerning the effect of student training, Table 3A introduces a dummy variable of whether
the first author is a PhD student. The effect is unclear; significant only for IF<2 (Model 2). Since the ability of
PhD students can significantly differ with university ranks (Shibayama et al., 2013), we incorporated theinteraction term between student first author and university rank (Table 3C). The result suggests thathigh-quality publication is facilitated when a paper is first-authored by students in high-ranked universities.
However, in low-ranked universities, student-led papers are of significantly lower-quality.15 The former result
may be consistent with the argument that young academics are the source of serendipity (Stephan and Levin,1992). The latter result, on the other hand, suggests that student training may lead to by-product mediocre
publication.
Third, we investigate the effect of time allocation. Time for research has a significantly positiveeffect on high-quality publication (Model 3: b = 1.22, p < .01) but not on low-quality publication (Models 1
and 2). Our interviewee suggested that high-quality research takes deep thought and is time-consuming. Thus,
it is likely that only those who can afford sufficient time produce high-quality research. Since time for researchdoes not increase publication count but rather decreases it (Model 4), academics with greater research time
seem to concentrate on high-quality research but not to engage in many projects. Time for administration
significantly lowers quality (Model 1: b = 2.22, p < .001; Model 3: b = 2.79, p < .01). Since this occurs after
time for research is controlled for, administrative effort might crowd out the motivation to achieve scientificexcellence. Time for teaching has no significant effect.

5.3.2.

Career and norms on quality
As for the norms on quality, we examined a few career factors. First, the results show that the

number of years since the first degree is positively associated with low-quality publication (Model 2: b = .30, p< .1) and negatively with high-quality publication (Model 3: b = -.27, p < .01). This is consistent with theobservation that younger generations are keener on quality than older generations.
Foreign experience shows a negative effect on low-quality publications (Model 1: b = -1.64, p< .001; Model 2: b = -1.90, p < .01) but only a weakly positive effect on high-quality publications (Model 3: b
= .59, p < .1). It also decreases publication count (Model 4: b = -1.79, p < .05). Therefore, returnees fromforeign experience may not only avoid low-quality research but also choose not to submit low-quality papers
for publication. In addition, we dichotomize the variable with different thresholds (the original measure is15

The effect of student first author on high-quality publication is -.363 (= -.583 + 1 * .220) in bottom-ranked universities (univrank = 1) and +.297 (= -.583 + 4 * .220) in top-ranked universities (univ rank = 4).

18

six-point scaled) and find that foreign experience longer than one year changes the portfolio of publication
quality. That is, too short a stay may not affect academics’ norms but that a one-year or longer stay facilitatesnormative assimilation.
Inbreeding shows a negative effect on high-quality publication (Model 3: b = -4.47, p < .01) and aweakly positive effect on low-quality publication (Model 2: b = 5.58, p < .1). The strongly negative effect on
high-quality publication implies that inbred academics tend not to engage in challenging and novel subjects.

5.3.3.

Research capacity
As for research capacity, we examined the effect of resource input with three variables: the numberof staff, per-staff PhDs, and per-member budget. The number of staff is positively associated with publication
count and citation count (Model 4: b = 4.08, p < .001; Model 5: b = 8.03, p < .001) and so is per-staff PhDs
(Model 4: b = 9.63, p < .05; Model 5: b = 25.20, p < .05). Budget also shows a positive effect (Model 4: b =1.40, p < .05; Model 5: b = 3.82, p < .05). These results are consistent to prior findings that scientificproduction depends on resources. On the portfolio of publication quality, results basically show that research
capacity is positively associated with high-quality publications (Model 3) and negatively associated with
low-quality publications (Models 1 and 2). These results are consistent with the assumption that biology
research is resource- and labor-intensive (Jacob and Lefgren, 2011; Johnston, 1994).
However, the quadratic terms imply that the effect is diminishing. The quadratic term of per-staff
PhD shows a negative coefficient on high-quality publication (Model 3: p = -1.12, p < .1). Per-staff PhD showsno effect on low-quality publication (Models 1 and 2). Therefore, PhD students may be the source ofhigh-quality research, but this effect is compromised when a laboratory accepts more students than its staff canoversee, possibly due to overcapacity for training. The result implies that one staff member can supervise at
most two PhD students effectively (as far as high-IF publications are concerned).
Research budget also shows a diminishing effect. The quadratic term of budget has a negative
coefficient on quality (Model 1: b = .02, p < .1; Model 2: b = .06, p < .01; Model 3: b = -.03, p < .001). Thus,
excessive budget compromises quality. A budget larger than 20 million JPY (approximately 200K USD) per
member per year is likely used inefficiently. Since the quadratic term on publication count is insignificant(Model 4), a larger budget linearly yields more papers, but their quality starts to decline at some point.

6. Discussion and conclusions
Modern science is characterized by the doctrine of publish or perish with a strong emphasis on
evaluation of scientific publication (Fanelli, 2010), but its impact on the progress of science has been
controversial. For example, recent years have seen a great increase in journals of questionable scientific quality(Bohannon, 2013), and a significant proportion of publications remain unread or uncited (Wallace et al., 2009).
Intriguingly, academics who are capable of producing excellent papers are found to produce also low-quality
19papers even under quality-oriented evaluation systems. To explicate this seemingly paradoxical behavior and to
explore the rationales of scientific production in general, this study examines the quality-based portfolio of
publication to disentangle complex rationales behind publication. Drawing on a case study of the Japanese
biology sector with in-depth interviews and a questionnaire survey, this study reveals distinctive determinantsof low and high-quality publication. We believe that this approach highlights the multifaceted nature ofscientific production that prior literature based on conventional productivity measures (e.g., citation count)might have overlooked. For example, our results indicate that excessive funding can lead to low-quality
publication, but such an effect is overshadowed if the aggregate citation is counted.
This study offers three implications for future research and science policy. First, this study indicatesthat scientific papers are produced for many different reasons, and that they may be different sorts of products
from authors’ viewpoints even if they appear the same to policymakers and administrators. In particular, ourresults imply that some papers are published as a by-product of student training and industry collaboration.
Thus, evaluating publication purely on its scientific merit might be misleading, especially when the role ofacademic organizations has become diverse in modern science (Dasgupta and David, 1994; Hackett, 1990).
Second, this study suggests that publication quality is governed by the norms on quality, which cansubstantially differ with academics’ career process such as inbreeding and mobility. Particularly, in theJapanese context, we found that returnees from foreign experience (mostly from the US) tend to demonstratepeculiar norms; they despise low-quality research and are unwilling to publish in low-IF journals. In fact,
returnees publish fewer papers than their domestic counterparts, to which two interpretations could be given.
One is that returnees may concentrate on high-quality and time-consuming research, while the other is thatthey scrap papers that are rejected or likely to be rejected by high-impact journals. Our interview implies thatthe latter is the case to some extent. In line with our finding, prior literature shows that negative data tend not
to be published, criticizing policies excessively emphasizing impact of publication (Dwan et al., 2008; Fanelli,2010). Though papers published in low-impact journals offer a weaker scientific contribution, on average, thisdoes not mean that they are useless. The norms on quality observed in our Japanese case and presumably
shared widely in American academics (Hamermesh and Pfann, 2012) may deter the disclosure of scientificdiscoveries.
Third, this study offers empirical evidence of a detrimental effect of specific Japanese policies onscientific production. The policies have emphasized competition and concentration of budget and humanresources on productive academics with the intention of fostering high-quality research. However, our results
indicate that excessive resource input can actually compromise quality. Possibly, academics with abundant20

resources might resort to mediocre ideas, or they might employ mass-production strategies without a thoroughdesign, hoping for accidental success. Furthermore, overconcentration can compromise the diversity of
research. Emphasis on competition and merit-based resource allocation is a global trend (Geuna and Martin,
2003; Hicks, 2012). Taken together with the so-called Matthew effect (Merton, 1973), such policies could leadto unjustifiably biased resource allocation. These results call for rethinking the evaluation systems in modern
science. Our results also imply that mobility policy can affect academics’ norms and their publication strategies,
and that the complex mix of academics’ missions can change publication quality. Since science polices similar
to those of recent Japan are commonly observed in other countries, their impact on scientific production should
be further examined with each country’s context for future development of science policies.
In sum, this study attempts to shed light on the complex rationale behind scientific publication and to
inform the policy debate on the evaluation of scientific publication. Some limitations needs to be addressed infuture research. One obvious limitation is its sample specificity in terms of the country and scientific field. We
believe that the unique context of the Japanese system offers some advantage for the goals of this study, but
external validity needs to be examined in future research. Second, publication quality is affected at multiple
stages of academics’ decision making (i.e., choice of missions, projects and journals). Future research should
add empirical evidence about the impact of specific factors on those different stages. Third, IF needs cautious
interpretation. Given that IF attains a reasonable consensus in life sciences (Mcallister et al., 1980), we
suppose that our dependent variables reflect academics’ motives and are strongly affected by policy andorganizational contexts. Nevertheless, we cannot deny the limitations of IFs (Seglen, 1997). Fourth, for thenature of cross-sectional data, the possibility of endogeneity cannot be ruled out. Future research should test
the impact of a specific policy intervention with dynamic data.

Acknowledgments
We would like to acknowledge David N. Laband, John P. Walsh, and Miho Funamori for guidance andcomments. We would also like to thank Noriyuki Morichika and Naohiro Shichijo for technical support.
Funding for this research was provided by the Konosuke Matsushita Memorial Foundation, Inamori
Foundation, Grant-in-Aid for Encouragement of Young Scientists (B) of the Japan Society for the Promotionof Science (#24710160), and Grant-in-Aid for Scientific Research (B) Program (#23330121) form The
Ministry of Education, Culture, Sports, Science and Technology (MEXT) of Japan.

21

References
Adams, J., King, C., Miyairi, N., Pendlebury, D. 2010. Global research report japan. Leeds, UK: Thomson
Reuters.
Anderson, M. S., Ronning, E. A., De Vries, R., Martinson, B. C. 2007. The perverse effects of competition onscientists' work and relationships. Science and Engineering Ethics, 13: 437-461.
Baba, Y., Shichijo, N., Sedita, S. R. 2009. How do collaborations with universities affect firms' innovative
performance? The role of "pasteur scientists" in the advanced materials field. Research Policy, 38:
756-764.
Baba, Y., Shichijo, N., Sedita, S. 2012. Comparing the contribution of differently motivated scientists: Pasteur
vs. Bohr, Academy of Management Meeting. Boston, Massachusetts.
Baldini, N. 2008. Negative effects of university patenting: Myths and grounded evidence. Scientometrics, 75:
289-311.
Bertalanffy, L. V., Rapoport, A., Meier, R. L. 1962. General systems yearbook. Washington, D.C.: Society for
General Systems Research.
Bhattacharjee, Y. 2011. Citation impact saudi universities offer cash in exchange for academic prestige.
Science, 334: 1344-1345.
Bohannon, J. 2013. Who's afraid of peer review? Science, 342: 60-65.
Broad, W. J. 1981. The publishing game - getting more for less. Science, 211: 1137-1139.
Butler, L. 2003. Explaining australia's increased share of isi publications - the effects of a funding formulabased on publication counts. Research Policy, 32: 143-155.
Calcagno, V., Demoinet, E., Gollner, K., Guidi, L., Ruths, D., De Mazancourt, C. 2012. Flows of researchmanuscripts among scientific journals reveal hidden submission patterns. Science, 338: 1065-1069.
Cohen, J. E. 1981. Publication rate as a function of laboratory size in 3 biomedical-research institutions.
Scientometrics, 3: 467-487.
Cole, J. R., Cole, S. 1972. Ortega hypothesis. Science, 178: 368-&.
Dasgupta, P., David, P. A. 1994. Toward a new economics of science. Research Policy, 23: 487-521.

22

David, P. A. 2004. Understanding the emergence of 'open science' institutions: Functionalist economics inhistorical context. Industrial and Corporate Change, 13: 571-589.
Delamont, S., Parry, O., Atkinson, P. 1997. Critical mass and pedagogic continuity: Studies in academic
habitus. British Journal of Sociology of Education, 18: 533-549.
Denrell, J., Liu, C. 2012. Top performers are not the most impressive when extreme performance indicatesunreliability. Proceedings of The National Academy of Sciences of The United States of America,
109: 9331-9336.
Dwan, K., Altman, D. G., Arnaiz, J. A., Bloom, J., Chan, A. W., Cronin, E., Decullier, E., Easterbrook, P. J.,
Von Elm, E., Gamble, C., Ghersi, D., Ioannidis, J. P. A., Simes, J., Williamson, P. R. 2008.
Systematic review of the empirical evidence of study publication bias and outcome reporting bias.
Plos One, 3.
Etzkowitz, H., Leydesdorff, L. 2000. The dynamics of innovation: From national systems and "mode 2" to a
triple helix of university-industry-government relations. Research Policy, 29: 109-123.
Fabrizio, K. R., Di Minin, A. 2008. Commercializing the laboratory: Faculty patenting and the open scienceenvironment. Research Policy, 37: 914-931.
Fanelli, D. 2010. Do pressures to publish increase scientists' bias? An empirical support from us states data.
Plos One, 5.
Frank, E. 1994. Authors criteria for selecting journals. Jama-Journal of the American Medical Association,
272: 163-164.
Frickel, S., Moore, K. 2005. The new political sociology of science. Madison, WI: The University of
Wisconsin Press.
Garfield, E. 1972. Citation analysis as a tool in journal evaluation - journals can be ranked by frequency andimpact of citations for science policy studies. Science, 178: 471-+.
Geuna, A., Martin, B. R. 2003. University research evaluation and funding: An international comparison.
Minerva, 41: 277-304.
Geuna, A., Rossi, F. 2011. Changes to university ipr regulations in europe and the impact on academic
patenting. Research Policy, 40: 1068-1076.
Gordon, M. D. 1984. How authors select journals - a test of the reward maximization model of submission
23

behavior. Social Studies of Science, 14: 27-43.
Hackett, E. J. 1990. Science as a vocation in the 1990s - the changing organizational culture of academic
science. Journal of Higher Education, 61: 241-279.
Hagstrom, W. O. 1974. Competition in science. American Sociological Review, 39: 1-18.
Hamermesh, D. S., Pfann, G. A. 2012. Reputation and earnings: The roles of quality and quantity in academe.
Economic Inquiry, 50: 1-16.
Hargens, L. L., Farr, G. M. 1973. Examination of recent hypotheses about institutional inbreeding. American
Journal of Sociology, 78: 1381-1402.
Hicks, D. 2012. Performance-based university research funding systems. Research Policy, 41: 251-261.
Hicks, D. M., Isard, P. A., Martin, B. R. 1996. A morphology of japanese and european corporate research
networks. Research Policy, 25: 359-378.
Hirsch, J. E. 2005. An index to quantify an individual's scientific research output. Proceedings of The
National Academy of Sciences of The United States of America, 102: 16569-16572.
Hoisl, K. 2007. Tracing mobile inventors - the causality between inventor mobility and inventor productivity.
Research Policy, 36: 619-636.
Holub, H. W., Tappeiner, G., Eberharter, V. 1991. The iron law of important articles. Southern Economic
Journal, 58: 317-328.
Horta, H., Sato, M., Yonezawa, A. 2011. Academic inbreeding: Exploring its characteristics and rationale injapanese universities using a qualitative perspective. Asia Pacific Education Review, 12: 35-44.
Inanc, O., Tuncer, O. 2011. The effect of academic inbreeding on scientific effectiveness. Scientometrics.
Jacob, B. A., Lefgren, L. 2011. The impact of nih postdoctoral training grants on scientific productivity.
Research Policy, 40: 864-874.
Johnston, R. 1994. Effects of resource concentration on research performance. Higher Education, 28: 25-37.
Kneller, R. 2007. Prospective and retrospective evaluation systems in context: Insights from japan. In Whitley,
R. & Glaser, J. (Eds.), Changing governance of the sciences: The advent of research evaluation
systems. Dordrecht, Netherlands.: Springer.
24

Laband, D. N., Tollison, R. D. 2003. Dry holes in economic research. Kyklos, 56: 161-173.
Macroberts, M. H., Macroberts, B. R. 1996. Problems of citation analysis. Scientometrics, 36: 435-444.
Martin, B. R. 2013. Whither research integrity? Plagiarism, self-plagiarism and coercive citation in an age of
research assessment. Research Policy, 42: 1005-1014.
Mcallister, P. R., Anderson, R. C., Narin, F. 1980. Comparison of peer and citation assessment of the influenceof scientific journals. Journal of the American Society For Information Science, 31: 147-152.
Merton, R. K. 1973. Sociology of science. Chicago: University of Chicago Press.
Morichika, N., Shibayama, S. 2011. Inbreeding in japanese university laboratories., Japan Society for Science
Policy and Research Management Annual Meeting. Yamaguchi Japan.
Nagaoka, S., Kondo, M., Flamm, K., Wessner, C. 2009. 21st century innovation systems for japan and theunited states: Lessons from a decade of change: Report of a symposium. Washington, D.C.: The
NATIONAL ACADEMIES PRESS.
Narin, F., Hamilton, K. S. 1996. Bibliometric performance measures. Scientometrics, 36: 293-310.
Oswald, A. J. 2007. An examination of the reliability of prestigious scholarly journals: Evidence andimplications for decision-makers. Economica, 74: 21-31.
Ravetz, J. R. 1973. Scientific knowledge and its social problems: Oxford University Press
Roach, M., Sauermann, H. 2010. A taste for science? Phd scientists' academic orientation and self-selection
into research careers in industry. Research Policy, 39: 422-434.
Salonius, A. 2008. Working in the lab: Social organization of research and training in biomedical researchlabs in canada and its relationship to research funding MCGILL UNIVERSITY, Quebec, Canada.
Scellato, G., Franzoni, C., Stephan, P. 2012. Mobile scientists and international networks, NBER Working
Paper, Vol. 18613.
Seglen, P. O. 1997. Why the impact factor of journals should not be used for evaluating research. British
Medical Journal, 314: 498-502.
Shibayama, S. 2011. Distribution of academic research funds: A case of japanese national research grant.
Scientometrics, 88: 43-60.
25

Shibayama, S. 2012. Conflict between entrepreneurship and open science, and the transition of scientificnorms. Journal of Technology Transfer, 37: 508–531.
Shibayama, S., Walsh, J. P., Baba, Y. 2012. Academic entrepreneurship and exchange of scientific resources:
Material transfer in life sciences and materials science in japanese universities. American
Sociological Review, 77: 804-830.
Shibayama, S., Baba, Y., Walsh, J. P. 2013. Social organization of life science laboratories., American
Sociological Association Annual Meeting. New York NY.
Shimizu, H., Nirei, M., Maruyama, K. 2012. Management of science, serendipity, and research performance:
Evidence from scientists' survey in the us and japan, Schumpeter Conference. Brisbane, Australia.
Sousa, R. 2008. Research funding: Less should be more. Science, 322: 1324-1325.
Stephan, P., Levin, S. 1992. Striking the mother lode in science: The importance of age, place, and time.
New York: Oxford University Press.
Stephan, P. E. 2012. How economics shapes science. Cambridge, MA: Harvard University Press.
Stokes, D. E. 1997. Pasteurs quadrant: Basic science and technological innovation Washington, D.C.:
Brookings Institution Press.
Subramanian, A. M., Lim, K., Soh, P. H. 2013. When birds of a feather don't flock together: Different scientists
and the roles they play in biotech r&d alliances. Research Policy, 42: 595-612.
Wallace, M. L., Lariviere, V., Gingras, Y. 2009. Modeling a century of citation distributions. Journal of
Informetrics, 3: 296-303.
Walsh, J. P., Cohen, W. M., Cho, C. 2007. Where excludability matters: Material versus intellectual property inacademic biomedical research. Research Policy, 36: 1184-1203.
Walsh, J. P., Huang, H.-I. 2007. Research tool access in the age of the ip society: Results from a survey ofjapanese scientists. Washington, D.C.: American Association for the Advancement of Science.
Wilhite, A. W., Fong, E. A. 2012. Coercive citation in academic publishing. Science, 335: 542-543.
Ynalvez, M. A., Shrum, W. M. 2011. Professional networks, scientific collaboration, and publication
productivity in resource-constrained research institutions in a developing country. Research Policy,
40: 204-216.
26

Figure 1

Portfolio of Publication Quality by Country aa

The numbers in the parentheses are the number of articles. Top 20 countries are chosen based on the article count and sorted by
the high-IF ratio. Thresholds of IFs follow the description in Ch.5. Publication data is obtained from WoS with the search criteriaof publication year (PY) = 2010, subject category (WC) = “Biochemistry & Molecular Biology,” and document type (DT) =article, letter, proceedings paper, or review. Then, each article is assigned with IF based on JCR and is linked to countries basedon authors’ address (AD).

27

Portfolio of Publication Impact at Individual Level a

0

10

20

%High IF (>8)
30
40

50

60

Figure 2

0

10

20

30

40 50 60 70%Low IF (<2)

a80

90 100N = 352. The circle size represents the publication count of each respondent. Respondents who published five or fewer papers
in 2006-2010 are omitted.

28

Table 1

Determinants of Publication Quality in Three Stages

Key determinants

Priority in multiple

Stages of decision making
1: Mission

3: Publicationxx

Norms on qualityxx

Capacity for researchx

missionsx

2: Project

Examples in the Japanese context(Results from interviews and regression analyses)
 Student-authorship compromises publication quality for early graduation.
 Research orientation toward practical application decreases publication
quality, but it increases quality if actually applicable to industrial needs.
 Industry coauthorship results in secrecy and in publication intended
primarily for advertisement with compromised quality.
 International mobility facilitates normative assimilation; returnees fromforeign research experience avoid publishing low-quality papers.
 Inbred academics avoid engaging in challenging projects leading tohigh-quality publication.
 Input of research budget is essential for maintaining scientific production,but excessive budget facilitates low-quality publication.
 Input of human resources is also indispensable but too many members(PhD students) in a lab lead to low-quality publications for overcapacity.

29Descriptive Statistics (Individual Level) a

Table 2
Variable
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19

a

%IF<1
%IF<2
%IF>8
#Total pub#Total adjusted cite
#Adjusted cite before tenure
Univ rank
FemaleTime for research
Time for teaching
Time for administration
Applied research
Industry collaboration#Years since first degree
Foreign experience
Inbred#Staff#PhD/#staff
Budget/#member

Mean

SD

Min

Max

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

9.64
23.89
9.53
32.76
46.43
2.60
2.26
.03
3.69
1.70
2.14
1.70
.55
25.99
3.65
.10
4.31
.77
4.08

14.30
19.88
11.27
25.13
68.66
4.58
1.08
.17
1.51
.95
1.23
.90
.50
5.87
1.54
.31
2.64
.73
3.84

.00
.00
.00
5.00
.00
.00
1.00
.00
1.00
1.00
1.00
1.00
.00
12.00
1.00
.00
1.00
.00
.28

100.00100.0053.85
165.00880.87
45.24
4.00
1.00
6.00
6.00
12.00
5.00
1.00
45.00
6.00
1.00
20.00
4.33
37.50

.706
-.228
-.169
-.189
-.141
-.085
.007
-.152
.074
.229
.001
-.175
.054
-.169
.108
-.147
.028
-.122

-.439
-.166
-.291
-.249
-.175
-.004
-.208
.137
.254
.153
-.151
.106
-.128
.112
-.246
.066
-.226

.070
.313
.335
.195
.000
.295
-.038
-.133
-.174
.112
-.195
.063
-.137
.261
-.029
.159

.644
.065
.230
-.109
.043
-.057
-.086
.111
.381
.084
-.091
.085
.471
.074
.078

.297
.132
-.062
.109
-.077
-.112
-.071
.271
-.056
-.103
.037
.354
.039
.091

.083
-.026
.127
-.092
-.114
-.085
.049
-.272
-.093
-.044
.172
-.064
.088

-.042
.098
-.174
-.032
-.143
.112
.049
-.047
.209
.312
.090
.015

.100
-.029
-.020
-.049
.029
-.046
.072
-.060
-.063
.022
-.051

-.111
-.431
-.191
.075
-.038
.074
-.061
.107
-.050
.204

.098
.079
-.059
-.056
.007
.002
-.085
.004
.041

.111
-.031
.056
-.014
.055
-.083
.080
-.154

.177
.084
.046
-.018
.004
.115
-.008

.079
-.041
.018
.234
-.009
.043

.226
.080
.028
.041
-.055

-.121
.024
.040
.014

.011
.047
-.032

-.243
.074

-.227

N = 365. Bold italic is significant (p < .05). Respondents who published five or fewer papers in 2006-2010 are omitted.

30

Prediction of Publication Quality at Individual Level a

Table 3

#Adjusted cite before tenure

Model 1
%IF<1
-.21

(.17)

Model 2%IF<2-.53 *(.22)

Model 3
%IF>8
.54 ***

Univ rank

-.61

(.75)

-1.32

(.97)

1.14

Female2.36(4.43).28(5.70).25

Time for research.01

Time for teaching.47

Time for administration
Applied research

2.22

***

.03

Industry collaboration#Years since first degree
Foreign experience
Inbred-4.76(1.54)(.49)(2.65).19(#PhD/#staff)
Budget/#member

(.66)

(.31)

-1.25
2

1.56

(2.43)-.28#PhD/#staff

(.78)

(.13)
***

3.77

#Staff

.15

(.87)
**

.17
-1.64

(.56)

(.89)

2.79

**

3.06

**

-4.71

*

.30

†

-1.90

**

5.58

†

-.94

*

-5.00

(.71)

1.22

(1.01)

.40

(.85)

.27

(1.12)(1.98)(.17)(.63)(3.12)(.40)(3.41)

1.17(1.14)(1.15)-4.39(3.40)(3.14)-9.72(6.78)-14.48(20.01)

(.39)

-.60

(.85)

.09

(2.51)(.55).07(1.20)-.82(3.53)

(.47)

-1.60

(1.01)

-2.57

(.62)

.79

(1.09)

12.98

(.09)

.27

-.27

**

.59

†

(.35)

-1.79

-4.47

**

(1.72)

3.72

.65

**

4.70

*

-1.12

†

1.08

***

(.28)

1.40

-.03

***

(.01)

-.03

*

4.08

***

(1.88)

9.63

*

(.63)

-.97

(.22)

-9.17

(3.92)(2.36)

28.19***(6.96)(.20).07(1.33)
***

(.75)

-4.08

(3.71)

8.12

(.60)
†

(10.97)

8.03(1.41)(4.05)

25.20*(11.96)(1.36)

-2.69

(.48)

(4.00)

F test4.06 ***

8.09 ***

9.72 ***

12.33 ***

7.85 ***

.123

.244

.284

.341

.238

352

352

352

352

352

2

(.02)

a

(2.22)

***

.02

(.51)

*

(2.97)*(Budget/#member)

Adjusted R
N

.06

1.33

.69

†

(.01)

**

**

(.53)

-.88

(.39)

-2.36

**

(.12)

Model 5
#Total adjusted cite
3.19 ***
(.77)

*

2

***

-1.60

*

Model 4
#Total pub
-.19
(.26)

(.60)

3.82

(.02)

-.10

*

(1.77)(.06)

Unstandardized coefficients and robust standard errors (parentheses). Two-tailed test. † p<0.10; * p<0.05; ** p<0.01;*** p<0.001. Ordinary least squares. Respondents who
published five or fewer papers in 2006-2010 are omitted.

31Prediction of Publication Impact at Article Level a

Table 4(A) Base Model

Model 1
IF<1
#Adjusted cite before tenure
Univ rank-.074 *

Model 2IF<2(.029)-.061 ***

Model 3IF<8(.016).044 ***(.443)-.105(.302)-.023(.002)-.020 ***(.001)-.009 ***(.001)(.016).092***(.012).306***(.008).145***(.009)(.042).126 **(.049).086 **(.030).099 *(.044)(.060).061(.071).051(.045)-.013(.065)(.058)(.554)-.077(.363)-.002(.005).011 ***(.001)(.028)-.160***(.065).010(.091).077.115

Time for administration.193

Applied research

.059

**

(.062).001.161#Authors

Time for teaching(.014)**-.009 ***

Female.002.093 ***.163(.058)-.154(.010)(.042)-.061

Time for research.102 ***.011(.090)#Organizations

Model 5
Impact Factor(.067)-.088***(.014)

Model 4
#Adjusted Cite

.169(.074).138**(.050).004(.068).031(.040)-.015(.098).230 ***(.064)-.237 **(.080)-.132 **(.047)-.326 ****

Industry coauthor-.017(.161).073(.096).035(.114).272

Student first author-.156(.108)-.174 *(.068).044(.089).030#Years since first degree.020

Foreign experience-.179

Inbred.408#Staff-.070#PhD/#staff
2(#PhD/#staff)
Budget/#member(Budget/#member)2

χ testObservation
N

†

.011

(.059)-.106(.270).259(.036)-.074(.010)
**

**

-.026

*

(.007)(.136)(.102)-.030**(.011)**(.042).073(.045)-.037(.029).132-.832 ***(.238)-.093(.131)-.563 **(.194)(.023).079**(.015).135***(.023)**(.268).247(.148).814***(.215)(.100)-.009(.048)-.191 **(.071)(.051).045*(.023).211***(.033)(.003)-.001 †(.001)-.005 ***(.001)(.310)-.311(.199).826(.103).055(.066)-.251 *

.003 †

-.003

.255
.044

(.069)(.039).018*(.012)(.110)(.082)†(.180)

-.254
-.119
2

(.016)**(.418)(.048)

-.155

***

***

(.031).187(.002).004 ***(.001)-.007 *(.024).023†

105.57 ***241.28 ***199.00 ***2230.60 ***637.26 ***

11385

11385

11385

11386

11385

366

366

366

366

366

a

Unstandardized coefficients and robust standard errors (parentheses). Two-tailed test. † p<0.10; * p<0.05; ** p<0.01;*** p<0.001. The random effects of academics arecontrolled for. Ordinary least squares in Models 4-5 in Table 2A, and logit regression in all other models. Only focal variables are presented in Tables B and C for parsimony.

32

(B) Effect of Industry Coauthorship
Model 1
IF<1
Industry coauthor
Industry collab but not coauthor

-.372
-.383

Model 2IF<2(.237)*(.188)

-.125
-.214

†

Model 3IF<8

Model 4
#Adjusted Cite(.149).272(.185).351 *(.138)(.124).255(.156).093(.098)(C) Effect of Student First Authorship

Univ rank

Model 1IF<1-.056(.092)

Model 2IF<2-.057(.059)

Model 3IF<8.134 *(.069)

Model 4
#Adjusted Cite.008(.045)

Student first author

.286

-.113

(.178)-.583 *(.270)-.024(.217)(.064)*(.088).021(.077)

Student first author x Univ rank-.179(.268)†(.101)

-.024

33

.220