Institute for Economic Studies, Keio University
Keio-IES Discussion Paper Series

Evaluating Professor Value-added: Evidence from Professor andStudent Matching in Physics
菊地 雄太、中嶋 亮

2017 年 3 月 24 日
DP2017-007
https://ies.keio.ac.jp/publications/7448/

Institute for Economic Studies, Keio University
2-15-45 Mita, Minato-ku, Tokyo 108-8345, Japan
ies-office@adst.keio.ac.jp
24 March, 2017

Evaluating Professor Value-added: Evidence from Professor and Student Matching in
Physics
菊地 雄太、中嶋 亮

IES Keio DP 2017-007
2017 年 3 月 24 日
JEL Classification: D83; I23; J24
キーワード: knowledge creation; postgraduate education; faculty quality;
research apprenticeship【要旨】

This paper estimates a professor's value added to a postgraduate student's researchachievement growth using unique panel data on matched advisor-advisee pairs in aworld-leading physics graduate program. To address an identification problem related
to the endogenous selection of advisors and advisees, we use professor turnover andestimate a semi-parametric lower bound of the variance in advisor quality affecting
advisee research performance. We find that a one-standard-deviation increase inprofessor quality results in a 0.54 standard deviation increase in a doctoral student's
research achievement growth, increasing the number of first-authored papers that are
published in top journals by 0.64 at the doctoral level.

菊地 雄太
北海道大学 大学院経済学研究院〒060-0809
札幌市北区北9条西7丁目

yuta.kikuchi@econ.hokudai.ac.jp.
中嶋 亮
慶應義塾大学経済学部〒108-8345
東京都港区三田2-15-45

nakajima@econ.keio.ac.jp

謝辞：本研究は、科学技術研究費補助金・基盤研究（C）（2014 年 4 月-2018 年 3 月）
研究課題「大学における知的相互作用が学術研究および科学技術の革新に与える効果の定
量分析」より研究助成を受けている。心より感謝申し上げる。

Evaluating Professor Value-added: Evidence from
Professor and Student Matching in Physics ∗Yuta Kikuchi† Ryo Nakajima‡
March 24, 2017

Abstract
This paper estimates a professor’s value added to a postgraduate student’s researchachievement growth using unique panel data on matched advisor-advisee pairs in aworld-leading physics graduate program. To address an identification problem related
to the endogenous selection of advisors and advisees, we use professor turnover andestimate a semi-parametric lower bound of the variance in advisor quality aﬀecting
advisee research performance. We find that advisor quality, operationalized as the ability
to enhance an advisee’s research achievement, varies substantially. A one-standarddeviation increase in professor quality results in a 0.54-standard-deviation increase in adoctoral student’s research achievement growth, increasing the number of first-authoredpapers that are published in top journals by 0.64 at the doctoral level.
Keywords: knowledge creation; postgraduate education; faculty quality; research apprenticeship
JEL codes: D83; I23; J24∗

This research was supported by Grants-in-Aid for Scientific Research Nos. 14J08184 and 26380313 fromthe Japan Society for the Promotion of Science. The authors thank Donna Ginther and Daiji Kawaguchi forsupportive comments and helpful suggestions for improving the manuscript. We are grateful to Kenichiro Aoki
and Kensuke Kobayashi for providing helpful comments on postgraduate education and academic culture in
Japanese physics communities, especially regarding the the University of Tokyo physics department.
†

Graduate School of Economics and Business Administration, Hokkaido UniversityKita 9 Nishi 7, Kita-ku,
Sapporo, 060-0809 JAPAN. E-mail: yuta.kikuchi@econ.hokudai.ac.jp.
‡
Corresponding Author: Department of Economics, Keio University, 2-15-45 Mita, Minato Tokyo, 1088345, JAPAN. E-mail: nakajima@econ.keio.ac.jp.

11Introduction

1.1

Overview

How is knowledge created? Economists have had a particular and long-standing interest in
knowledge production. Indeed, the new economic growth theory literature regards the way
in which knowledge is created and accumulated as crucial for a nation’s growth (Lucas 1988;
Romer 1990; Grossman and Helpman 1991).
Recently, there has been an increase in the number of empirical studies on knowledge
creation in the field of science and technology. They have investigated how an individual’s
knowledge creation is aﬀected by knowledge created by others, with a particular emphasison knowledge spillovers between individuals within and across institutions. The evidence
obtained thus far, however, has been mixed. Some studies (e.g., Azoulay, Zivin and Wang
2010; Borjas and Doran 2014; Moser, Voena and Waldinger 2014) provide evidence in favor of
positive knowledge spillovers, while others (Borjas and Doran 2012; Waldinger 2012) do not.
Surprisingly little attention has been devoted to knowledge reproduction processes acrossgenerations. As is often argued, scientific and technological knowledge is tacit (Polanyi 1958,
1966). It is not easily translated and thus needs to be intentionally articulated, codified anddiﬀused. Therefore, knowledge has long been reproduced through a deliberate process ofeducation and learning, whereby those with knowledge take voluntary action to pass it on to
those who do not. Thus, to investigate the creation and diﬀusion of scientific and technologicalknowledge, it seems natural to distinguish vertical knowledge flows (i.e., knowledge flowing
from an individual with high expertise to one with low expertise) from horizontal knowledge
flows (i.e., knowledge flowing among individuals with the same level of expertise). If thetwo lines of knowledge flow diﬀer in the eﬃciency with which know-how is transmitted, such
diﬀerences might explain the mixed results obtained by prior studies with respect to theextent of spillovers.
This paper focuses on the knowledge reproduction process whereby knowledge is conveyed
through vertical relationships, including master-apprentice, teacher-student and senior-juniorcollaborator relationships. We specifically focus on the advisor-advisee relationship in postgraduate education to examine its eﬀectiveness in expanding scientific frontiers. Under the
hypothesis that a professor’s “quality” has a consequential impact on the growth of a student’s
research achievement, we estimate the professor’s (advisor’s) value added as the contribution
to a student’s (advisee’s) progress on research outcomes.
Empirically estimating a graduate school professor’s value added is complicated by theendogenous selection process involving students, professors and schools. We anticipate thatpromising students will apply and be admitted to highly ranked schools. Moreover, professors

2with good academic standing are likely to have faculty positions at prestigious schools. Thesetypes of selective recruitment will lead to nonrandom sorting of students and professors acrossgraduate programs. Furthermore, the existence and extent of sorting can be reinforced by theadvisor-advisee matching process within a school whereby students will choose and be chosen
by faculty members.1
This paper estimates within-school professor value added at a world-leading postgraduate
program in physics in Japan. To disentangle the influence of professors on students fromthe sorting and matching eﬀects, we use an identification strategy that exploits professor
turnover from events such as retirement, relocation, or death. We borrow this idea from
Rivkin, Hanushek and Kain (2005, henceforth, RHK), who estimate a lower bound of theteacher quality eﬀect on student achievement gains. Japanese graduate schools provide an
ideal setting for applying RHK’s strategy of turnover-based value-added estimation. Whenan advisor exits a Japanese graduate school due to turnover, the advisees usually remain in
the same program and continue their research projects under the supervision of new advisor.2
Therefore, the advisees who experience advisor turnover are influenced by two advisors of
diﬀerent quality. Thus, the student’s research achievement growth, under the varied influencesof diﬀerent advisors, would be more volatile than that of advisees who did not suﬀer advisor
turnover.
Figure 1 shows the distribution of students across initial advisors (on the vertical axis)
and across cohorts (on the horizontal axis) for the physics graduate program that we use as
an empirical testing ground. We base our analysis on a lab, defined by a cohort of students
who were assigned to the same advisor. The red square and blue circle markers represent
a lab in the treatment group where the advisor was replaced due to turnover and a lab in
the control group where the advisor was not replaced, respectively. We demonstrate thatcohort-to-cohort variation in the lab average of student research achievement gains is larger
for the treatment group than for the control group and is driven primarily by the change inadvisor quality following turnover.
Insert Figure 1
Certainly, factors other than advisor quality might aﬀect an advisee’s research performance. This paper employs a semi-parametric education production function, which is widelyused in the economics of education literature, that attributes the student’s achievement gains
to various fixed eﬀects. Repeated observations of an individual student’s research outcomes,
1

These choices give rise to “assortative matching” between students and professors within a school withrespect to research ability.
2
There might be a concern that the original advisor who had left the program continued to provide researchguidance to the students who remained in the program. In other words, the advisor listed as a thesis advisor
might be a “surrogate” of the true advisor. We address this concern in Section 5.2 and show that ourlower-bound estimate of the variance of advisor quality remains valid in such cases of “surrogate” advisors.

3

which are measured by publication records, in master’s and doctoral degree programs enable
us to eliminate student fixed eﬀects by taking the diﬀerence of the research outcomes of agiven student from one degree program to another.
The estimation results provide strong evidence for the existence of professor value added.
Indeed, the results consistently demonstrate that advisors vary greatly in quality, measured
as their ability to enhance an advisee’s research achievement at the doctoral level, whichis consistent with the expectation that knowledge and ideas are transmitted vertically fromadvisor to advisee. Specifically, our estimates indicate that a one-standard-deviation increasein advisor quality will increase a doctoral advisee’s research achievement by 0.54 standard
deviations. We also find that a one-standard-deviation increase in advisor quality entails an
increase in the number of articles published by a doctorate student in top journals as a firstauthor by 0.64.
The findings of this paper are robust to diﬀerent definitions of student research outcomes
and are also insensitive to many diﬀerent model specifications. Notably, we continue to findevidence of substantial professor value added even in a severe case for students in which theirresearch contribution to the published papers is forced to zero whenever they collaborated
with a advisor. We also find that the results are robust to a falsification exercise that examines
whether the timing of the increased variability in the student research achievement gain agrees
with that of advisor turnover, as predicted by the empirical model.
We also examine the robustness of the estimation results under a weaker conditional independence assumption on advisor switches. Our identification strategy for professor value
added relies on the assumption that an advisor switch due to turnover is incidental, that is, isorthogonal to an advisee’s unobservables, conditional on the advisor’s observable characteristics. However, this assumption may not hold. There is a concern that the advisor switch (andnon-switch) is intentional in the sense that students, having prior information on “scheduled”
turnover of faculty members, self-select themselves into or out of specific labs. To address
this concern, we restrict turnover events to those that seem incidental rather than intentional.
Since students would hardly be able to predict their advisor’s relocation or death ex ante, anadvisor switch due to such non-retirement reasons is deemed more likely to be idiosyncratic.
It thus does not seem unrealistic to assume that such a switch is orthogonal to unobserved
student characteristics that aﬀect research performance. We estimate the regression modelusing only turnover due to non-retirement-based reasons and still find a strong advisor qualityeﬀect.
We finally investigate alternative mechanisms for knowledge transmission other than thatbased on learning through the advisor-advisee relationship. The data indicate that advisor
turnover does not have a significant unidirectional, positive or negative, impact on an advisee’s
research achievement gain, per se, as is consistent with the mechanism that our value-added4

model postulates, and is thus not fully explained by the other mechanisms such as thatemphasizing (i) a disruption eﬀect of advisor turnover or (ii) a recombination role of various
extant pieces of knowledge, Further analysis reveals that the eﬀect of knowledge transmission
from advisor to student within a lab tends to outweigh that from non-advisor to student
across labs.
While we find a significant eﬀect of advisor quality on advisee research performance growth,
we concede that this finding may be specific to the local contexts of the physics disciplineor the postgraduate education system in Japan. However, at a minimum, in a process ofscientific inquiry in which prominent researchers, including several Nobel laureates, have been
involved, advisors matter, and their quality of supervision varies substantially. Our findingson the advisor eﬀect speak to a broad range of studies that evaluate the heterogeneous impactof single individuals on organizational performance.3 For instance, the recent study by Lazear
et al. (2015) analyzes micro-data from one large firm and provides supportive evidence that asupervisor aﬀects the productivities of the workers on his or her team. This appears to agreewith our finding that an advisor significantly influences the performance of advisees in the labthat he or she supervises. Further research needs to be conducted and accumulated to explore
whether similar supervisor eﬀects can be observed in other institutions and organizations inboth academia and industry.
The remainder of the paper proceeds as follows. A brief literature review is provided in theremainder of this section. Section 2 describes the institutional background of postgraduate
physics education in Japan. Section 3 presents the empirical model and describes a regressionbased approach to estimate the lower bound of professor value added. Section 4 explains thedata set used for the analysis. Section 5 discusses some empirical issues concerning valueadded estimation. Section 6 presents the estimation results and provides robustness checks.
Section 7 concludes.

1.2

Related LiteratureThis paper contributes to the literature by measuring the eﬀectiveness of professors in promoting students’ research productivity growth at a postgraduate institution. The most closely
related work to this paper is Waldinger (2010), who estimates the causal eﬀect of prominent
professors on the research outcomes of Ph.D. students in mathematics at German universities
during the Nazi era. Although we share his view that “university quality is believed to be oneof the key drivers for a successful professional career of university graduates” (Waldinger 2010,
p.787), we highlight the importance of direct interactions between advisor and advisee as a3See, for example, Jones and Olken (2005) for political leaders; Malmendier and Tate (2009) for chief
executive oﬃcers; Branch et al. (2012) for school principals; and Lacetera et al. (2016) for auctioneers.

5medium whereby knowledge is memorized, transferred and accumulated. Indeed, anecdotal
evidence (e.g., Zuckerman 1977) suggests the importance of vertical social ties in scientificenterprises at academic institutions. However, to the best of our knowledge, no systematicquantitative study, especially one that carefully controls for endogenous matching between
master (teacher, advisor or senior collaborator) and apprentice (student, advisee or juniorcollaborator), has been conducted to date.
Our findings validate the view of earlier studies (e.g., Azoulay, Zivin and Wang 2010;
Borjas and Doran 2014; Moser, Voena and Waldinger 2014) that vertical social interactions
among scientists are enduring and important for scientific and technological knowledge to becreated and diﬀused. For example, a recent study by Moser, Voena and Waldinger (2014),
who estimate the eﬀect of German Jewish émigrés on U.S. innovation, suggests that knowledge externalities occurred and were amplified by the educational and collaborative ties inscientist networks such that U.S. junior scientists were trained by and collaborated withprominent Jewish senior scientists who emigrated. Borjas and Doran (2014) study the impact of the influx of Soviet mathematicians into the United States after the collapse of theSoviet Union and conclude that positive knowledge spillovers are generated through the relationships among collaborating mathematicians who regularly interact when at least one ofthem is an outstanding knowledge producer.
This study is also related to a voluminous education economics literature that evaluatesteacher value added (e.g., Hanushek and Rivkin 2006, 2010). We base our empirical analysis
on the value-added model approach that is widely employed in the literature. Specifically, as
mentioned above, we adopt a semi-parametric value-added model and employ the turnover
estimator proposed by RHK. However, we depart from the previous literature on teacher
value added in that we focus on value added at a level higher than secondary education.
Although numerous studies estimate value added at the primary and secondary educationlevels (e.g., Hanushek and Rivkin 2012, for a recent review), few studies (e.g., Hoﬀmann andOreopoulos 2009; Carrell and West 2010) estimate a professor’s value added in the context of
post-secondary institutions. While these studies on professor value added attempt to estimate
the eﬀectiveness of professors in improving students’ grade gains at the undergraduate level,
we turn to professors’ value added to students’ research achievement gains at the postgraduatelevel and thus evaluate the eﬀectiveness of professors in terms of their “quality” in advisingor mentoring graduate students’ research projects.
To the best of our knowledge, no studies assess the impact of professor quality on graduate
student research productivity growth by shedding light on the value-added contribution. A
partial exception is the study by Hilmer and Hilmer (2009), who find a positive eﬀect of anadvisor’s research prominence on advisees’ early career publication success in U.S. economics
Ph.D. programs. While they are successful at disentangling the eﬀect of advisor quality from6

that of program quality on Ph.D. students’ publication outcomes, they do not address endogenous advisor-advisee matching between professors and students within and across institutions.
Thus, it seems questionable to interpret their finding of a positive correlation between the
research productivity of advisors and advisees as causal.

2

Institutional Background2.1

Postgraduate Physics Education in Japan

Postgraduate education in Japan, including in physics, has a two-tiered structure, that is, a
two-year master’s degree program followed by a doctoral program that typically lasts three orfour years.4 Leading Japanese research universities typically oﬀer both master’s and doctoral
courses. In most cases, students enrolled in a doctoral degree program graduate with amaster’s degree from the same school. However, they are institutionally separated. Thus, amaster’s student seeking to pursue a doctorate must take an entrance examination, which islargely based on a master’s thesis, to be admitted to a doctoral course even if it is oﬀered by
the same institution. In a sense, the master’s degree program implicitly serves as a screening
device for doctoral programs in Japan.
Three features are notable for graduate education in physics for master’s programs inJapan. First, Japanese physics master’s students are closely linked to their faculty advisors
immediately after enrollment in a program. Indeed, applicants to a master’s degree program
must declare their desired field of specialization and submit a short list of faculty advisorsfrom whom mentorship is sought upon admission. Only those who are approved for support
by designated advisors are admitted to a graduate school.5
Second, physics education in Japan at the master’s level is best characterized by researchbased apprentice training, which is often contrasted by coursework-based training in the U.S.
(Abe and Watanabe 2012). Although Japanese master’s students in physics are required totake some “coursework” credits toward their degrees, they can earn most of their credits
through learning-by-doing style research “seminars” taught by a faculty advisor.6
Finally, for Japanese physics graduate students, a thesis is required to complete the master’s program. It is expected to be original, as a doctoral thesis should be, although they areevaluated according to diﬀerent criteria of scholarly maturity. Students are encouraged to4

The basic structure has remained unchanged since World War II, although the organizational structureof universities has been reformed (see Ushiogi 1993; Ogawa 2002)
5
This contrasts with U.S. graduate students, who are matched with their supervisors through the rotationof faculty labs after they complete their coursework and become Ph.D. candidates (see Gumport 1993).
6
For example, for the master’s degree program in physics at the University of Tokyo, students must take atleast thirty credits of coursework at the master’s degree level. However, lab-based research “seminars” oﬀered
by thesis advisors constitute two-thirds of their total credits.

7begin original research in their chosen fields at an early stage of the master’s degree program
under the instruction and guidance of a faculty advisor. Because the master’s thesis is a
critical factor for admittance to doctoral programs, Japanese students and professors attach
great importance to a master’s thesis as a pathway to doctoral study.
In contrast, the doctoral programs in physics at Japanese universities are more similar
to their counterparts in Western countries than are the master’s programs. Specifically,
Japanese doctoral students and American Ph.D candidates are considered comparable inthat there is no coursework requirement. Japanese students at the doctoral level, similar
to Ph.D. candidates in the U.S., begin the research for their doctoral dissertations under
the supervision of their research advisors. In general terms, Japanese physics students are
required to write several articles published in refereed journals as a prerequisite for a doctoral
degree. These publications are usually included in a doctoral thesis.

2.2

Physics Labs in Japanese Universities

The interaction between a graduate student and a faculty advisor is lab-oriented in Japanesephysics graduate programs. Upon enrollment in the master’s program, Japanese physics
students are assigned individually to a lab, and the lab’s leader (or sometimes sub-leader)
becomes their thesis research advisor. Students acquire the knowledge necessary to conduct
their research through frequent interaction with their advisors in a lab setting. The content of
this lab-based teaching and learning includes basic research skills, such as how to read scientificarticles, how to select research topics, how to present results at seminars and conferences, andhow to write publishable papers, as well as the culture of physics such as the style of work,
mode of thought, and a taste for “good” physics (Abe and Watanabe 2012).
Japanese physics labs are generally democratic in tone. The “laboratory democracy” inJapanese physics communities can be traced back to the end of World War II, the period
when there were immediate and insistent calls for the creation of a new “scientific Japan”
under the control of the allied occupation (Low 2005). To place this in perspective, it isbroadly understood that Japanese physics labs are less prescriptive and less hierarchical thantheir U.S. counterparts (Gumport 1993; Kawashima and Maruyama 1993). There is also nostrict division of labor among lab members, even between faculty members and graduate
students, in Japanese physics labs (Traweek 1988).7 Hence, although it is not uncommon for
the research topics of master’s and doctoral theses to be suggested by advisors as a part of alarge, ongoing project in a given lab, Japanese physics graduate students are, generally, given
some autonomy to pursue their own research based on their original ideas.
7Interested readers should consult Appendix A that explains in detail the “laboratory democracy” among
Japanese physicists.

8

3

Empirical ModelIn this section, we introduce a simple value-added model that associates growth in student
research achievement with the “quality” of the professor supervising the student. Then, we
present a regression-based approach to estimate a lower bound of the variance in professor
quality, which can be interpreted as the extent to which any professor diﬀerences matter indetermining student research outcome growth.

3.1

Value-added Specification

Following the standard value-added modeling approach (e.g., Hanushek and Rivkin 2010), we
employ a semi-parametric specification of a professor’s contribution to a student’s achievement
growth.
Consider graduate student i who entered the master’s program of a graduate school inyear c. Below, we treat year c as the student’s cohort. We denote the research outcome
growth of a graduate student in the master’s degree program by g = m and in the doctoral
degree program by g = d. The growth is measured by the gains in research output from theprevious degree program to the current degree program.8 Let ∆outcomeciag be the researchoutcome growth of student i under the supervision of professor a ∈ A in degree programg ∈ {m, d} in cohort c ∈ C . We assume that it is given by the following function:c∆outcomeciag = γi + θag + νiag,(1)

where γi is student i’s individual fixed eﬀect, θag is professor a’s quality that influences thecstudent’s research outcome growth in degree program g, and νiag
is an idiosyncratic random

shock.9
We assume that matching between student and professor is many-to-one, that is, multiple
students are assigned to one advisor. Let us define a lab as a group of students (advisees)in the same cohort who were assigned to the same professor (advisor). Specifically, we useℓ(a, c) to denote a lab in which students are in cohort c and assigned to professor a as anadvisor. Let L be the number of all labs in a school, and let students in lab ℓ(a, c) be
indexed by i = 1, · · · , I ℓ(a,c) , where I ℓ(a,c) is the number of students in lab ℓ(a, c). We use
I ℓ(a,c) ≡ {1, · · · , I ℓ(a,c) } to denote the set of students in the lab.
8

We assume that the research output of students at the bachelor level is zero. We compute a publicationbased research proficiency score, which is explained in detail in Section 4, for students in the sample when
they are undergraduate students and find that it is negligible.
9Note that other eﬀects, such as school fixed eﬀects and research field fixed eﬀects, are not included in thevalue-added model. We opt not to include these fixed eﬀects because they are subtracted out of the estimation
model in the process of “diﬀerencing”, as presented below.

9

We take the average of Equation (1) over all students in the same lab ℓ(a, c). Because thestudents in the same lab have the same advisor quality, we have the following equation forthe lab-level average of student research outcome growth:ℓ(a,c)∆outcomeagℓ(a,c)= γ̄ ℓ(a,c) + θag + ν̄ag,

(2)

where the overbar notation indicates the group average.
Note that the superscript a denotes the initial advisor to whom the students in lab ℓ(a, c)
were assigned, while the subscript a denotes the advisor who supervised the students in degree
program g. Thus, the advisors represented by the superscript and subscript could be diﬀerent.
For example, suppose that a turnover incident causes the students in lab ℓ(a, c) to switch theirresearch advisor from professor a in the master’s degree program to professor b in the doctoral
degree program. Here, the average student research outcome gain at the doctoral level, whichℓ(a,c)

is the left-hand side of Equation (2), is given by ∆outcomebd, where the index a in the

superscript diﬀers from the index b in the subscript.
We use the event of professor turnover (e.g., retirement, relocation and decease) to identify
the variance in professor quality. We implicitly assume that when a professor exits a graduate
program due to turnover, the students in the lab whom he or she initially supervised are
re-assigned to a new advisor and continue their research projects in the same program.10 In
what follows, we therefore assume that an event of professor turnover on the faculty side leads
to an event of advisor switch on the student side. In other words, we treat these two events,
advisor turnover and advisor switch, identically. When advisor turnover occurs in a lab, twofaculty members, whose quality levels are generally diﬀerent, will have advised the students
in that lab.11
It should be noted that the professor, say b, who was assigned to the students in the lab ofa professor, say a, after the latter exited due to turnover was not necessary drawn at random
from a pool of professors available at the school at that time. Indeed, the newly assigned
professor might select the students that he or she is willing to take over. We thus allow thestudent fixed eﬀect, γi , to be correlated with the quality of the re-assigned professor, θbd , inthe same way as we assume it to be correlated with the quality of the original advisor, θad .

10

A joint transfer of faculty and students is quite rare in Japanese universities, and hence, even if a faculty
member changes aﬃliation, the students usually remain in the same program.
11
Based on the observed pattern of advisor replacement in our data, when advisor turnover occurred, thestudents were usually either assigned to a junior faculty member or the sub-leader of the same lab or they
were moved to a diﬀerent lab in closely related research fields within the same institution and were supervisedby the faculty member who managed that lab.

10

3.2

A Lower-bound Estimation of the Variance in Advisor Quality

We are interested in estimating the variance of advisor quality, σg2 , rather than the the advisor’s individual quality, θag . Although the individual value added would be useful to answer
important questions concerning which or which types of professors tend to have higher orlower eﬀectiveness in research supervision, such estimation requires more extensive data andstricter identification assumptions than we have employed in this study.12 We therefore set
more modest goal, that is, to measure a minimum degree, if any, to which each professor
contributes to the performance growth of his or her lab students. For this purpose, we decompose the total variation in student outcome gains into the variation that can be attributed
to professor quality, θag . First, take the diﬀerence of Equation (2) between the master’s degreeand doctoral degree programs. Doing so eliminates the student fixed eﬀect, γi , because it isconstant across degree programs for a given student. If advisor turnover did not occur in labℓ(a, c), it is given by the following between-degree diﬀerence equation:ℓ(a,c)∆outcomeadℓ(a,c)− ∆outcomeamℓ(a,c)= (θad − θam ) + (ν̄adℓ(a,c)− ν̄am).

(3)

In contrast, assume that there was advisor turnover in lab ℓ(a, c). As the students switched
their advisors from advisor a in the master’s program to advisor b in the doctoral program,the between-degree diﬀerence equation, corresponding to Equation (3), is given byℓ(a,c)∆outcomebdℓ(a,c)− ∆outcomeamℓ(a,c)= (θbd − θam ) + (ν̄bdℓ(a,c)− ν̄am).

(4)

Comparing Equations (3) and (4) shows that advisor turnover influences the developmentof student research achievement in diﬀerent ways. There is a clear diﬀerence in student
research outcome growth, which appears on the left-hand side of each equation, that responds
diﬀerently to a change in advisors due to the diﬀerence in degree-level advisor eﬀects, (θad −θam ) and (θbd − θam ), which are generally not equal. This plays a key role in the identificationof the eﬀect of advisor quality on student research outcome growth at each degree level.
The point is illustrated by Figure2, which depicts three labs with diﬀerent cohorts, c0 , c1
and c2 , whose initial advisor is professor a. In the figure, each lab is portrayed by a connectedline segment, which represents the two-year master’s degree program (the first half of thesegment) and the three-year doctoral degree program (the last half of the segment).13 Here,advisor turnover did not occur in labs l(a, c0 ) or l(a, c1 ) before cohort c2 , and hence, thestudents in these labs were supervised by the same professor, a, throughout both the master’s
12

See Chetty et al. (2014), for example, on techniques to estimate a reliable individual teacher value added.
For the ease of exposition, the labs’ cohorts are not overlapped in the figure, although this is not necessarily
the case in the actual sample.
13

11

and doctoral programs. However, in lab l(a, c2 ), professor a exited the school due to turnover,and professor b took charge of the doctoral students.
Note that, on average, the research outcome gains of lab l(a, c0 ) and l(a, c1 ) students are
the same, which is given by (θad − θam ), whereas, following advisor turnover, the average
student research outcome gain of lab l(a, c2 ), which is given by (θbd − θam ), could be betteror worse than that of the previous cohorts, depending on whether the supervising quality of
the newly assigned professor, b, is higher than that of the departing professor, a. In either
case, irrespective of whether the achievement growth is positive or negative, an instance ofturnover triggers a change in professor quality at the doctoral level and could thus result ina disparity in the between-degree research achievement gains between cohorts. We will use
the induced divergence in research outcome growth as evidence of an advisor’s impact on anadvisee.
Insert Figure 2
To improve the identification, we use the double-diﬀerencing approach as proposed by
RHK to estimate a lower bound of the variance in unknown teacher quality. We take thediﬀerence between Equations (3) and (4) with respect to cohort year. Let c′ denote thecohort before c, and let τ be the years between c and c′ . For professor a, consider two labs,′ℓ(a, c) and ℓ(a, c′ ). Let W ℓ(a,c,c ) denote a dummy variable indicating a change in advisor
due to turnover: it takes value one if professor a is replaced in lab ℓ(a, c) due to turnoverand zero otherwise. Without loss of generality, we assume that supervisor replacement isfrom professor a to professor b such that, if there were advisor turnover, the students would
have been supervised by two diﬀerent professors, a and b, in the master’s and doctoral degree
programs, respectively. Then, we have the following double-diﬀerenced (DD) average student
research outcome growth:ℓ(a,c,c′ )==DD∆outcome′′[∆outcomeℓ(a,c) − ∆outcomeℓ(a,c) ] − [∆outcomeℓ(a,c ) − ∆outcomeℓ(a,c ) ] if W ℓ(a,c,c′ ) = 1
bd
am
ad
pm
[∆outcomeℓ(a,c) − ∆outcomeℓ(a,c) ] − [∆outcomeℓ(a,c′ ) − ∆outcomeℓ(a,c′ ) ] if W ℓ(a,c,c′ ) = 0
ad
am
pd
pm

(θbd − θad ) + error term if W ℓ(a,c,c′ ) = 1(5)
error termℓ(a,c,c′ )
if W
= 0,

where the error term is a catchall random noise term that combines the average idiosyncraticerrors.
Equation (5) shows that all of the fixed eﬀects, except for doctoral-level advisor quality, areeliminated after the double diﬀerence is taken with respect to degree programs and cohorts.
The DD measure is more variable, on average, for the pair of labs with and without a change
12

′

′

in advisor (W ℓ(a,c,c ) = 1) than that for the pair of labs without such a change (W ℓ(a,c,c ) = 0).
The gap is attributable to a discrete change in doctoral-level advisor quality from θad toθbd due to advisor turnover. Note that advisors’ quality levels can be correlated with the′lab averages of student fixed eﬀects, γ̄ ℓ(a,c) and γ̄ ℓ(a,c ) , and they can also be correlated with
one another, that is, Corr(θad , θbd ) ̸= 0. In what follows, we ascribe the sample variation in
the DD measure as a series of variance and covariance components of advisor quality andidiosyncratic shocks.
The Assumption on Advisor Quality and Idiosyncratic Shocks
We make the following assumptions concerning the distribution of advisor quality.
assumption 1.1: The expectation and variance of advisor quality are given by E(θag ) =µg and Var(θag ) = σg2 , for any a ∈ A , g ∈ {m, d}, and c, c′ ∈ C .
assumption 1.2: The correlation of advisor quality across professors, a ̸= b ∈ A, is given
by Corr(θag , θbg ) = ρg , for any a, b ∈ A , a ̸= b, g ∈ {m, d} and c, c′ ∈ C .
The assumptions concern the stationarity of the advisor quality distribution, which characterizes the notion that the professors’ advising quality levels are drawn from a common
distribution for each degree type. It requires that the grade-program-specific mean and variance do not vary across cohorts and that the correlation with any given advisor is constant.
Specifically, we interpret µg and σg2 as the long-run mean and variance of the stationary distribution of advisor quality in degree program g within a school. The stationarity assumption
simplifies the estimation of professor value added because it reduces the number of parameters
to be considered.
Appendix B.1 presents the assumptions regarding the moments of the idiosyncratic shockc, the averageafter demeaning by each cohort. Let ν̄g be the average of the random shock, νiagof which is taken over all cohorts in each degree program, g, such that the demeaned random
ccshock is given by ν̃iag= νiag− ν̄g . We assume that the random shocks demeaned by cohort

are independent of turnover incidents (assumption 2.1). They can be serially correlated
between degree programs within a student (assumption 2.2) and between students in each
degree program if they are supervised by the same advisor (assumption 2.3). Otherwise, they
are neither cross- nor serially correlated (assumptions 2.4 and 2.5). Note that, even if thecdemeaned random shock, ν̃iag, is uncorrelated with others under assumptions 2.4 and 2.5, thecoriginal random shock, νiag
, is allowed to be correlated through the common mean factor, ν̄g .

13

The Regression Model
Finally, given the assumptions presented above, we square both sides of Equation (5) and takethe expectation conditional on the occurrence of turnover. We have the following result:14
[(
]
(
)
)
{
}
1
1
′
ℓ(a,c,c′ ) 2ℓ(a,c,c′ )E DD∆outcome=α|W+ l(a,c′ ) + 2σd2 (1 − ρd ) W ℓ(a,c,c ) , (6)ℓ(a,c)
I
I
where α is a composite term of variances and covariances of the demeaned random shocks.
Equation (6) provides a basis for estimating the variance in advisor quality at the doctorallevel. Using the cohort examples, c0 , c1 , and c2 , that are depicted in Figure 2 for illustration,
the squared diﬀerence measure of student research outcome growth, which is the right-handside of Equation (6), is greater for ℓ(a, c1 , c2 ) than that for ℓ(a, c0 , c1 ) by 2σd2 (1 − ρd ). We cantherefore ascribe the large sample variation of the right-hand side of Equation (6), if any, tothe variance in doctoral-level advisor quality, σd2 , unless the correlation coeﬃcient, ρd , is equal
to one.
We now present a regression model to obtain a lower-bound estimate of the variance of
σd2 . Consider the following:()2
DD∆outcomen = αXn + βWn + εn ,(7)where n = 1, · · · , N is the index of observations. Here, the unit of observation is eachelement of (a, c, c′ ) for any advisor a ∈ A and cohort c, c′ such that 0 < c − c′ ≤ τ , whereτ is the period over which the diﬀerence is taken.15 . Note that, analogous to Equation (6),′the covariate Xn = 2(1/I ℓ(a,c) + 1/I l(a,c ) ) is introduced into the regression. The random term
εn is interpreted as the prediction error between the expected and observed values of thedivergence measures, that is:
[

]
(
)2
(
)2
εn ≡ E DD∆outcomen Wn − DD∆outcomen .
Assume for a moment that the advisor switch indicator, Wn , is independent of the prediction error, εn . If the value of ρd were known perfectly, the OLS estimate β̂ in Equation (7)would provide a consistent estimate of σd2 through the following equation:{}β̂ = 2σ̂d2 (1 − ρd ) .

(8)

14

See Appendix B.2 for the derivation.
To obtain the double-diﬀerenced average of the research outcome gain, which is the(left-handside of)τ!
Equation (7), we take the diﬀerence between all cohorts within a period of τ years. As τ +1=2
(τ −2)!2!
15

τ !L
samples are created for each lab, the total sample size of the regression is given by N = (τ −2)!2!
, where L is
the total number of labs.

14

As the correlation is imperfect (ρd < 1), a lower-bound estimate of σd2 is given by the lastterm of the following equation:σ̂d2 =β̂β̂≥ .
2(1 − ρd )4

(9)In other words, a lower-bound estimate of the within-school variance of faculty quality at thedoctoral level is equal to the estimated coeﬃcient, β̂, of the regression model (7) divided by
four.

4

Data

We assemble data sets of professors and students in a graduate program in physics in Japan.
Among the numerous Japanese research universities that oﬀer both master’s and doctoral
programs in the field of physics, we focus on the graduate program at the University of Tokyo(henceforth, UTokyo), which is the oldest institution of its kind in the country and has enjoyedconsiderable prestige in the global academic community.16
The graduate program in physics at UTokyo consists of the department of physics as
its core and other physics-related research institutes on campus.17 The average number ofgraduates in recent years is 105.6 for the master’s program and 58.4 for the doctoral program18 .
At present, there are more than 130 full-time faculty members. Many subfields of physics
are covered by laboratories in UTokyo’s physics graduate programs, such as nuclear physics,
particle physics, condensed matter physics, and biophysics.

4.1

Data on Advisor and Advisee Pairs

To extract the information on matched advisor-advisee pairs, we use the master’s and doctoral
thesis catalogs for graduate students in UTokyo’s physics program.19 For each thesis entry
in the catalog, the available information includes the degree date, the title of the thesis, thename of the student, and the name of the faculty advisor who supervised the student.
We compile the thesis data for the students who obtained their doctoral degrees in thecohorts between 1970 and 2004 (35 years). Among all of the graduate students who werelisted in both the master’s and doctoral thesis catalogs, we restrict our attention to those16

According to several world university rankings, UTokyo has been in the top 10 in the discipline of physics.
The alumni include five Nobel laureates in physics as of 2015.
17
The institutes are the Institute of Cosmic Ray Research, (ICRR), the Institute of Solid State Physics(ISSP), and the International Center for Elementary Particle Physics (ICEPP).
18
These are the average figures over the period from 2010 to 2014.
19
The catalogs are available on the department’s website at http://www.phys.s.u-tokyo.ac.jp/TOSHO/
ronbun.html.

15

who earned doctorates within six years of enrollment. In addition, we restrict the analysis tothose who were supervised by faculty members with the ranks of full and associate professors
in the physics department or on-campus physics-related research institutions. There are 119
advisors (professors) and 1484 advisees (students).20 The average number of students is 1.5
for each lab and 12.5 for each advisor in the sample.21

4.2

Data on Advisor Turnover and Switch

We obtain information on faculty turnover from the University Personnel Directory Book(“Zenkoku Daigaku Shokuin Roku”) published by Koujyun Sha, which includes informationon the full name, rank, department, school, specialized fields and year of birth of all staﬀ
members at every Japanese university, public or private, in a given year. By compiling theroster of faculty members at UTokyo, we can obtain their turnover information.
We classify the reasons for turnover into the following three categories: (1) retirement if theinstance of turnover occurred at the mandatory retirement age predetermined by UTokyo;22(2) move if turnover occurred before the retirement age and the faculty name began to reappear on other universities’ roasters beginning in the year after the turnover instance; and(3) decease/quit otherwise.23 Figure 3 presents the graphs that plot the number of turnoverincidents in each year of the sample period, broken down by the reasons.24
Insert Figure 3
The matched advisor-advisee data reveal that approximately 14.4 percent of graduate
students switched advisors between the master’s program and the doctoral program. Instancesof professor turnover are responsible for some, although not all, of the students’ observedchanges in advisors. As mentioned previously, in Japanese universities, a joint transfer of a20

We restrict advisors in the sample to those who supervised at least two cohorts of students within three
years because, as shown in later sections, the observation unit of the baseline regression is set as the diﬀerence
between two consecutive cohorts within three years.
21
It should be noted that the reported average numbers of students per lab and per professor are undersized
relative to the actual sizes that make up the graduate program because the estimation samples are restricted
to the students who took both master’s and doctoral degrees in physics at UTokyo.
22
Before fiscal year 2000, the mandatory retirement age at UTokyo was 60. After the 2001 fiscal year, it
was increased by one year every three years until it reached 65. As of 2004, which is the end of the sample
period, the retirement age was 61.
23
Note that the reasons for faculty turnover are not perfectly distinguishable. Indeed, the majority offaculty members categorized as “retire” did not actually retire from academic life and were reemployed atother universities or research institutions. This is possible because of the gap in retirement ages between
universities: UTokyo set its faculty retirement age at 60 during the most of the sample period, while otherJapanese universities, public and private, adopted retirement ages that were several years older.
24
There is a considerable number of incidents in 1997, when the Institute for Nuclear Study (INS) at
UTokyo, which was one of the on-campus research institutes aﬃliated with the physics department, wasclosed and merged with the High Energy Accelerator Research Organization (also known as the KEK (Kō
Enerugī Kasokuki Kenkyū Kikō.), and some of the faculty members at the INS chose to leave UTokyo for the
KEK.

16

faculty member and student is quite rare. If a faculty member exits a graduate program,
another other faculty member – usually a sub-leader of the same lab or, sometimes, a faculty
member from a diﬀerent lab in the same institution whose research area is closely related tothe professor who exited – becomes the new advisor of the students who are left behind. In
either case, the student remains in the same program.
We identify an advisor switch due to turnover if a student’s master’s thesis advisor exited
UTokyo before the student earned a doctoral degree. Such cases account for 53.2 percent of
all advisor switches in the sample. We exclude students who switched advisors on their own
initiative from the sample observations, as such student-side advisor switches are likely to
be caused by a mismatch between advisor and advisee and could be correlated with student
research outcomes.
From Figure 1, where a red square marker represents a lab where an advisor switch due
to turnover occurred, while a blue circle marker represents a lab with no advisor turnover (adarker colored marker means more students in a lab), it appears that faculty turnover incidents
are relatively frequent. There thus appears to be suﬃcient variation in the dependent variableof the regression equation (7) to identify the variance of advisor eﬀectiveness on advisee
research achievement gains within the same institution.

4.3

Data on Student Research Achievement

To measure a graduate student’s research achievement, we use the number of journal articles
that he or she published. To obtain this information, we employ the Thomson Reuters Web of
Science (WoS) archive. We collect physics articles with author names that match the name of
the graduate student under consideration. We further restrict our attention to those articles
published around the period when the target student was enrolled.
Selecting articles by author name matching may generate false positives: these articles
could have an author who coincidentally has the same name as the graduate student in
the sample but is in fact a diﬀerent person. To minimize such identification errors, we add afurther restriction; that is, for an article to be identified as written by the student in question,
we impose a restriction that the words in the article title should overlap to some extent withthose in the title of the master’s or doctoral thesis. 25
Based on a student’s publication records, we define the research proficiency score as thenumber of publication counts during a given year. Here, we employ two quality adjustment
methods. First, we limit the publications to those published in twelve high-quality peerreviewed journals, including three high-reputation general-interest science journals and nine25

See Appendix C.1 for details on the score of word overlap in titles.

17

highly ranked physics journals.26 Second, we consider a student’s share of credit for an article
if there are multiple authors. In physics, as in other scientific disciplines, papers are usually
written by a group of authors whose contributions are not necessarily equal. We follow astandard bibliometric method (e.g., Liu and Fang 2012; Waltman 2012) based on the byline
hierarchy rule to quantify a coauthor’s share of credit for an article with multiple authors.27
Figure 4 plots the average research proficiency score for our sample graduate students ineach year. Note that, in the figure, we begin the graduate school year index at one in theyear when a student entered the master’s program and increase it throughout the durationof the graduate program. For the sake of expedience, the graduate school year is also definedfor the postdoctoral period after the student obtained a doctorate degree. In the figure, thiscorresponds to the period after the 6th year.
Insert Figure 4
The figure illustrates the time pattern of how physics graduate students at UTokyo develop
their research outcomes: the achievement curve rises and reaches its peak in the years near
the completion of the doctoral degree (D1 and P1). Then, the research outcomes begin to
decline during the postdoctoral periods (P1-P5). We suspect that this reflects two types oflag structure: the first relates to a publication lag, that is, the time lag from the submission
to publication of articles in journals. The second concerns a gestation lag, that is, the timelag between project inception and completion.

5

Empirical Issues

In this section, we describe the empirical issues involved in estimating a lower bound of
professor quality based on the regression model in Equation (7). We first address how toconstruct the squared diﬀerence measure of the student outcome growth variable, which isused as the dependent variable in the regression model. We next highlight the possibility thatsupervisory period is overestimated and discuss the bias in the lower-bound estimate of anadvisor’s value added. We finally consider the non-randomness of professor turnover, which
could cause an endogeneity problem and thus threaten the validity of the estimates. We thenpropose a method to address this endogeneity concern.

26

Nature, Science and Proceedings of the National Academy of Sciences of the United States of America(PNAS) are included as the general-interest science journals, and Physical Reviews A, B, C, D, and E ;Physical Reviews Letters; and Physics Letters A and B are included as the top physics journals. We received
advice from physicists regarding the selection of the top journals.
27
See Appendix C.2 for details on the computation of a coauthor’s credit share.

18

5.1

Student Research Outcome Variable

Two issues arise regarding the double-diﬀerenced student research outcome measure: (i) the
choice of years over which the student research outcomes are aggregated at the program leveland (ii) the choice of interval years between the pair of cohorts that are diﬀerenced.28
To address the first issue, we examine the distribution of the student average research
proficiency scores over the years of the graduate program. Figure 5 decomposes the student
average research proficiency scores into those related to the master’s thesis and those related
to the doctoral thesis. The proficiency score associated with the master’s thesis peaks in thesecond year of the doctoral program (D2) and decreases thereafter, while the score related tothe doctoral thesis continues to increase. We thus opt to aggregate the research proficiency
scores over the period from the first year of the master’s program (M1) to the second year ofthe doctoral program (D2) to compute the research outcome at the master’s level. For the
research outcome at the doctoral level, we assemble the research proficiency scores from thefirst year of the doctoral program (D1) up to the fourth year of the postdoctoral period (P4).
We choose a rather long aggregation period at the doctoral level in light of the lag between
the time of article publication and the time the degree is awarded.
Insert Figure 5
In sum, our benchmark student research outcomes are aggregated over the period from
M1 to D2 and the period from D1 to P4 for the master’s degree and doctoral degree programs,
respectively. Table 1 presents the descriptive statistics.29
Insert Table 1
We turn to the second issue concerning the interval in years between cohorts. In Section 3.2, τ denotes the number of years between two cohorts, c and c′ , such that c − c′ ≤ τwhen determining the DD measure of the student research achievement growth. Note that
there is no theoretical rule for which year should be used as τ . On the one hand, the longer theinterval is, the more eﬃcient the estimator because it yields more samples for the regression
analysis.30 On the other hand, the shorter interval is, the better because it requires a weakerassumption on the covariance stationarity of the distribution of the demeaned random shocks(assumption 2.3).31 In light of balance, we adopt the adjacent cohort period of τ = 3, 4 and28

Because the value-added model focuses on the student research achievement gain while in school, themagnitude may be minute and unnoticeable if it is measured by the annual gain. We thus select the unit ofmeasure as each degree program period.
29
The box plots of the research outcome distributions at the master’s and doctoral levels are presented inTable D.2 of Appendix D.1.
τ !L
30
As presented in footnote 15, the total sample size is given by N = (τ −2)!2!
, which is an increasing functionof the adjacent period, τ , ceteris paribus.
31
To be more precise, assumption 2.3 states that the covariance of the demeaned error terms is constant
between any two students, i and j, in diﬀerent cohorts, c and c′ . This assumption may be reasonable only for19

5 years as the benchmark when implementing the regression.

5.2

Overestimation of Supervisory Period

Another empirical issue concerns imprecise measurement of the supervisory period. We havethus far implicitly assumed that, when the incident of advisor turnover happened, the professor
to which a student was reassigned supervised him or her for the entire doctoral course period.
However, the assumed supervisory period may be overstated. For instance, suppose that a
student experienced advisor turnover in the middle of his or her doctoral course. Then, theadvisor who was newly assigned may supervise the student for a period shorter than theentirety of the doctoral program.
The issue of an overestimated supervisory period also arises when the professor who isrecorded as a research advisor in the thesis catalog is a “surrogate” of the true advisor.
In some cases of turnover, especially in cases where advisor relocates to other school, theoriginal advisor who left the program continues to provide substantial research influence on
the doctoral project of the former student. In this case, even if the student who experienced
advisor turnover was assigned a new advisor, this change is nominal. The substantive researchguidance is not provided by the“surrogate” advisor but by the “hidden” advisor. The lengthof supervision of the “surrogate” advisor should be considered zero.
In Appendix D.1, we discuss how an overstated supervisory period aﬀects the the lowerbound estimate of the variance of advisor quality and show that the estimate is biased downward. Nevertheless, since the estimated value is considered a conservative lower limit ofadvisor eﬀectiveness in enhancing an advisee’s research achievement gain, ultimately, theinterpretation of the estimate will not be changed.

5.3

Non-Random Turnover

Thus far, we have assumed that professor turnover is independent of various factors in thevalue-added model and thus does not aﬀect student research performance except through thechange in advisor quality. However, this assumption may be untenable. Arguably, a professor’s decision of whether to retire, move, or remain at a graduate program may be endogenous
to the student’s performance. Table D.1 in Appendix D presents evidence suggesting thatthe sample is not balanced, that is, there are systematic diﬀerences between the groups withand without professor turnover on some characteristics.
Consequently, the regression model in Equation (7) may suﬀer from the standard endogenous variable problem, as the catch-all error term, εn , which influences student researchadjacent cohorts.

20

outcome growth, will be confounded by the advisor switch dummy variable, Wn , through theheterogeneity of advisors, who systematically diﬀer between those with and without turnover.
In this case, we may not be able to obtain an unbiased estimate of β from the regression andwould thus be unable to obtain a reliable estimate of the lower bound of advisor quality.
To make the sample balanced and comparable, we thus employ a propensity score matching
method. The basic idea is to match a turnover case with a case of no turnover that has
approximately the same conditional likelihood, typically called the propensity score, thatan incident of advisor turnover would have occurred. After constructing a new balanced
sample based on the propensity score matching procedure, we estimate the regression modelin Equation (7) using the balanced sample, as if advisor changes due to turnover occurred at
random.
Note that, to account for the endogeneity of the advisor switch dummy variable in the
regression model, we only control for advisor characteristics. It is potentially justifiable not tobalance the sample on student characteristics because we exclude all cases in which a change inadvisor occurs for a student’s own reasons, as described in Section 4.2. The sample restriction
can eliminate the possibility that student factors are confounded with the occurrence of anadvisor switch, and therefore, it is deemed to occur exclusively for reasons on the faculty side.
Hence, we control for the professor’s characteristics in the propensity score analysis.
Appendix D.2 presents a detailed procedure for the propensity score matching method that
we employ to address the issue of non-random advisor turnover. Following standard practice inthe literature, we estimate the propensity scores using a logit model. We determine a baseline
specification of the model by a stepwise likelihood-test-based procedure, suggested by Imbens(2014) and Imbens and Rubin (2015). To address the problem caused by the limited common
support of the propensity score distribution, we employ a systematic approach proposed by
Crump et al. (2009).

6

Estimation Results

6.1

Benchmark Results

This section presents the estimation results for professors’ value added to the students’ research achievement gains.
Variance Comparison
To facilitate visual inspection of the results, we provide a graphical representation of how the
DD measure of student research achievement growth, given by Equation (5), fluctuates when
an incident of advisor turnover occurred. As explained previously, we ascribe the observed
21

change in the variance in the DD measure to a disconnected change in advisor qualities due
to turnover.
Figure 6 (left) presents the sample variances of the DD measure for (a) labs in the cohort
where turnover occurred and (b) labs with the same advisor but with the last cohort before
turnover. Although the variances increase after turnover in all three cases of adjacent cohort
periods, the changes are not substantial. Figure 6 (right) repeats the same comparison of
the sample variances. However, here, with the non-randomness of turnover being taken into
account, we compare that for (c) labs in the treatment group where turnover occurred withthat for (d) labs in the corresponding control group that are matched through the propensity
score method. In contrast to the previous result, the increase in the variance is considerable
for all three cases of adjacent periods.
Insert Figure 6

Baseline Regression Estimates
To quantify the magnitude of a professor’s value added, we estimate the econometric model(7). The main estimate of interest is the lower bound of the variance in advisor quality at thedoctoral level, which is given by one-fourth of the coeﬃcient of the advisor switch indicatorvariable in the regression model.
Table 2 presents the baseline results. We report the regression estimates in rows (1) and(2). Columns (1), (2) and (3) are used to report the estimation results for the three casesof adjacent periods between cohorts, τ = 3, 4 and 5 years, respectively. As the estimatedpropensity scores are used for the true values, we compute resampling-based standard errors
to correct for the additional sampling variability arising from estimation.32 All estimates of
βs are positive and statistically significant from zero at the 10 percent level except for one
case.
Insert Table 2
Row (3) of Table 2 presents the estimated lower bound of advisor quality variance at thedoctoral level. As the variance must be non-negative, we perform one-sided tests such that∧∧

Lower bound of σd2 = 0 against the alternative Lower bound of σd2 > 0. The results indicatethat the null hypothesis is rejected at least at the 5 percent level for all cases, indicatingthat a professor’s quality has a measurable eﬀect on the research performance growth of thestudent to whom he or she is assigned.
32

Abadie and Imbens (2008) demonstrate that the bootstrap method generates biased estimates of the standard errors for a nearest-neighbor matching estimator and suggest using the subsampling method developed
by Politis and Romano (1994). We therefore use the subsampling method whereby we draw fewer observations
than the same size at each iteration without replacement.

22For the results that we have presented thus far, we base the student research outcome on
the research proficiency scores that are adjusted for the share of credit of each author. Alternatively, we can quantify the research outcome of a student without credit share adjustment.
To this end, we count the number of first-authored articles that the student published as alead author in the selected top general and field journals in physics. While the alternative
research outcome measure might be crude and subject to a certain amount of noise — it mightunderrate the research achievement of a student because it ignores the articles for which he or
she is not a lead author, or it might overrate the student’s attainment because it accords him
or her all of the credit, even for multi-authored articles, irrespective of how many coauthors
are involved — it nonetheless serves as a simple and easily interpreted yardstick.
The estimation results using the alternative research outcome measure are presented incolumns (4) to (6) of Table 2. The regression estimates are larger than the previous results that
adjusted the author’s credit share. This is unsurprising because the first-author-based measureis greater than the original measure to the extent that the credit share is not weighted.33
The estimated values of the lower bound of σd2 , reported in row (3), are correspondingly
larger than those previously reported. Reassuringly, the null hypothesis that the variance inadvisor quality is zero cannot be rejected at least at the 5 percent level. We therefore obtain
qualitatively similar evidence on the professor’s value added as that found previously.
The results presented above indicate the eﬀectiveness of professors in improving doctoral students’ research productivity growth. Indeed, better advisor quality causally aﬀects
advisees’ research achievement gains in graduate school. If we use 0.0489 as the most conservative estimate of the lower bound of the advisor quality variance among those reported
in columns (1) to (3) of Table 2, we find that a one-standard-deviation increase in professor quality raises the average student research achievement gain at the doctoral level by atleast 0.221, which corresponds to approximately 0.54 standard deviations of the total doctoral
program research outcome distribution.
If we base the estimation results on the first-author-based research outcome measure reported in columns (4) to (6) of Table 2, we find that, if professor quality increases by one
standard deviation, the average student publishes 0.64 more first-authored articles in top
journals at the doctoral level.34 We are thus able to conclude that a professor’s value added
to graduate student research outcomes is substantial.
Our estimates of value added provide an interesting comparison with the professor valueadded estimates at the undergraduate level reported by previous studies. For example, Hoﬀmann and Oreopoulos (2009) estimate professor value added to student’s achievement gains,
33

The mean and standard deviation of the first-author-based research outcome at the doctoral level are
0.39 and 0.96.
34
When computing the standard deviation increase, we use 0.410 as the estimated value of the lower boundof advisor quality variance.

23

measured by undergraduate course grades in a large Canadian university. They report thata one-standard-deviation increase in professor quality yields an approximately 0.05-standarddeviation increase in a student’s grade. Carrell and West (2010) obtain a similar value-added
estimate for professors at the U.S. Air Force Academy who teach introductory courses at
the undergraduate level. They report that the standard deviation of value added is approximately 0.05. Therefore, our estimates of professor-value added at the postgraduate level are
substantially larger than those standard-deviation estimates at the undergraduate level.
The observed diﬀerence in the estimates might not be too surprising considering the factorsthat distinguish our study from other studies. First, the professor quality that we measure is
diﬀerent. We evaluate the dimension of professor quality that promotes a student’s researchcapability, whereas those previous studies assess the aspect of quality that enhances a student’s
academic capability. Second, closely related to the first point, the student outcome is diﬀerent.
We focus on the research achievement gains of postgraduate students, while previous studies
investigate the academic achievement gains of undergraduate students.
Robustness against Various Specifications
We perform a series of checks on the estimation results regarding diﬀerent specificationsof student research outcomes. First, we consider alternative configurations in terms of theperiod over which the research proficiency scores are aggregated for each degree program.
Specifically, in addition to the benchmark case (M1-D2 for the master’s program and D1-P4
for the doctoral program), we examine alternative cases that change the aggregation period
at the master’s and doctoral levels. Second, we examine whether the results are driven by aspecific value of the threshold that is used to compute students’ research proficiency scores.
As explained above, we consider research articles that are actually published by a target
student if the author’s name matches the student’s name and, in addition, the degree of word
overlap in the titles between the article and the student’s thesis exceeds some predeterminedthreshold value. While the default value is set to minimize both type 1 and type 2 errors, weemploy both over-matching and under-matching criteria in the robustness exercise. Finally,
we check the sensitivity of our estimates to the particular choice of top journals, we replicate
the baseline analysis by narrowing the coverage to nine journals (two general-interest scienceand seven field journals) instead of twelve journals.35
The estimation results are presented in Table E.1 and Table E.2 of Appendix E.2. We
find that all of the results are qualitatively similar to the previously reported findings. The
null hypothesis that the variance in doctoral-level advisor quality is zero is rejected at the 10
35

The three of the original twelve journals excluded here are PNAS in the general-interest science journal
category and Physics Letters A and B in the field journal category. This is based on suggestions that we
received from several physics researchers.

24percent level in all cases that change the aggregation period for the default value of the title
matching threshold, and in most cases for both over-matching and under-matching criteria.
Considering the results, we can conclude that the findings from the regression model are
not merely artifacts of the particular specifications of student research outcomes and endorse
the conclusion that professor quality plays a distinct role in enhancing a student’s research
capacity in the doctoral program.

6.2

Robustness Tests

This section provides various robustness checks for the benchmark results. First, we implement a falsification test that investigates whether a false instance of an advisor switch predicts
an increase in the volatility of student research outcomes between programs and cohorts. Second, we discuss the possibility that the lower bound of the estimate of the advisor quality
variance might be overestimated. Third, we examine the robustness of the estimation results
under a weaker conditional independence assumption on advisor switch due to turnover.
Falsification Test
In our estimation framework, the variance in advisor quality is identified by an increase in thesquared diﬀerence of the student research outcome gain at the time of advisor turnover. We
thus implement a falsification exercise that examines whether the timing agrees with what ispredicted by the empirical model.
To do so, we construct a false advisor switch dummy variable, W̃n , that takes value one
for the lab in one cohort before the actual incident and zero otherwise. Specifically, given
lab ℓ(a, c), where advisor turnover occurred, the variable W̃n is one in the latest cohort, c′ ,in which advisor a supervised at least one student before cohort c. We estimate a regression
similar to regression model (7) using the dummy variable W̃n as the regressor instead of using
the true advisor switch dummy variable, Wn , with β̃ being the coeﬃcient of the variable W̃ .
We report the estimated value of β̃ in panel (A) of Table 3, where we adopt the samedefinition of the student research outcome measures as in the baseline case, and replicate theregression results except that we use the false advisor switch dummy variable.36 It is shownthat the false advisor switch dummy variable is sometimes negative and has no systematic
impact on the the squared diﬀerence of the student research outcome gain. Indeed, in all cases
except one, the false advisor switch dummy variable is not statistically significant, suggestingthat the results survive the falsification test.

36

The full estimation results are reported in Table E.3 of Appendix E.2.

25Factors that Lead to Upward Bias
As we are interested in estimating a lower bound of the variance in advisor quality, downward
bias would not be problematic. There is, however, a set of potential sources of upward bias.
The first possibility that might introduce upward bias concerns the allocation of the research credit share between advisor and advisee. One would assume that students are merely
assigned a part of a larger research project, or subtopic, that the advisor has pursued, andthus, their contribution to the project in collaboration with their advisors is marginal.37 If
this is true, our turnover estimator for the lower bound of the variance in advisor quality
might suﬀer from systematic upward bias, as we would then mistakenly ascribe the advisor’s
research contribution to the student’s research achievement.
Because the actual collaboration process is not observed for joint research activities, it is
impossible for us to allocate the true share of credit to each member of an advisor-advisee
pair that engaged in a joint research project. We therefore consider an extreme case in which
the student’s contribution is zero whenever he or she collaborated with a research advisor tohighlight the sensitivity of the previous estimation results to the assumption on the allocationof research credit.
Panel (B) of Table 3 presents the estimated lower bound of advisor quality variance,
assuming that the research proficiency score of a student publication is equal to zero if it
is coauthored with the advisor.38 Looking across the columns of the table, the size of theestimated coeﬃcients and the lower bound of advisor quality variance tend to be lower.
Nonetheless, the one-sided test of the null hypothesis that doctoral-level advisor quality has
no eﬀect on an advisee’s research achievement growth is rejected at the 10 percent level.
Because we consider a severe restriction on the allocation of the credit share to the side ofadvisees, which is overly severe for the advisees in terms of their research contributions, thereported evidence of positive professor value added reassuringly supports the conclusion thatprofessors enhance their students’ research achievement gains by advising and mentoring theirresearch projects at the postgraduate level.
Another potential source of upward bias is that the assumption on the time-invarianceof advisor quality, given by assumption 1.1, might be violated. Suppose, contrary to theassumption, that it varies across cohorts within a professor. In particular, if it fluctuates
as the end of a professor’s research career approaches, the squared diﬀerence measure, the37Note that our empirical study relies on the assumption that the student made an original and substantial
contribution to his or her thesis research projects and that the articles with titles that are closely associatedwith the master’s thesis or doctoral dissertation can be used as an unbiased yardstick to gauge the student’s
in-school research achievement. The assumption appears somewhat reasonable for physics departments inJapanese universities, where, as described in Section 2.2, graduate students are typically accorded a fairamount of autonomy when choosing a research topic and approach.
38
The full estimation results are reported in Table E.4 of Appendix E.2.

26

dependent variable in regression model (7), becomes more volatile in the last cohorts before
a professor’s turnover. In this case, the regression coeﬃcient of the advisor switch dummy
variable might overstate the lower bound of advisor quality variance.
To shed some light on this concern, we augment the regression model in Equation (7)by including a set of dummy variables that capture the possible change in advisor quality
variance in the period near turnover.39 Panel (C) of Table 3 presents the estimated lower
bound of the variance in advisor quality from the augmented specification.40 As shown,
the estimates are not substantially aﬀected by the inclusion of the cohort-specific dummy
variables. Encouragingly, the null hypothesis that advisor quality in the doctoral program
has no eﬀect on student research outcome growth is rejected at least at the 10 percent levelin all cases.41
Insert Table 3

Non-Retirement Turnover
Our identification strategy for a lower limit of advisor quality relies on the assumption thatadvisor switch is incidental, that is, it is independent of an advisee’s unobservables conditionalon the advisor’s observable characteristics. The assumption leads us to consider the baseline
regression model where the advisor switch indicator, Wn , is assumed to be orthogonal to thestudent-specific catch-all error term, εn , once the observables of professors are controlled forby the propensity score method.
There might be a concern about the validity of the conditional randomness of advisor
switch, however. Indeed, advisor switch (and non-switch) will be intentional for students.
It might be possible for students, having prior information on faculty members, to predict“scheduled” retirement events and choose their advisors and labs. If this type of self-selection
occurs, students in a lab with turnover might be systematically diﬀerent from those in thelab without turnover. This may cause the advisor switch indicator to be correlated with theerror term in the regression model through confounding factors related to a student’s research
performance. The problem of unobserved heterogeneity in the regression cannot be eliminatedby the propensity score method since it takes into account only selection on observables on
the side of advisors.
To mitigate this concern, we examine how the lower-bound value estimates of the variancein advisor quality are robust under a more plausible assumption on the conditional indepen(a,c,c′ )39Specifically, we consider a regression that includes dummy variable Dk
for k = 1, 2, and 3. The
dichotomous variable takes value one if cohort c is within k years before professor a exited and zero otherwise,.
40
The full estimation results are reported in Table E.5 of Appendix E.2.
41
Furthermore, the estimation results shown in Table E.5 indicate that none of the coeﬃcients concerning
the added dummy variables are statistically significant.

27

dence of advisor turnover. We perform regression analyses under an assumption that turnover
due to non-retirement reasons is accidental and thus that the event of advisor switch causedby the turnover is incidentally orthogonal to unobserved student characteristics.42 It may betrue that students can predict the timing of the scheduled mandatory retirement of a faculty
member with some degree of accuracy.43 Nonetheless, it does not seem unreasonable to assume that students do not have suﬃcient information to foresee the future relocation, not tomention decease, of a faculty member ax ante at the time of graduate enrollment. Given thelimited ability of students to predict the timing of faculty members’ relocation and death,it seems less controversial to assume that an advisor switch caused by non-retirement-based
turnover is more incidental than one caused by retirement-based turnover and thus is likely
to be uncorrelated with student-specific unobserved factors.
Table 4 reports the estimated lower bounds for the variance of advisor quality when the incidents of turnover are restricted to those due to non-retirement reasons.44 Panel (A) presents
the estimates when we replicate the baseline specification, while the panel (B) presents theestimates for the case in which the research proficiency score of a student publication is set
to zero if it is a joint work with the advisor.
Overall, we find that the estimates are positive and statistically significant, which bolsters
the previous finding that advisor quality matters for an advisee’s research performance growth.
Indeed, the estimated lower bounds for the variance of advisor quality are larger than thosefound in the corresponding baseline case. However, as the results only pertain to advisor
switches that occurred due to non-retirement reasons, we do not use them to update theinformation on the magnitudes of the variance of advisor quality. It seems nonetheless safe
to say that the previous finding on the eﬀectiveness of advisors is not driven by self-selection
behavior by advisees.
Insert Table 4

42

Since the available information on students’ background characteristics is rather limited comparing tothat on professors, we do not opt for including as many characteristics as possible to absorb all the potential
student-side heterogeneity across labs.
43
It appears that not all retirement events are perfectly predictable, however. As explained in footnote
22, the mandatory retirement policy changed at UTokyo in the year 2001. The policy was made an oﬃcial
decision in the year 2000, but there had been considerable uncertainty as to whether the policy would beenacted at the time. Thus, it seems highly likely that faculty members, not to mention graduate students,
who were at UTokyo’s physics program in the late 1990s were able predict the change in the retirement age
that happened in the early 2000s.
44
The full estimation results are reported in Table E.6 and Table E.7 of Appendix E.2.

28

6.3

Additional Evidence for Professors’ Influence on Students

Other Mechanisms
The estimation results have shown that advisor turnover generates significant variations in
an advisee’s research achievement gains at UTokyo’s department of physics. According to a
standard value-added model, we ascribe the increased diversity of student research achievement gains to the discrete change in advisor quality at the time of turnover. Admittedly,
however, there may remain other mechanisms that create such a pattern.
One possibility is that professor turnover always has a positive eﬀect on students’ research
capacity and thus increases the variability in student achievement gains between cohorts withand without turnover. The positive advisor turnover eﬀect could be caused by a mechanismthat reflects a well-known understanding that innovation (and thus economic growth) is dueto the recombination of existing ideas (e.g., Weitzman 1998). This follows from this view
that, as new innovation is likely to arise from recombining old knowledge elements, students
who are supervised by diﬀerent professors would have access to a wider variety of knowledge
and ideas and can thus enhance their research capabilities.
Another mechanism yields a negative eﬀect of professor turnover on students’ researchachievement gains. Turnover may lead to a disruption that impacts the students who areforced to change their research advisors. As is often noted in the education literature (e.g.,
Wisker and Robinson 2013), if an advisor is lost due to turnover, an advisee who becomesan “orphan” occasionally perceives this as a traumatic event and suﬀers from psychologicalproblems that might occasionally result in the under-development of academic achievement. If
this understanding is correct, advisor turnover would retard the advisee’s research progress,
irrespective of how high the quality of the newly assigned advisor is and thus generate a
noticeable gap in student research outcome gains between cohorts with and without turnover.
Recall that, according to the mechanism captured by the value-added model, the advisee’s
research outcome growth can be positive or negative after turnover – indeed, as explained in
Section 3.1, the direction of growth depends decisively on the relative levels of advisor qualitythat were switched when turnover occurred and will thus not be predicted a priori unless theinformation on the exact quality levels is available. In the analysis that follows, we investigate
which mechanism is more likely by estimating a regression similar to the regression model()

in Equation (7), except with the dependent variable being in levels, DD∆outcomen , not in()2

squares DD∆outcomen . To identify the mechanism in place, we focus on the sign of theturnover eﬀect on the advisor’s research achievement gain.
Panel (A) of Table 5 presents the regression results for which all estimated coeﬃcientsof the advisor switch indicator are shown to be positive but are not statistically significant

29

in all cases.45 We can interpret the results as indicating that, contrary to the predictions of
the alternative mechanisms, advisor turnover can have a positive or negative impact on anadvisee’s research productivity growth. As the individual impacts cancel one another out, theaggregate eﬀect, as reflected by the integration, is not significantly diﬀerent from the null inlevels. It thus appears to confirm that the mechanism that the value-added model postulates
should be a main driver of the empirical findings obtained thus far.
Indirect Influence
Our analysis thus far has concentrated on the advisor-advisee relationship within a lab andintended to measure the eﬀectiveness of knowledge transmission through a direct interaction
channel within a lab. However, knowledge might be transmitted beyond master-apprenticestyle contact.
We modify the baseline specification in Equation (2) by incorporating an “indirect” eﬀectof professors. Consider the following augmented model of student research outcome gains:ℓ(e,c)∆outcomeeg= γ̄ ℓ(e,c) + θeg +∑ℓ(e,c)
πef θf g + ν̄eg
,

(10)

f ∈A

where we consider lab ℓ(e, c) of professor e ∈ A in cohort c ∈ C . The parameter πef captures
the magnitude of the indirect influence from non-advisor faculty member f on the average
research outcome gain in program g for students in lab ℓ(e, c). In what follows, for the purposeof simplicity, we assume that πef = π if the research field or subfield of professor f is thesame as or closely related to that of the direct advisor, e, and πef = 0 otherwise.46
Analogous to Equation (6), we compute the conditional expectation of the squared doublediﬀerenced average student research outcome growth and construct a regression model basedon the comparison of the conditional expectations between labs with and without a “treat′ment” assignment. To achieve this aim, let us use V ℓ(e,c,c ) to denote an assignment indicator′of an “indirect” turnover incident. Specifically, define V ℓ(e,c,c ) = 1 if a professor whose research subfield is the same as that of professor e is replaced due to turnover in cohort c and′V ℓ(e,c,c ) = 0 otherwise.
We obtain the following result under the same assumptions as above on the distributions

45

The full estimation results are reported in Table E.8 of Appendix E.2.
In other words, we restrict the scope of indirect influence to that between professors and students within
the same research field (or subfields).
46

30

of advisor quality and the idiosyncratic error terms:[()ℓ(e,c,c′ ) 2ℓ(e,c,c′ )ℓ(e,c,c′ )]DD∆outcome|W= 0, V
(
)
{
}
1
1
′
=α
+ l(e,c′ ) + π 2 2σd2 (1 − ρd ) V ℓ(e,c,c ) ,ℓ(e,c)
I
I

E

(11)

where α is the same as that given in Equation (6). This, in turn, leads to the following
regression model using the subsamples that consist of labs in which advisor turnover did not′occur (W ℓ(e,c,c ) = 0):()2DD∆outcomem = αind Xm + βind Vm + εm ,(12)

where m = 1, · · · , M is the index of observations.47
Comparing Equations (11) and (12) leads to the parameter relationship that βind =π 2 {2σd2 (1 − ρd )}. Let us use β̂dir to denote an estimate of the coeﬃcient of Wn from thebaseline regression model given by Equation (7), and let β̂ind be an estimateof the coeﬃcient√of Vm from Equation (12) presented above. We therefore obtain π̂ =β̂ind /β̂dir , which canbe used as a measure of indirect knowledge transfer from a non-advisor professor in the same
research field.
An empirical challenge is to identify a group of professors whose research subjects wereclose enough to that of the professor experiencing turnover. As the type of data necessary tojudge the similarity between research subjects is absent or rarely present, we adopt a simple
and heuristic method for identifying the same research subject groups, which exploits theinformation revealed by the actual turnover events. It is conceivable that, when an instanceof professor turnover occurred, the students in the lab of the professor who exited were highly
likely to be re-assigned to a professor whose research area was closely related to that of theoriginal professor. In the empirical analysis that follows, we therefore assume that the original
professor who exited and the re-assigned professor were working in the same research area.
The detailed situation is illustrated in Appendix E.1.
The results shown in panel (B) of Table 5 represent the regression estimates.48 We adopt
the default setting for the student research outcome and use the same estimation method as
before.49 As shown, the estimates are ambiguous for βind . One of the estimates is negative,
and in the case in which the estimates are positive, they are not statistically significant at′

Here, the unit of observation is each element of (e, c, c′ ) such that W ℓ(e,c,c ) = 0 for any advisor e ∈ A
and cohorts c, c′ such that 0 < c − c′ ≤ τ .
48
The full estimation results are reported in Table E.9 of Appendix E.2.
49
The research outcomes in the master’s degree and doctoral degree programs are aggregated over theperiod from M1 to D2 and the period from D1 to P4, respectively. Furthermore, the set of “top journals”
here consists of twelve journals.
47

31

the 10 percent level in any specification except one. The estimates of the squared indirect
influence parameter, π 2 , are reported in row (3).50 Again, the signs of the estimates diﬀer.
The maximum estimate is 0.269, while the value where β̂ind is statistically significant is as
low as approximately 0.1, as reported in column (6). This result implies that the indirect
knowledge transfer eﬀect from non-advisor faculty is π̂ = 0.33, suggesting that it is, at most,less than one-third of the direct eﬀect from the advisor.
On balance, therefore, there appears to be little or no indirect influence from non-advisor
faculty members across labs on doctoral student research productivity growth.

7

Conclusion

In this paper, we investigated the extent to which professors can aﬀect the development of
the research performance of the graduate students whom they supervise. By using detailed
data on professors and students at UTokyo’s department of physics, we estimated a lower
bound of the professor value added to student research achievement growth while in school.
The estimation results consistently show that postgraduate research education based on anadvisor-advisee relationship is quite eﬀective — professors have a substantial impact on thestudents’ achievement gains in terms of the number of publications in top journals in physics.
This corroborates the view of earlier studies (e.g., Azoulay, Zivin and Wang 2010; Borjas andDoran 2014; Moser, Voena and Waldinger 2014) that research interactions among scientists
in vertically aligned relationships, including senior-and-junior-collaborator, teacher-student,
and advisor-advisee relationships, matter for the creation and diﬀusion of scientific ideas andknowledge.
Our findings also suggest that the accumulation of prominent scientists in a comparatively
small number of universities is explained, at least partially, by the results of successful education at the postgraduate level. For example, in Japan, five out of ten Nobel Prize winners
in physics completed their doctoral degrees at UTokyo, and four earned their doctorate degrees at Nagoya University. Given our results on the eﬀectiveness of professors in enhancing
students’ research capability growth, we can speculate that the relatively high concentrationof physics Nobel laureates in these two universities in Japan might be caused not only by theprocesses of students’ self-selection or schools’ selective recruitment but also by the beneficial
reproduction of elite physicists, which was enabled by a deliberate process of teaching andlearning in a lab. While previous studies (e.g., Waldinger 2010) suggest that high-quality
universities can facilitate human capital accumulation among graduate students, our paper
specifically adds that this outcome is based on advisor-advisee-based education.
50

We compute π̂ 2 as the quotient of the estimate β̂ind over the estimate β̂dir that appears in the corresponding number of columns in Table 2.

32

We need to highlight some limitations of this paper. First, our analysis of the professor’s
value added is essentially short run. Although the estimation results reveal that researchadvisors can influence the research development of their students, the impact might be limited to the short span of time while the student is in graduate school or several years after
the completion of graduate school. It is left to future research to examine whether a professor’s supervision during a graduate program has a long-term impact on students’ research
performance during their postgraduation careers.
Second, the analysis in the paper is limited to a small, albeit prominent, group of physicists.
Thus, our conclusion regarding a professor’s value added might not be generalizable to groupsof other scientists from diﬀerent disciplines or other graduate schools. We hope that thefindings of this paper regarding the eﬃcacy of professors in promoting student progress inresearch performance will be helpful to stimulate further research in related areas including
the economics of higher education and the economics of science and technology.

33

References
Abadie, Alberto and Guido W. Imbens, “Notes and Comments on the Failure of the
Bootstrap,” Econometrica, 2008, 76 (6), 1537–57.
Abe, Yasumi and Satoshi P. Watanabe, “Some Thoughts on Implementing US Physics
Doctoral Education in Japanese Universities,” Asia Pacific Education Review, 2012, 13 (3),403–15.
Azoulay, Pierre, Joshua S. Graﬀ Zivin, and Jialan Wang, “Superstar Extinction,”
Quarterly Journal of Economics, 2010, 125 (2), 549–89.
Borjas, George J. and Kirk B. Doran, “The Collapse of the Soviet Union and theProductivity of American Mathematicians,” Quarterly Journal of Economics, 2012, 127(3), 1143–203.
and, “Which Peers Matter? The Relative Impacts of Collaborators, Colleagues, andCompetitors,” Review of Economics and Statistics, 2014. forthcoming.
Branch, Gregory F, Eric A Hanushek, and Steven G Rivkin, “Estimating the Eﬀect ofLeaders on Public Sector Productivity: The Case of School Principals,” Technical Report,
National Bureau of Economic Research 2012. NBER Working Paper #17803.
Carrell, Scott E. and James E. West, “Does Professor Quality Matter? Evidence from
Random Assignment of Students to Professors,” Journal of Political Economy, 2010, 118(3), 409–32.
Chetty, Raj, John N Friedman, and Jonah E Rockoﬀ, “Measuring the Impacts ofTeachers I: Evaluating Bias in Teacher Value-added Estimates,” American Economic Review, 2014, 104 (9), 2593–2632.
Crump, Richard K., Joseph V. Hotz, Guido W. Imbens, and Oscar A. Mitnik,“Dealing with Limited Overlap in Estimation of Average Treatment Eﬀects,” Biometrika,
2009, 96 (1), 187–99.
Grossman, Gene M. and Elhanan Helpman, “Quality Ladders in the Theory of
Growth,” Review of Economic Studies, 1991, 58 (1), 43–61.
Gumport, Patricia J., “Graduate Education and Organized Research in the United States,”in Burton R. Clark, ed., The Research Foundations of Graduate Education: Germany,
Britain, France, United States, Japan, Berkeley, CA: University of California Press, 1993,
pp. 225–60.
34

Hanushek, Eric A. and Steven G. Rivkin, “Teacher Quality,” in Eric A. Hanushek
and Finis Welch, eds., Handbook of the Economics of Education, Vol. 2, Elsevier, 2006,
chapter 18, pp. 1051–78.
and, “Generalizations about Using Value-added Measures of Teacher Quality,” The

American Economic Review, 2010, 100 (2), 267–71.
and, “The Distribution of Teacher Quality and Implications for Policy,” Annual Review

of Economics, 2012, 4, 131–57.
Hilmer, Michael J. and Christiana E. Hilmer, “Fishes, Ponds, and Productivity:
Student-Advisor Matching and Early Career Publishing Success for Economics PhDs,”
Economic Inquiry, 2009, 47 (2), 290–303.
Hoﬀmann, Florian and Philip Oreopoulos, “Professor Qualities and Student Achievement,” Review of Economics and Statistics, 2009, 91 (1), 83–92.
Imbens, Guido W., “Matching Methods in Practice: Three Examples,” NBER Working
Paper #19959, National Bureau of Economic Research 2014.
and Donald B. Rubin, Causal Inference in Statistics, Social, and Biomedical Sciences,
New York, NY: Cambridge University Press, 2015.
Jones, Benjamin F and Benjamin A Olken, “Do Leaders Matter? National Leadershipand Growth since World War II,” Quarterly Journal of Economics, 2005, 120 (3), 835–864.
Kawashima, Tatsuo and Fumihiro Maruyama, “The education of advanced students
in Japan: engineering, physics, economics and history,” in “The Research Foundationsof Graduate Education: Germany, Britain, France, United States, Japan,” University ofCalifornia Press, 1993, pp. 326–353.
Lacetera, Nicola, Bradley J Larsen, Devin G Pope, and Justin R Sydnor, “Bid
Takers or Market Makers? The Eﬀect of Auctioneers on Auction Outcome,” American
Economic Journal: Microeconomics, 2016, 8 (4), 195–229.
Lazear, Edward P, Kathryn L Shaw, and Christopher T Stanton, “The value ofBosses,” Journal of Labor Economics, 2015, 33 (4), 823–861.
Liu, Xuan Zhen and Hui Fang, “Fairly Sharing the Credit of Multi-authored Papers andits Application in the Modification of h-index and g-index,” Scientometrics, 2012, 91 (1),37–49.

35

Low, Morris, Science and the Building of a New Japan, New York, NY: Palgrave Macmillan,
2005.
Lucas, Robart E., “On the Mechanics of Economic Development,” Journal of Monetary
Economics, 1988, 22, 3–42.
Malmendier, Ulrike and Geoﬀrey Tate, “Superstar CEOS,” Quarterly Journal of Economics, 2009, 124 (4), 1593–1638.
Moser, Petra, Alessandra Voena, and Fabian Waldinger, “German-Jewish Émigrés
and US Invention,” American Economic Review, 2014, 104 (10), 3222–55.
Ogawa, Yoshikazu, “Challenging the Traditional Organization of Japanese Universities,”
Higher Education, 2002, 43 (1), 85–108.
Polanyi, Michael, Personal Knowledge: Towards a Post-Critical Philosophy, Chicago, IL:
University of Chicago Press, 1958.
, The Tacit Dimension, London: Routledge, 1966.
Politis, Dimitris N. and Joseph P Romano, “Large Sample Confidence Regions Based on
Subsamples under Minimal Assumptions,” The Annals of Statistics, 1994, 22 (4), 2031–50.
Rivkin, Steven G., Eric A. Hanushek, and John F. Kain, “Teachers, Schools, andAcademic Achievement,” Econometrica, 2005, 73 (2), 417–58.
Romer, Paul M., “Endogenous Technological Change,” Journal of Political Economy, 1990,
98 (5), S71–102.
Rosenbaum, Paul R. and Donald B. Rubin, “Constructing a Control Group Using Multivariate Matched Sampling Methods that Incorporate the Propensity Score,” The American
Statistician, 1985, 39 (1), 33–8.
Tanabashi, Masaharu, “Shoichi Sakata: His Life and Physics, Collections of Materials in
Sakata Memorial Archival Library,” Progress of Theoretical Physics Supplement, 2012, 197,
1–14.
Tang, Li and John P. Walsh, “Bibliometric Fingerprints: Name Disambiguation Basedon Approximate Structure Equivalence of Cognitive Maps,” Scientometrics, 2010, 84 (3),763–84.
Trajtenberg, Manuel, Gil Shiﬀ, and Ran Melamed, “The ”Names Game”: Harnessing
inventors’ Patent Data for Economic Research,” NBER Working Paper #12479, National
Bureau of Economic Research 2006.
36

Traweek, Sharon, Beamtimes and Lifetimes: The World of High Energy Physicists, Cambridge, MA: Harvard University Press, 1988.
Ushiogi, Morikazu, “Graduate Education and Research Organization in Japan,” in Burton R. Clark, ed., The Research Foundations of Graduate Education: Germany, Britain,
France, United States, Japan, Berkeley, CA: University of California Press, 1993, pp. 299–
325.
Waldinger, Fabian, “Quality Matters: The Expulsion of Professors and the Consequences
for PhD Student Outcomes in Nazi Germany,” Journal of Political Economy, 2010, 118 (4),787–831.
, “Peer Eﬀects in Science: Evidence from the Dismissal of Scientists in Nazi Germany,”
Review of Economic Studies, 2012, 79 (2), 838–61.
Waltman, Ludo, “An Empirical Analysis of the Use of Alphabetical Authorship in Scientific
Publishing,” Journal of Informetrics, 2012, 6 (4), 700–11.
Weitzman, Martin L., “Recombinant Growth,” Quarterly Journal of Economics, 1998, 113(2), 331–60.
Wisker, Gina and Gillian Robinson, “Doctoral “Orphans”: Nurturing and Supporting
the Success of Postgraduates Who Have Lost Their Supervisors,” Higher Education Research & Development, 2013, 32 (2), 300–13.
Zuckerman, Harriet, Scientific Elite: Nobel Laureates in the United States, New Brunswick,
NJ: Transaction Publishers, 1977.

37

Advisor ID

Figures
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59

60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119

1970

1975

1980

1985

1990

1995

1 (w/o turnover)1 (w/ turnover)

2000

2005

1970

2 (w/o turnover)
2 (w/ turnover)

1975

1980

1985

1990

1995

2000

2005

3+ (w/o turnover)3+ (w/ turnover)Figure 1: Distribution of Students across Initial Advisors and Cohorts
Notes: Initial advisors are on the vertical axis and cohorts are on the horizontal axis. The
red square and blue circle markers represent a lab where the advisor was replaced due toturnover and a lab where the advisor was not replaced, respectively. The darker the marker
color is, the more students in a lab.

timelineturnoverprofessor aℓ(a, c0 )ℓ(a, c1 )ℓ(a, c2 )professor b
Figure 2: Example of Labs with and without TurnoverNotes: There are three labs with diﬀerent cohorts, c0 , c1 and c2 , whose initial advisor isprofessor a. Each lab is portrayed by a connected line segment, which represents the twoyear master s degree program and the three-year doctoral degree program. Advisor turnover
did not occur in labs ℓ(a, c0 ) or ℓ(a, c1 ) before cohort c2 . However, in lab ℓ(a, c2), professor
a exited the school due to turnover, and professor b took charge of the doctoral students.

38

10
5
70

75

80

85

retire

90

95

move

00

other

Figure 3: Number of Turnover Incidents in Each Year (1970-2004)

0.01

Research Proficiency Score
.02
.03

.04

Notes: The reasons for turnover are classified into the following three categories: (1) retirement if the instance of turnover occurred at the mandatory retirement age; (2) move if
turnover occurred before the retirement age and the faculty name began to appear on otheruniversities’ roasters beginning in the year after the turnover instance; and (3) decease/quit
otherwise.

M1

M2

D1

D2

D3

P1

P2

P3

P4

P5

Figure 4: Average Student Research Proficiency Scores
Notes: The research proficiency score is defined as the number of publication counts of a
student’s publication records during a given year. Two quality adjustment methods are
employed. First, we limit the publications to those published in twelve high-quality peerreviewed journals. Second, we consider a student’s share of credit for an article if there are
multiple authors.

39

.03
Research Proficiency Score
.01
.02
0

M2

D1

D2

master thesis related

D3
doctor thesis related

Figure 5: Decomposition of the Student Research Proficiency Scores.5(a) Lab with Turnover(b) Lab before Turnover

.3
.2
3

4

.1

.1

.2

.3

.4

(c) Treatment(d) Control.4.5Notes: The average student research proficiency scores are decomposed into those related
to the master’s thesis and those related to the doctoral thesis. We determine whether
research articles are actually published by a target student if the degree of word overlap in
the titles between the article and the student’s master’s and doctoral theses exceeds somepredetermined threshold value.

5

adjacent cohort period (tau)

3

4

5

adjacent cohort period (tau)

Figure 6: Comparison of the Double-diﬀerenced (DD) Average Student Research Outcome Growth between Labs with and without TurnoverNotes: The left figure presents the sample variances of the DD measure for (a) labs inthe cohort where turnover occurred and (b) labs of the same advisor but their cohort isthe last before turnover. The right figure provides the same comparison of the sample
variances between (c) labs in the treatment group where turnover occurred and (d) labs inthe corresponding control group that are matched through the propensity score method.

40

Tables
Table 1: Descriptive Statistics for the Student Research Outcomes in Levels and inDiﬀerences

Mean
S.D.
Min
Max
Sample Size

Research Outcome
at the Master’s Level

Research Outcome
at the Doctoral Level

Research Outcome Gainat the Doctoral Level

outcomeciam

outcomeciad∆outcomeciad

0.06770.21840.00002.31751019

0.22020.50750.00004.73031019

0.14810.4068-0.47384.73031019

Notes: The research outcome at each degree level is computed based on the research proficiency scores. The aggregation years are M1-D2 for the master’s level and D1-P4 for the
doctoral level. The gain in research outcome at the doctoral level is given by the diﬀerencein the research outcome from the doctoral level to the master’s level. Since the researchoutcome at the bachelor’s level is normalized to zero, the gain in research outcome at themaster’s level is equal to the research oucome at the master’s level.

41

Table 2: Baseline Estimation Results: The Eﬀect of Advisor Turnover on Student Research Outcome Growth at the Doctoral Level
Dependent
Adjacent Period(1) α̂
(2) β̂
∧(3) Lower bound of σd2

Credit Share Weighted

First-authored-paper Based[DD∆outcome]2[DD∆outcome]2

τ =3
τ =4
τ =5
(1)
(2)
(3)
0.0667 *** 0.0742 *** 0.0960 ***(0.0237) (0.0225) (0.0134)
0.3371 * 0.2663 ** 0.1956 **(0.1746) (0.1204) (0.0985)
0.0843 ** 0.0666 ** 0.0489 **

τ =3
τ =4
τ =5
(4)
(5)
(6)
0.05700.1267 ** 0.4025 ***(0.0682) (0.0545) (0.0720)
2.3091 * 2.1322 ** 1.6401 **(1.3249) (0.9220) (0.8251)
0.5773 ** 0.5331 ** 0.4100 **[0.0268][0.0135][0.0236][0.0407][0.0104][0.0234]

925
104

1202
186

1446
271

925
104

1202
186

1446
271

Sample Size
Total
After matching

Notes: The dependent variable is the squared double-diﬀerenced average student researchoutcome growth. The total sample size is given by the number of observations for each tupleof (a, c, c′ ) for any advisor a in A and cohort c, c′ such that 0 < c − c′ ≤ τ , where τ isthe period over which the diﬀerence is taken. To make the sample balanced, a propensity
score matching method is used. A logit model is used to estimate the propensity scores. The
after-matching sample size is the sum of the numbers of observations for the treatment groupwhere turnover occurred and the corresponding control group that are matched through thepropensity score method. The standard errors, which are computed by the subsampling
method of Politis and Romano (1994), are in parentheses. The numbers in square brackets
are p-values for the one-sided tests such that Lower bound of σd2 = 0 against the alternative∧∧

Lower bound of σd2 > 0.
*** Significant at the 1 percent level.
** Significant at the 5 percent level.
* Significant at the 10 percent level.

42

Table 3: Robustness Test Results
Dependent
Adjacent Period

Credit Share Weighted

First-authored-paper Based[DD∆outcome]2[DD∆outcome]2

τ =3
(1)

τ =4
(2)

τ =5
(3)

τ =3
(4)

τ =4
(5)

τ =5
(6)

(A) Falsification testβ̃
0.29790.20390.0814
0.6315 ** -0.0969 -0.7784(0.2123) (0.1725) (0.1270)(0.2616) (0.3965) (0.4350)(B) The student proficiency score is zero if coauthored with advisor
Lower bound of σd2
0.05140.0416 * 0.0593 **0.3444 * 0.3343 ** 0.3422 ***[0.1168] [0.0803] [0.0101][0.0790] [0.0247] [0.0093](C) When a change in advisor auality variance is allowed
Lower bound of σd2
0.0848 ** 0.0608 ** 0.0536 **0.5830 * 0.5302 ** 0.4277 **[0.0326] [0.0301] [0.0220][0.0503] [0.0144] [0.0283]∧∧

Notes: The dependent variable is the squared double-diﬀerenced average student researchoutcome growth. The estimated coeﬃcient of the false advisor switch indicator, W̃n , and theestimated lower bounds of the variance in advisor quality at the doctoral level are reported.
The full estimation results are in Table E.1, Table E.2 and Table E.3 of Appendix E.2. To
make the sample balanced, a propensity score matching method is used. A logit modelis used to estimate the propensity scores. The standard errors, which are computed by
the subsampling method of Politis and Romano (1994), are in parentheses. The numbers insquare brackets are p-values for the one-sided tests such that Lower bound of σd2 = 0 against∧∧

the alternative Lower bound of σd2 > 0.
*** Significant at the 1 percent level.
** Significant at the 5 percent level.
* Significant at the 10 percent level.

43

Table 4: Estimation Results when Non-Retirement Turnover Events Are Used: The
Estimated Lower Bounds for the Variance of Advisor Quality at the Doctoral
Level
Dependent
Adjacent Period

Credit Share Weighted

First-authored-paper Based[DD∆outcome]2[DD∆outcome]2

τ =3
(1)

τ =4
(2)

τ =5
(3)

τ =3
(4)

τ =4
(5)

τ =5
(6)

(A) Baseline Case
Lower bound of σd2∧

0.1863 ** 0.1872 *** 0.1240 ***1.2319 * 1.3214 ** 1.0214 **[0.0496] [0.0035] [0.0087][0.0917] [0.0129] [0.0097](B) The student proficiency score is zero if coauthored with advisor
Lower bound of σd2
0.1482 * 0.1433 ** 0.1326 **0.90070.8801 ** 0.7196 **[0.0926] [0.0195] [0.0103][0.1019] [0.0227] [0.0164]∧

Notes: The dependent variable is the squared double-diﬀerenced average student researchoutcome growth. The estimated lower bounds of the variance in advisor quality at thedoctoral level are reported. The full estimation results are in Table E.4 and Table E.5of Appendix E.2. To make the sample balanced, a propensity score matching method
is used. A logit model is used to estimate the propensity scores. The standard errors,
which are computed by the subsampling method of Politis and Romano (1994), are inparentheses. The numbers in square brackets are p-values for the one-sided tests such that
Lower bound of σd2 = 0 against the alternative Lower bound of σd2 > 0. The standard errors, which are computed by the subsampling method of Politis and Romano (1994), are inparentheses.
*** Significant at the 1 percent level.
** Significant at the 5 percent level.
* Significant at the 10 percent level.

∧

∧

44

Table 5: Additional Evidence for Professors’ Influence on Students
Dependent

Credit Share Weighted

Adjacent Period

τ =3
τ =4
τ =5
(1)
(2)
(3)
(A) The DD Measure in Levels Is Used
0.14390.14800.0310β̂(0.1746) (0.1204) (0.0985)(B) Effect of Non-Advisor Turnoverβ̂ind-0.0230
0.02740.0527(0.0374) (0.0320) (0.0336)
π̂ 2-0.0682
0.10300.2694First-authored-paper Based
τ =3
(4)

τ =4
(5)

τ =5
(6)

0.26550.39680.2276(2.1883) (1.5374) (1.3783)
0.07640.10070.1768(0.1192) (0.0924) (0.0924)
0.03310.04720.1078Notes: The dependent variables are the double-diﬀerenced average student research outcome
growth in level and in square in panels (A) and (B), respectively. The estimated coeﬃcientsof the advisor switch indicator due to turnover, Wn , and the assignment indicator of “indirect” turnover, Vm , are reported. The full estimation results are in Table E.6 and Table E.7
of Appendix E.2. We define π̂ 2 = β̂ind /β̂dir , where β̂dir is taken from the estimates in Table 2. To make the sample balanced, a propensity score matching method is used. A logit
model is used to estimate the propensity scores. The standard errors, which are computedby the subsampling method of Politis and Romano (1994), are in parentheses.
*** Significant at the 1 percent level.
** Significant at the 5 percent level.
* Significant at the 10 percent level.

45Appendix
A

Laboratory Democracy

Low (2005) notes that professor Shouichi Sakata at the physics department of Nagoya Imperial
University, an influential physicist at that time, played an important role in developing the
new democratic lab system in the Japanese physics community. Sakata, who was under
the philosophical influence of Marxism, introduced a charter for the physics department at
Nagoya in 1946. The charter holds that democracy should serve as the guiding principle indepartment aﬀairs; all faculty members and students should be treated equally concerning
physics research (Department of Physics, Nagoya University, 2015). The idealism of Sakata’s“laboratory democracy” then spread. Soon after the Nagoya Charter was announced, several
physics departments at other universities introduced similar systems. See Tanabashi (2012)
for details on Sakata’s laboratory democracy.
Traweek (1988) reports that decision-making in Japanese physics labs was based on theconsensus of the members. Traweek, an anthropologist who studied various research groupsof elementary particle physicists in Japan and the U.S., oﬀers a first-hand account of thedemocratic nature of labs in Japan by asking group leaders of a lab for the source of new
ideas for experimental design or data analysis. She writes, (p.147) “ [lab leaders] generally
credited the graduate students ... they said the group then responds to their ideas, perhaps
modifying or amplifying them”.

46

B

The Assumption on the Random Shock

B.1

Assumptions on the Moments of the Idiosyncratic Shocks

The following assumptions impose restrictions on the moments of the idiosyncratic shock after
demeaning by each cohort.
assumption 2.1: The conditional expectation and variance of the demeaned random
′

′

c
c
shock, ν̃c , for student i ∈ I ℓ(a,c) are E(ν̃iag|W ℓ(a,c,c ) ) = 0 and Var(ν̃iag|W ℓ(a,c,c ) ) = ϕ2g ,respectively, for any a ∈ A , g ∈ {m, d}, and c, c′ ∈ C .
assumption 2.2: The covariance of the demeaned random shocks between degree pro′cc
grams within the same student, i ∈ I ℓ(a,c) , is given by Cov(ν̃iam, ν̃iad|W ℓ(a,c,c ) ) = ϕmd

for any a ∈ A , and c, c′ ∈ C .
assumption 2.3: The covariance of the demeaned random shocks between diﬀerent students i ∈ I ℓ(a,c) and j ∈ I ℓ(a,c) who are advised by the same professor in degree
′

′

′

c
c
c
c
|W ℓ(a,c,c ) ) = ψg , for any, ν̃jag|W ℓ(a,c,c ) ) = Cov(ν̃iag, ν̃jag
program g is given by Cov(ν̃iaga ∈ A , g ∈ {m, d}, and c, c′ ∈ C .
assumption 2.4: The covariance of the demeaned random shocks between diﬀerent stu′′

dents i ∈ I ℓ(a,c) and j ∈ I ℓ(a ,c ) who are advised by diﬀerent professors in degree
program g is zero, that is,
′

′

′

c
c
ℓ(a,c,c )ccℓ(a,c,c )Cov(ν̃iag, ν̃ja) = Cov(ν̃iag, ν̃ja) = 0,′ g |W′ g |W

for any a, a′ ∈ A , a ̸= a′ , g ∈ {m, d}, and c, c′ ∈ C .
assumption 2.5: The covariance of the demeaned random shocks between diﬀerent stu′′

dents i ∈ I ℓ(a,c) and j ∈ I ℓ(a ,c ) between degree programs is zero, that is,
′

′

′

′

ℓ(a,c,c )cℓ(a,c,c )ccc)=0, ν̃ja) = Cov(ν̃iad, ν̃jaCov(ν̃iam′ m |W′ d |W

for any a, a′ ∈ A , and c, c′ ∈ C .

B.2

Derivation of Equation (6)We compute the conditional expectation of the squared left-hand side of Equation (5). Undercthe assumption that the random shock, νiag
, is orthogonal to advisor quality, θg , for any

student i ∈ I ℓ(a,c) , professor a ∈ A, cohort c ∈ C and program g ∈ {m, d}, the conditional47

expectation is given as follows:[()ℓ(a,c,c′ ) 2Eℓ(a,c,c′ )]DD∆Outcome
W
[
]
′
2
ℓ(a,c,c′ )= E (θbd − θad ) W= 1 · W ℓ(a,c,c )
[{(
]
) (
)}2
′
ℓ(a,c)ℓ(a,c′ )ℓ(a,c)ℓ(a,c′ )ℓ(a,c,c′ )+Eν̄bd − ν̄am− ν̄ad− ν̄amW= 1 · W ℓ(a,c,c )
]
[{(
) (
)}2
′
ℓ(a,c)ℓ(a,c′ )ℓ(a,c,c′ )ℓ(a,c)ℓ(a,c′ )W= 0 · (1 − W ℓ(a,c,c ) ). (A.1)+Eν̄ad − ν̄am− ν̄ad− ν̄amUnder assumption 1.1-1.2, we can compute the first part of Equation (A.1 ) as follows:[
E (θbd − θad ) W
2ℓ(a,c,c′ )]= 1 = 2σd2 (1 − ρd ).

(A.2)

We turn to the second part of Equation(A.1 ), which is related to the conditional expec′

tation of when turnover occurred, W ℓ(a,c,c ) = 1. We have the following equality concerning
the value within the expectation operator:
{(
) (
)}2
ℓ(a,c)ℓ(a,c′ )ℓ(a,c)ℓ(a,c′ )ν̄bd − ν̄am− ν̄ad− ν̄am{() ()}2ℓ(a,c)ℓ(a,c′ )ℓ(a,c)ℓ(a,c′ )¯¯¯¯=ν̃bd − ν̃am− ν̃ad− ν̃am{(( ℓ(a,c′ ) )( ℓ(a,c′ ) )}( ℓ(a,c) )2( ℓ(a,c) )( ℓ(a,c) )} {( ℓ(a,c′ ) )2 ( ℓ(a,c′ ) )2ℓ(a,c) )2
ν̃¯am
ν̃¯am
+ ν̃¯ad+ ν̃¯am− 2 ν̃¯ad= ν̃¯bd+ ν̃¯am− 2 ν̃¯bd{(( ℓ(a,c) )( ℓ(a,c′ ) ) ( ℓ(a,c) )( ℓ(a,c′ ) ) ( ℓ(a,c) )( ℓ(a,c′ ) )}ℓ(a,c) )( ¯ℓ(a,c′ ) )
¯
ν̃¯am
ν̃¯ad
ν̃¯am
(A.3)
−2 ν̃bdν̃ad− ν̃¯bd− ν̃¯am+ ν̃¯am,ℓ(a,c)

where ν̃¯agℓ(a,c)

cis the lab ℓ(a, c) average of ν̃iag, and hence, we use ν̄agℓ(a,c)= ν̃¯ag+ ν̄g in thecomputation above. We take the conditional expectation of each piece of the last term ofEquation (A.3 ) under assumptions 2.1 - 2.5.
ℓ(a,t)
1. Consider the squared term of the average demeaned error, ν̃¯pg , where professor p ∈{a, b}, program g ∈ {d, m} and cohort t ∈ {c, c′ }. We have the following equation:( ℓ(a,t) )21
ν̃¯pg=  ℓ(a,t)
I
(
=

2
∑ ( )
t

ν̃pig
i∈I ℓ(a,t)

1)2  ∑

I ℓ(a,t)∑( t )2ν̃pig + 2i∈I ℓ(a,t)

j∈I ℓ(a,t) k̸=j∈I48( t )( t )ν̃jpg ν̃jpg.

ℓ(a,t)∑

Assumptions 2.1 and 2.3 lead to the following conditional expectation:[]( ℓ(a,t) )2 ℓ(p,c)ϕ2g + 2ψg
WE ν̃¯pg=1 =.
I ℓ(a,t)(A.4)ℓ(a,t)ℓ(a,t)
2. Consider the cross-term of the average demeaned errors, ν̃¯pd and ν̃¯am , between mas-

ter’s and doctoral programs within lab ℓ(a, t) for cohort t ∈ {c, c′ }, where professor
p = b if the professor switched from a to b due to turnover and p = a if not. Then, we
have:
 


∑
∑
)
)
(
(
( ℓ(a,t) )( ℓ(a,t) )
1
t
t
· 1

ν̃¯pd
ν̃¯am
=  ℓ(a,t)
ν̃ipd
ν̃iam
ℓ(a,t)IIi∈I ℓ(a,t)
i∈I ℓ(a,t)
()2  ∑∑∑ ()( t )
( t )( t )1t.
+ν̃jpg ν̃kam
=
ν̃
ν̃

I ℓ(a,t)  ℓ(a,t) ipd iamℓ(a,t)ℓ(a,t)j∈Ii∈I

k̸=j∈I

Given Assumption 2.2 , the conditional expectation is given by:]( ℓ(a,t) )( ℓ(a,t) ) ℓ(p,c)
ϕmd
ν̃¯pm
W
= 1 = ℓ(a,t) .
E ν̃¯pd
I
[

(A.5)
′

ℓ(a,c)ℓ(a,c )
3. Consider the cross-term of the average demeaned errors between ν̃¯pg and ν̃¯p′ g′ acrosscohorts c and c′ , where professors p ∈ {a, b} and p′ ∈ {a, b} and grad programs g ∈{d, m} and g ′ ∈ {d, m}. It is equal to:( ℓ(a,c) )( ℓ(a,c′ ) )ν̃¯pgν̃¯p′ g′




 
∑
∑
( c′ )( c )11=  ℓ(a,c)ν̃jp′ g′ ν̃ipg  ·  ℓ(a,c′ )II′i∈I ℓ(a,c)
j∈I ℓ(a,c )


(
)(
) ∑

∑
(
)(
)
1
1
c
c′
=
ν̃ipg ν̃jp′ g′.
I ℓ(a,c)
I ℓ(a,c′ )  ℓ(a,c)ℓ(a,c′ )i∈Ij̸=i∈IThe conditional expectation is zero under Assumption 2.4-2.5. That is:[]( ℓ(a,c) )( ℓ(a,c′ ) ) ℓ(a,c,c′ )E ν̃¯pgν̃¯p′ g′W
= 1 = 0.

(A.6)

Using results (A.4 ), (A.5 ), and (A.6 ) presented above, the conditional expectation of49

Equation (A.3 ), regardless of whether an advisor switch occurred, is equal to:
[{(
]
) (
)}2
ℓ(a,c)ℓ(a,c′ )ℓ(a,c)ℓ(a,c′ )ℓ(a,c,c′ )ν̄bd − ν̄amE
− ν̄ad− ν̄am
W
=1
(
)
)
}
(
{ 2
1
1
2
+
4
ψ
+
ψ
−
2ϕ
.
=
+
ϕ
+
ϕ
d
m
md
′
m
d
I ℓ(a,c) I ℓ(a,c )(A.7)

Similar computation reveals that the third part of Equation (A.1 ), for the case in which′

turnover did not occur, W ℓ(a,c,c ) = 0, is the right-hand side of Equation (A.7 ).
Based on Equations (A.2 ) and (A.7 ), we therefore have the following result:[(
E

DD∆Outcome)ℓ(a,c,c′ ) 2Wℓ(a,c,c′ )

]

(
=α

1
I ℓ(a,c)+()
where α = ϕ2d + ϕ2m + 4 ψd + ψm − 2ϕmd and β = 2σd2 (1 − ρd ).

50

1
I ℓ(a,c′ ))′+ β · W ℓ(a,c,c ) ,CSupplementary Materials for Section 4

C.1

On computation of coauthor’s credit share

What follows illustrates how the coauthor’s credit share is constructed. Suppose that thenames of the authors are ordered alphabetically. Then, the contribution weight is fractional:
each author receives equal credit. Suppose this alphabetical approach is not used. Then, eachauthor receives a share of credit that decreases in the authorship ranking. Following Liu andFang (2012), the credit formula is given by n−1/k r−(1−1/k) for the r-th author of a paper with nauthors. The integral constant, k, controls the declining rate of credit allocated in proportion
to that of the first author. According to the suggestion of Liu and Fang (2012), we set k = 3
for our analysis. Waltman (2012) notes that authorship could unintentionally be alphabetical,
especially when the number of authors is small, despite the authors’ intention to list theirnames based on a non-alphabetical criterion. Therefore, we account for the probabilities ofboth such incidental and intentional alphabetical authorship and use the expected value as
the final research outcome measure.

C.2

A Score of Word Overlap in Titles

As described in Section 4, our measure of a graduate student’s research achievement is basedon the number of articles that he or she published in selected physics journals.
To identify the articles that were authored by each student in the sample, we compile
physics papers from the Thomson Reuters WoS archive that satisfy the following three conditions: (1) the author names match the name of the student; (2) the publication dates are in
the period from the year in which the student was enrolled in graduate school to four years
after he or she received a doctoral degree; and (3) the words in the title overlap to someextent with those in the title of the student’s master or doctorate thesis.
The first and second conditions can be easily verified because the authors’ names andpublication dates of articles are available from the WoS database, whereas the student names
and the degree date of each student are found in the the master’s and doctoral thesis catalogs
of UTokyo’s physics department.
To enforce the third condition, we define a score that assesses the degree of overlap in thewords in titles. Let R̃i be the set of all physics articles that are associated with student i ∈ Iafter the first and second conditions presented above are satisfied. Note that, although allarticles in the set R̃i include authors whose names are the same as student i, the student may
or may not actually be the author of these articles. Such misidentification arises because offalse positives in author name matching.
We use t(rij ) to denote the title of article rij ∈ R̃i and use ti to denote the title of student
51

i’s thesis (either master’s or doctoral, depending on the context). Each title of an article ora thesis consists of words. For each article rij ∈ R̃i , we compute the following score of word
overlap in titles:
∑
{

mij =
max

∑

w∈{ti ∩t(rij )} ϕ(w)w∈ti ϕ(w),},w∈t(rij ) ϕ(w)

∑

(A.8)

where ϕ(w) is a weighting of word w that measures the rareness of the word.
Indeed, the frequency of words used in article titles varies substantially, some being common and others rare. Clearly, such information is potentially useful in deciding whether anarticle sharing the author name with a thesis is actually authored by the person who wrote
the thesis. If the words included in both the titles of an article and thesis are relatively rare,
there is a higher likelihood that the authors are the same, whereas the converse is true if thewords are relatively common.
To utilize the intuition, ϕ(w) assigns high weight to relatively rare words and low weight torelatively common words. Following a similar approach to that proposed by Tang and Walsh(2010), we determine the weight, ϕ(w), based on the relative frequency of word w, whichis computed by dividing its count frequency by the total counts of all technical terms thatappear in all titles of the master’s and doctoral theses of UTokyo’s students. Specifically, wesort all words used in titles into five categories or quintiles based on their relative frequencies.
For word wk that is in the k-th quintile, the weight is given by ϕ(wk ) = (6 − k)−2/3 for
k = 1, 2, 3, 4, 5.
One remaining issue concerns words referring to the same concept in physics that are
rendered diﬀerently. For instance, words such as “energy”, “energies”, “energetics”, and“energetic” are considered to represent the same notion. We address this issue by “standardizing” the words. Specifically, we undertake the following actions. First, we transform all
non-letter, non-Greek characters and symbols into spaces. Second, we convert all words into
lower case. Third, we reduce inflected (or derived) words to their word stem using a stemmingalgorithm.51 For instance, the stemming algorithm reduces the words “energy”, “energies”,“energetics”, and “energetic” to the unique root word, “energi”. Fourth, we eliminate allof the non-informative “stopwords”, that is, very high-frequency words such as the, to, of ,and study. For example, consider an article with the title “ENERGY-LEVEL STATISTICS
OF METALLIC FINE PARTICLES.” In this case, the title is decomposed into the set ofstandardized root words as “energi”, “level”, “statist”, “metal”, “fine” and “particl”.
We use the title word overlap score, given by Equation (A.8 ), when we identify that
51

Specifically, we use Porter’s stemming algorithm, which is the most commonly used algorithm for word
stemming in English.

52

article rij ∈ R̃i is authored by student i, depending on whether the score, mij , exceeds thepredetermined threshold, m̄. Let R̂i be the set of articles associated with student i by theword-overlapping-score method presented above such that R̂i ⊆ R̃i .

C.3

An Optimal Threshold

How can we determine the threshold, m̄, for the title word overlap score when matchingarticles and theses? Two types of matching errors are possible. We refer the first as a type 1
error, which occurs if we under-match articles, i.e., if we miss articles that are indeed authoredby a student by regarding them as being written by another author. However, the second
error, referred to as a type 2 error, arises when we include articles that are not authored bya target student. A type 1 error is likely to occur when we impose a threshold value, m̄, thatis too high, whereas a type 2 error will be more likely when we impose a low threshold, m̄,and end up with spurious matches that actually belong to diﬀerent authors.
One fundamental problem regarding the problem of identifying students’ publications isthat the true set, Ri , is unknown for student i ∈ I , and therefore, the degrees of type 1 andtype 2 errors cannot be assessed.
However, we might be able to obtain a reasonably accurate approximation set of published articles for certain students, especially for those who became academic researchers andpublished their CVs on the web. Let I¯ ⊆ I be the set of such students/researchers. We
acquired the CVs of 40 such researchers by a random web search and parsed the research
publication information to create the benchmark set of articles. Our expectation is that thebenchmark article set, R̄i , will contain reliable and comprehensive information on the true
set, Ri , at least for student/researcher i ∈ I¯. Nevertheless, the set R̄i might include somearticles that are not directly related to their thesis projects. In this regard, the benchmark
set should be close to but somewhat larger than the true set.
We use the benchmark article set to evaluate the performance of the matching procedure
based on the word overlap score in titles. Specifically, to gauge the performance at eachthreshold value, we use two goodness-of-fit indices, GOF I2a and GOF I2b, proposed by
Trajtenberg et al. (2006). Let R̄i be the benchmark set of student i ∈ I¯ and R̂i (m) the
corresponding set estimated by the matching procedure based on the word overlap score intitles, with m being the threshold value.
Those measures are defined as:[]GOF I2a(m) ≡ Average |R̄i ∩|R̄R̂ii|(m)|[]|R̄i ∩R̂i (m)|GOF I2b(m) ≡ Average |R̂ (m)| ,i

53

where the average is taken over all persons in the selected set I¯. In essence, if our matching procedure tends to under-match or over-match, GOF I2a(m) or GOF I2b(m) decrease,respectively. Therefore, we should seek to increase these indices to avoid type 1 and type 2
errors to the greatest extent possible, but a trade-oﬀ exists between the two goals.
Figure C.1 presents those two indices for various values of m in increments of 0.05.
GOF I2b(m), which is presented as a solid blue line, increases in the range of a smaller
threshold value, m, and reaches nearly 0.65 when m = 0.25 with no improvement being observed if m > 0.25. This leads to the implication that type 2 error will no longer be reduced
dramatically if we set m > 0.25. Turning to GOF I2a(m), which is presented as a dashed redline, it decreases consistently as the threshold value, m, rises, implying that type 1 error willbe alleviated as the value of m decreases.
Accordingly, we consider the optimal threshold to be m̄ = 0.25, as this is the value thatbalances the two goodness-of-fit measures — GOF I2a(m) is maximized (thus, type 1 error
is minimized) on the condition that GOF I2b(m) remains at a high level (thus, an increase intype 2 error is reduced as much as possible).

0

.2

.4

.6

Figure

.1

.15

.2

.25

.3
thresholdGOFI2a

.35

.4

.45

.5

GOFI2bFigure C.1: Comparison of Two Goodness-of-Fit Indices over Various Thresholds for the Word Overlap Score in Titles
Notes: Two goodness-of-fit indices, GOF I2a and GOF I2b proposed by Trajtenberg et al.
(2006), are plotted for various values of the threshold value of the matching procedure basedon the word overlap score in titles in increments of 0.05.

54

D

Supplementary Materials for Section 5

D.1

Discussion on Overstated Supervisory Period

In this subsection we discuss the eﬀect of overstated supervisory period on the the lower
bound estimate of the variance of advisor quality.
Consider a situation where advisor turnover happened in lab ℓ(a, c) and professor a left
the program and professor b took over the students who were left behind. Let i be a student
in lab ℓ(a, c) and let λi be a number between zero and one that captures the timing of student
i’ advisor switch. Figure D.1 illustrates the situation. Student i was supervised by professor
a at the master’s degree course and the earlier part of the doctoral course. Since advisor
switch happened in the middle of the doctoral course, the remaining proportion of (1 − λi )of the doctoral course period was supervised by the newly assigned professor b. Assume forsimplicity that λi is random and independent of the factors that aﬀect an advisee’s research
performance as well as the incident of advisor switch. Note that, in the baseline specification,
we assume that λi = 0.
Given mixed influence on research performance from two diﬀerent advisors,the researchoutcome growth of student i is given byc∆outcomeciad = γi + λi θad + (1 − λi )θbd + νiad
.

(D.1)

It should be noted that the specification above encompasses the case of “hidden” supervision
where the advisor who left the program due to turnover continued to provide research guidanceon the former student, and the professor who was recorded as the research advisor on the thesis
catalog was indeed a “surrogate” of the true advisor. In this case, we interpret the parameterλi as the magnitude of the research influence on student i from the true but “hidden” advisor
a. In other words, (1 − λi ) represents the fraction of the research achievement growth ofstudent i that is attributable to the nominal “surrogate” professor b.
It follows that the DD average student research outcome growth between two labs, ℓ(a, c)and ℓ(a, c′ ), conditional on the advisor switch due to turnover, W ℓ(a,c) = 1, is given byℓ(a,c,c′ )DD∆outcome)(= 1 − λ̄ℓ(a,c) (θbd − θad ) + error term

(D.2)

where λ̄ℓ(a,c) is the average of λi over students in lab ℓ(a, c) and the definitions of the othervariables are the same as those shown in Equation (5) in Section 3.2. Then, we have the55following conditional expectation of the squared left-hand side of Equation (D.2 )[()ℓ(a,c,c′ ) 2ℓ(a,c,c′ )]DD∆OutcomeW[]′ℓ(a,c) 22ℓ(a,c,c′ )= E (1 − λ̄) (θbd − θad ) W= 1 · W ℓ(a,c,c )
]
[{(
)}2
) (
′
ℓ(a,c)ℓ(a,c′ )ℓ(a,c′ )ℓ(a,c,c′ )ℓ(a,c)+Eν̄ad − ν̄am− ν̄ad− ν̄amW= 1 · W ℓ(a,c,c )
]
[{(
)}2
) (
′
ℓ(a,c′ )ℓ(a,c)ℓ(a,c,c′ )ℓ(a,c′ )ℓ(a,c)
W= 0 · (1 − W ℓ(a,c,c ) ). (D.3)− ν̄ad− ν̄am+Eν̄ad − ν̄am
E

The first part of Equation (D.3 ) is the same as the one found in Equation (A.1 ) in the
Appendix B.2, except that the squared diﬀerence of advisor qualities, (θbd −θad )2 , is multiplied()2
by 1 − λ̄ℓ(a,c) , while the second and third part of Equation (D.3 ) is exactly the same as
the corresponding terms of Equation (A.1 ). Under the assumptions presented in Section 3.2,
the first part of Equation (D.3 ) is equal to(1 − λ̄l(a,c) )2 × 2σd2 (1 − ρd ).

(D.4)

Considering the agreement between Equation (A.1 ) and Equation (D.3 ) except the firstterm, we therefore have the following result:[(
E

DD∆Outcome)ℓ(a,c,c′ ) 2Wℓ(a,c,c′ )

(

]
=α

1

N

+
ℓ(a,c)1Nℓ(a,c′ ))′+ β̌ · W ℓ(a,c,c ) ,(D.5)()
where α = ϕ2d + ϕ2m + 4 ψd + ψm − 2ϕmd and β̌ = 2σd2 (1 − ρd )(1 − λ̄l(a,c) )2 .
()2
If we turn to the regression equation where DD∆Outcomen is regressed on advisor
switch indicator variable Wn , as presented in Equation (7) in Section 3.2, the coeﬃcient β̌ of
Wn relates to the variance through the following equationβ̌ = 2σd2 (1 − ρd )(1 − λ̄l(a,c) )2(D.6)

Since correlation is imperfect (|ρd | < 1) and supervision is partial (0 < λ̄d < 1), a lower boundestimate is still given by the last term of the following inequalityσ̂d2 =

β̌
β̌
β̌
≥ .
≥
l(a,c)
2
2(1 − ρd )42(1 − ρd )(1 − λ̄
)

(D.7)

∧

That is, Lower bound of σd2 is given by one-fourth of the estimated coeﬃcient of the advisor
switch indicator variable in the regression model.

56

Figuretimelineturnoverprofessor a
Master’s
degreeDoctoraldegreeℓ(a, c)λi

professor b
Figure D.1: Example of Partial Supervisory Period
Notes: Advisor turnover happened in lab ℓ(a, c) and professor a left the program and professor b took over the students who were left behind. Let λi be a number between zero andone that captures the timing of student i’ advisor switch.

D.2

Discussion on Non-random Turnover(1) Evidence suggesting that the sample is not balanced
Table D.1 reports the descriptive statistics for some characteristics of advisors and compares
those of advisors when turnover occurred and the corresponding advisor characteristics when
it did not. We find that, for some characteristics, the diﬀerences in means between thetwo groups, professors with turnover in column (1) and those without in column (2), arestatistically significant at the 5 percent level. We also find that the absolute values of thestandardized diﬀerences, reported in column (3), are large for some characteristics.52(2) A procedure for the propensity score matching method
To make the sample balanced and comparable, we thus employ a propensity score matching
method. Following standard practice in the literature, we estimate the propensity scores using
a logit model. - We include all of the characteristics presented in Table D.1 scores. Specifically,in the first step, we begin with a set of basic covariates and add an additional linear term basedon a likelihood ratio test for the null hypothesis that the coeﬃcient of the added variable is52

The standardized diﬀerence considers the size of the diﬀerence in means of a conditioning variable, scaledby the square root of the variances of the treatment and control groups in the original sample. According to
the suggestion of Rosenbaum and Rubin (1985), an absolute value of the standardized diﬀerence greater than
0.2 should be considered “large”.

57equal to zero. In the second step, we proceed to the choice of the quadratic and cross-productterms and apply the same type of likelihood test as that used in the first step. We follow
the suggestion of Imbens and Rubin (2015) that the threshold values for the likelihood ratio
test should be CL = 1.0 and CQ = 2.71 for the linear and quadratic terms, respectively. The
results of the logit estimation of the propensity score can be found in Table XX Given theestimated propensity scores, we match a case with Wn = 1 (a lab with an advisor switch)
to one with Wn = 0 (a lab without an advisor switch) that share approximately identicalestimated propensity scores. We employ a one-to-one nearest-neighbor matching method.
To assess the quality of the propensity score matching, we present Figure D.3 that depicts
the absolute values in the standardized diﬀerences of the variables for the original and matchedsamples. The imbalance between the treatment and control cases is attenuated on many
professor characteristics. For example, professor’s age diﬀers between the treatment andcontrol labs by more than the average standard deviation (the absolute standard deviation is1.129) before matching, whereas the diﬀerence is considerably reduced (the absolute standard
deviation is 0.006) after matching.
Figure D.4 presents the distributions of the estimated propensity scores for the treatmentlabs (left) and control labs (right) in each case of the adjacent period, τ = 3, 4, and 5. The top
and bottom groups in the graphs correspond to those before and after matching, respectively.
Before matching, the shapes of distributions diﬀer considerably between the treatment andcontrol groups. Nevertheless, the propensity score distributions have some degree of overlap.
Moreover, after matching, the dissimilarity of the distributions between the treatment andcontrol groups is considerably reduced.
One might worry that the spread of the common support of the propensity score distributions should not be across the full range [0 1] and hence that the observations of the treatmentgroup, especially those with high propensity scores, are matched forcibly with those of thecontrol group, the propensity scores of which are not suﬃciently close. To address the problem caused by the limited common support of the propensity score distribution, we employ
a systematic approach proposed by Crump et al. (2009) and discard all observations withestimated propensity scores outside the range of [0.1 0.9].

58

0

1

Research Outcome
2
3

4

5

Figuresmaster’s level

doctoral level

Figure D.2: Box Plots of the Student Research Outcome Distributions in the
Master’s and Doctoral Programs
Notes: Student research outcomes are aggregated over the period from M1 to D2 for the
master’s degree, and the period from D1 to P4 for doctoral degree program.

Age
Num_Stud
Outcome5
Rank_Assoc
Rank_Prof
Dept_Phys
Inst_Solid
Inst_Other
Period_70s
Period_80s
Period_90s
Period_00s
0

.2

.4

.6

Before Matching

.8

1

1.2

After Matching

Figure D.3: Comparison of the Absolute Values of the Standardized Diﬀerences
between Treatment and Control Groups
Notes: The standardized diﬀerence considers the size of the diﬀerence in means of a conditioning variable, scaled by the square root of the variances of the treatment and control
groups in the original sample. According to the suggestion of Rosenbaum and Rubin (1985),
an absolute value of the standardized diﬀerence greater than 0.2 should be considered large.

59

Figure D.4: Distribution of the Propensity Score for the Treatment and Control
Groups: Before and After Matching
Notes: The distribution of the estimated propensity scores are presented for the treatmentlabs and control labs (right) in each case of the adjacent period, τ = 3, 4, and 5. The top andbottom groups in the graphs correspond to those before and after matching, respectively.

60

Tables
Table D.1: Descriptive Statistics of Advisors: Comparison between Advisors When TurnoverOccurred and When It Did Not

Variable

Description

With
Without
Turnover Turnover

Age

Professor Age

N um Stud

Number of Students

53.72(6.37)
1.16(0.41)
0.18(0.29)
0.21(0.41)
0.79(0.41)
0.72(0.45)
0.72(0.41)
0.34(0.48)
0.15(0.36)
0.16(0.37)
0.57(0.50)
0.12

Outcome5

Profeessor’s Research Outcome(5 years average)
Rank Assoc Associate Professor Dummy

Rank P rof Full Professor Dummy
Dept P hys

Department of Physics DummyInst Solid

Institute of Solid State DummyInst Other

Other Institutes Dummy

P eriod 70s

70’s Dummy

P eriod 80s

80’s Dummy

P eriod 90s

90’s Dummy

P eriod 00s

00’s Dummy

47.06(5.77)
1.27(0.52)
0.21(0.40)
0.44(0.50)
0.53(0.50)
0.75(0.43)
0.22(0.41)
0.29(0.45)
0.21(0.41)
0.26(0.44)
0.35(0.48)
0.19

t-statAbsolute
Standardized
Diﬀerence-9.00 ***

1.10

1.63

0.22

0.50

0.07

3.73 ***

0.51-4.27 ***

0.59

0.62

0.08

0.27

0.03

-0.85

0.11

1.19

0.16

1.71 *

0.23-3.72 ***

0.46

1.46

0.20

Notes: The absolute standardized diﬀerence is given by the size of the diﬀerence in meansof a conditioning variable, scaled by the square root of the variances in the original samples(Rosenbaum and Rubin 1985).
*** Significant at the 1 percent level.
** Significant at the 5 percent level.
* Significant at the 10 percent level.

61

Table D.2: Estimation Results of Propensity Score
Adjacent PeriodAge
N um StudRank AssocInst Other
P eriod 90s
P eriod 00s
Age2
N um Stud2
Age × P eriod 80s
N um Stude × P eriod 80s
Outcome5 × Inst Other
Rank Assoc × Inst SolidInst Solid × P eriod 00s
Inst Other × P eriod 80s
Constant
Sample Size

τ =3-5.823[9.70]
21.900[0.02]
0.832[0.76]-0.208[0.46]
1.014[1.99]-1.724[2.67]
0.063[10.11]-7.263[0.02]
1.465[2.47]-80.110[2.43]-1.816[1.76]
3.487[3.02]-3.700[2.95]
111.7[0.16]
1446

***

**
***
***

τ =4
-6.494[9.39]-0.701[1.86]
1.109[1.00]-0.462[0.99]
1.777[4.11]-1.375[2.44]
0.072[9.58]

***
*

***
**
***

**
**
*

***

-2.325[2.06]
2.346[1.90]
3.168[2.29]

**
*

1202

1.157
[1.56]-63.27[1.53]-3.059[1.99]
2.941[1.83]

***

*
***

**
*

**

***

138.900[8.98]

τ =5-6.784[6.55]-0.212[0.46]-0.552[0.26]-0.308[0.46]
1.101[1.49]-1.782[1.92]
0.073[6.84]***-2.757[1.83]
148.800[6.05]****

925

Notes: The dependent variable is the advisor switch indicator due to turnover, Wn . The
definitions of the independent variables are given in Table D.1. The specification of themodel is given by a stepwise likelihood-test-based procedure, suggested by Imbens (2014)
and Imbens and Rubin (2015).
*** Significant at the 1 percent level.
** Significant at the 5 percent level.
* Significant at the 10 percent level.

62

E

Supplementary Materials for Section 6

E.1

Indirect Influence from Non-advisor

We explain an indirect eﬀect from non-advisor faculty members on students across labs by
Figure E.1 illustrate, which parallels that illustrated in Figure 2. As we have assumed previously, there are three cohorts, c0 , c1 and c2 where an instance of turnover involving professor
a occurred in cohort c2 , and the students in lab ℓ(a, c2 ) switched their research advisor fromprofessor a to professor b in the doctoral program. Then, professor b, whose research areais the same as that of professor a, took over the students in lab ℓ(a, c2 ), whereas he hadsupervised two labs, ℓ(b, c0 ) and ℓ(b, c1 ), before the incident occurred, and oversaw anotherlab, ℓ(b, c2 ), at the time of the incident. We assume that professor a’s turnover aﬀects thedoctoral research productivity of the students in lab ℓ(b, c2 ) because the indirect influence
from the professor, θad , ceases to exist after turnover.
In the estimation that follows, we choose the lab of professor b that was influenced “indirectly” by professor a’s turnover if the turnover occurred while the students in the lab werein the doctoral program (i.e., from the first doctoral year to the final year of the doctoral
program). We require this because the indirect influence from professor a at the doctoral
degree level, not the master’s degree level, needs to be changed. In this case, to identify themagnitude of the indirect impact, we essentially compare the gap in student research outcome
growth between labs ℓ(b, c2 ) and ℓ(b, c1 ) (treatment group with V ℓ(b,c2 ,c1 ) = 1 ) with the same
gap between labs ℓ(b, c1 ) and ℓ(b, c0 ) (control group with V ℓ(b,c1 ,c0 ) = 0).

63

Figuretimelineturnoverprofessor aℓ(a, c0 )ℓ(a, c1 )ℓ(a, c2 )professor bℓ(b, c0 )indirect effect fromℓ(b, c2 ) non-advisor's turnoverℓ(b, c1 )Figure E.1: Example of Labs with and without Turnover for Advisors and NonAdvisors
Notes: There are three cohorts, c0 , c1 and c2 where an instance of turnover involving
professor a occurred in cohort c2 , and the students in lab ℓ(a, c2 ) switched their researchadvisor from professor a to professor b in the doctoral program. Then, professor b, whose
research area is the same as that of professor a, took over the students in lab ℓ(a, c2 ), whereas
he had supervised two labs, ℓ(b, c0 ) and ℓ(b, c1 ), before the incident occurred, and oversaw
another lab, ℓ(b, c2 ), at the time of the incident.

64

E.2

Supplementary Tables

Table E.1: Estimation Results Regarding Diﬀerent Specifications of Student Research
Outcomes: The Estimated Lower Bound of the Variance in Advisor Quality
at the Doctoral Level Is Reported for the Case of Twelve Journals
Dependent
Adjacent Period

Credit Share Weighted

First-authored-paper Based[DD∆outcome]2[DD∆outcome]2

τ =3
(1)

τ =4
(2)

τ =5
(3)

Threshold 0.25 (default)
M1-D2/D1-P4†‡
0.0843 ** 0.0666 ** 0.0489 **[0.0268] [0.0135] [0.0236]
M1-D2/D1-P3†
0.0689 ** 0.0498 ** 0.0257 *[0.0232] [0.0181] [0.0973]
M1-D1/D1-P4†
0.1460 ** 0.1243 ** 0.0910 **[0.0488] [0.0210] [0.0389]
M1-D1/D1-P3†
0.1215 ** 0.0966 ** 0.0572 *[0.0485] [0.0271] [0.0913]
Threshold 0.20 (overmatch)
M1-D2/D1-P4†
0.0953 ** 0.0790 ** 0.0576 **[0.0407] [0.0184] [0.0341]
M1-D2/D1-P3†
0.0621 ** 0.0425 ** 0.0205[0.0323] [0.0341] [0.1505]
M1-D1/D1-P4†
0.1659 * 0.1466 ** 0.1214 **[0.0580] [0.0231] [0.0247]
M1-D1/D1-P3†
0.1159 * 0.0913 ** 0.0639 *[0.0565] [0.0349] [0.0665]
Threshold 0.30 (undermatch)
M1-D2/D1-P4†
0.0509 * 0.0378 ** 0.0191[0.0712] [0.0476] [0.1037]
M1-D2/D1-P3†
0.0401 * 0.0282 * 0.0023[0.0719] [0.0609] [0.4395]
M1-D1/D1-P4†
0.1123 * 0.0988 ** 0.0683 *[0.0599] [0.0237] [0.0541]
M1-D1/D1-P3†
0.0923 * 0.0783 ** 0.0360[0.0582] [0.0261] [0.1507]

τ =3
(4)

τ =4
(5)

τ =5
(6)

0.5773 ** 0.5331 ** 0.4100 **[0.0407] [0.0104] [0.0234]
0.4578 ** 0.4099 *** 0.2793 **[0.0310] [0.0080] [0.0338]
0.9274 ** 0.8522 ** 0.7525 **[0.0450] [0.0133] [0.0145]
0.7711 ** 0.6911 ** 0.5745 **[0.0384] [0.0119] [0.0175]
0.7246 ** 0.6833 ** 0.5492 **[0.0460] [0.0115] [0.0213]
0.4672 ** 0.4179 *** 0.2641 **[0.0280] [0.0071] [0.0438]
1.1114 ** 1.0467 ** 2.2262 ***[0.0490] [0.0134] [0.0085]
0.7805 ** 0.7055 ** 0.6066 **[0.0364] [0.0105] [0.0127]
0.3490 ** 0.3135 *** 0.1795 *[0.0222] [0.0042] [0.0548]
0.2662 ** 0.2282 *** 0.0884[0.0110] [0.0019] [0.1246]
0.6166 ** 0.5541 *** 0.4385 **[0.0345] [0.0094] [0.0220]
0.4971 ** 1.0168 *** 0.3002 **[0.0265] [0.0094] [0.0331]

Notes: The dependent variable is the squared double-diﬀerenced average student researchoutcome growth. The estimated lower bounds of the variance in advisor quality at thedoctoral level, Lower bound of σd2 are reported. To make the sample balanced, a propensity
score matching method is used. A logit model is used to estimate the propensity scores.
The standard errors that are computed by the subsampling method of Politis and Romano(1994) are in parentheses. The numbers in square brackets are p-values for the one-sided
tests such that Lower bound of σd2 = 0 against the alternative Lower bound of σd2 > 0. The
standard errors that are computed by the subsampling method of Politis and Romano (1994)
are in parentheses.
† (master’s level aggregation period) / (doctoral level aggregation period)‡ The baseline cases.
*** Significant at the 1 percent level.
** Significant at the 5 percent level.
* Significant at the 10 percent level.
∧

∧

∧

65

Table E.2: Estimation Results Regarding Diﬀerent Specifications of Student Research
Outcomes: The Estimated Lower Bound of the Variance in Advisor Quality
at the Doctoral Level Is Reported for the Case of Nine Journals
Dependent

Credit Share Weighted

First-authored-paper Based[DD∆outcome]2[DD∆outcome]2

Adjacent Period

τ =3
(1)

M1-D2/D1-P4†

0.0449 *** 0.0363 *** 0.0211 ***[0.0005] [0.0000] [0.0079]
0.0377 *** 0.0262 *** 0.0070[0.0027] [0.0000] [0.2448]
0.0829 *** 0.0675 *** 0.0355 **[0.0080] [0.0017] [0.0431]
0.0680 *** 0.0495 *** 0.0115[0.0085] [0.0026] [0.2533]

M1-D2/D1-P3†
M1-D1/D1-P4†
M1-D1/D1-P3†

τ =4
(2)

τ =5
(3)

τ =3
(4)

τ =4
(5)

τ =5
(6)

0.1862 *** 0.1591 *** 0.0241[0.0039] [0.0003] [0.3100]
0.1403 *** 0.1117 *** 0.0753 **[0.0010] [0.0000] [0.0449]
0.4216 ** 0.3441
0.2079 **[0.0130] [0.5151] [0.0395]
0.3389 *** 0.2588 *** 0.1168 *[0.0084] [0.0025] [0.0873]

Notes: The dependent variable is the squared double-diﬀerenced average student researchoutcome growth. The estimated lower bounds of the variance in advisor quality at thedoctoral level, Lower bound of σd2 are reported. To make the sample balanced, a propensity
score matching method is used. A logit model is used to estimate the propensity scores.
The standard errors that are computed by the subsampling method of Politis and Romano(1994) are in parentheses. The numbers in square brackets are p-values for the one-sided
tests such that Lower bound of σd2 = 0 against the alternative Lower bound of σd2 > 0. The
standard errors that are computed by the subsampling method of Politis and Romano (1994)
are in parentheses.
† (master’s level aggregation period) / (doctoral level aggregation period)*** Significant at the 1 percent level.
** Significant at the 5 percent level.
* Significant at the 10 percent level.
∧

∧

∧

66

Table E.3: Falsification Test Results (Full Results)
Dependent
Adjacent Period(1) α̃
(2) β̃

Credit Share Weighted

First-authored-paper Based[DD∆outcome]2[DD∆outcome]2

τ =3
τ =4
τ =5
(1)
(2)
(3)
0.1827 *** 0.1902 *** 0.1736 ***(0.0418) (0.0409) (0.0443)
0.29790.20390.0814(0.2123) (0.1725) (0.1270)

Sample Size
Total
After matching

887
422

1136
763

1351
603

τ =3
τ =4
τ =5
(4)
(5)
(6)
0.2819 *** 0.6911 *** 0.8996 ***(0.0458) (0.1828) (0.2287)
0.6315 ** -0.0969-0.7784(0.2616) (0.3965) (0.4350)
887
422

1136
763

1351
603

Notes: The dependent variable is the squared double-diﬀerenced average student researchoutcome growth. The total sample size is given by the number of observations for eachtuple of (a, c, c′ ) for any advisor a in A and cohort c, c′ such that 0 < c − c′ ≤ τ where τ isthe period over which the diﬀerence is taken. To make the sample balanced, a propensity
score matching method is used. A logit model is used to estimate the propensity scores. The
after-matching sample size is the sum of the numbers of observations for the treatment groupwhere turnover occurred and the corresponding control group that are matched throughthe propensity score method. The standard errors that are computed by the subsampling
method of Politis and Romano (1994) are in parentheses. The numbers in square brackets
are p-values for the one-sided tests such that Lower bound of σd2 = 0 against the alternative∧∧

Lower bound of σd2 > 0.
*** Significant at the 1 percent level.
** Significant at the 5 percent level.
* Significant at the 10 percent level.

67

Table E.4: Estimation Results: The Student Proficiency Score is Set to Zero If the Student
Coauthored with the Advisor (Full Results)
Dependent
Adjacent Period(1) α̂
(2) β̂

Credit Share Weighted

First-authored-paper Based[DD∆outcome]2[DD∆outcome]2

τ =3
τ =4
τ =5
(1)
(2)
(3)
0.0544 *** 0.0670 *** 0.0439 ***(0.0230) (0.0225) (0.0091)
0.20570.16650.2374 **(0.1727) (0.1187) (0.1022)

τ =3
(4)
0.0215(0.0448)1.3776(0.9758)

τ =4
τ =5
(5)
(6)
0.0713 * 0.0528 **(0.0419) (0.0250)
1.3374 ** 1.3690 **(0.6807) (0.5814)

0.0514[0.1168]

0.0416 *[0.0803]

0.0593 **[0.0101]

0.3444 * 0.3343 ** 0.3422 ***[0.0790] [0.0247] [0.0093]

925
104

1202
186

1446
271

∧

(3) Lower bound of σd2

Sample Size
Total
After matching

925
104

1202
186

1446
271

Notes: The dependent variable is the squared double-diﬀerenced average student researchoutcome growth. The total sample size is given by the number of observations for eachtuple of (a, c, c′ ) for any advisor a in A and cohort c, c′ such that 0 < c − c′ ≤ τ where τ isthe period over which the diﬀerence is taken. To make the sample balanced, a propensity
score matching method is used. A logit model is used to estimate the propensity scores. The
after-matching sample size is the sum of the numbers of observations for the treatment groupwhere turnover occurred and the corresponding control group that are matched throughthe propensity score method. The standard errors that are computed by the subsampling
method of Politis and Romano (1994) are in parentheses. The numbers in square brackets
are p-values for the one-sided tests such that Lower bound of σd2 = 0 against the alternative∧∧

Lower bound of σd2 > 0.
*** Significant at the 1 percent level.
** Significant at the 5 percent level.
* Significant at the 10 percent level.

68

Table E.5: Estimation Results: When a Change in Advisor Quality Variance Is Allowed
during the Period Near Turnover (Full Results)
Dependent
Adjacent Period(1) α̂
(2) β̂
(3) δ̂1(4) δ̂2(5) δ̂3

Credit Share Weighted

First-authored-paper Based[DD∆outcome]2
τ =3
τ =4
τ =5
(1)
(2)
(3)
0.0717 *** 0.0604 *** 0.0982 ***(0.0234) (0.0156) (0.0144)
0.3391 * 0.2431 * 0.2145 **(0.1839) (0.1294) (0.1065)—-0.1207-0.0085(0.032)(0.022)-0.15120.4291-0.2504(0.2259)(0.296)(0.152)-0.1195——(0.0431)[DD∆outcome]2
τ =3
τ =4
τ =5
(4)
(5)
(6)
0.06220.1395 ** 0.4297 ***(0.0674) (0.0614) (0.0752)
2.33202.1209 ** 1.7108 *(1.4204) (0.9704) (0.8976)—-0.2789-0.5772(0.123)(0.181)-0.4563-0.1015-1.4842(1.5777) (0.938)(1.093)-0.1036——(0.0843)0.0848 ** 0.0608 ** 0.0536 **[0.0326][0.0301][0.0220]0.5830 * 0.5302 ** 0.4277 **[0.0503] [0.0144] [0.0283]∧(6) Lower bound of σd2

Sample Size
Total
After matching

925
104

1202
186

1446
271

925
104

1202
186

1446
271

Notes: The dependent variable is the squared double-diﬀerenced average student researchoutcome growth. The total sample size is given by the number of observations for eachtuple of (a, c, c′ ) for any advisor a in A and cohort c, c′ such that 0 < c − c′ ≤ τ where τ isthe period over which the diﬀerence is taken. To make the sample balanced, a propensity
score matching method is used. A logit model is used to estimate the propensity scores. The
after-matching sample size is the sum of the numbers of observations for the treatment groupwhere turnover occurred and the corresponding control group that are matched throughthe propensity score method. The standard errors that are computed by the subsampling
method of Politis and Romano (1994) are in parentheses. The numbers in square brackets
are p-values for the one-sided tests such that Lower bound of σd2 = 0 against the alternative∧∧

Lower bound of σd2 > 0.
*** Significant at the 1 percent level.
** Significant at the 5 percent level.
* Significant at the 10 percent level.

69

Table E.6: Estimation Results when Non-Retirement Turnover Events Are Used: Baseline
Case (Full Results)
Dependent
Adjacent Period(1) α̂
(2) β̂
∧(3) Lower bound of σd2

Sample Size
Total
After matching

Credit Share Weighted

First-authored-paper Based[DD∆outcome]2[DD∆outcome]2

τ =3
τ =4
τ =5
(1)
(2)
(3)
-0.00470.01040.0659 **(0.0369) (0.0174) (0.0264)
0.7453 * 0.7490 *** 0.4959 **(0.4522) (0.2782) (0.2085)
0.1863 ** 0.1872 *** 0.1240 ***[0.0496] [0.0035][0.0087]

894
27

1154
52

τ =3
τ =4
τ =5
(4)
(5)
(6)
0.09530.09960.3749 ***(0.2117) (0.1453) (0.1269)
4.92775.2857 ** 4.0857 **(3.7047) (2.3709) (1.7469)
0.0000 * 0.9448 ** 0.9980 **[0.0000] [0.0907] [0.0110]

1381
85

894
27

1154
52

1381
85

Notes: The dependent variable is the squared double-diﬀerenced average student researchoutcome growth. The total sample size is given by the number of observations for eachtuple of (a, c, c′ ) for any advisor a in A and cohort c, c′ such that 0 < c − c′ ≤ τ where τ isthe period over which the diﬀerence is taken. To make the sample balanced, a propensity
score matching method is used. A logit model is used to estimate the propensity scores. The
after-matching sample size is the sum of the numbers of observations for the treatment groupwhere turnover occurred and the corresponding control group that are matched throughthe propensity score method. The standard errors that are computed by the subsampling
method of Politis and Romano (1994) are in parentheses. The numbers in square brackets
are p-values for the one-sided tests such that Lower bound of σd2 = 0 against the alternative∧∧

Lower bound of σd2 > 0.
*** Significant at the 1 percent level.
** Significant at the 5 percent level.
* Significant at the 10 percent level.

Table E.7: Estimation Results when Non-Retirement Turnover Events Are Used: The
Student Proficiency Score is Set to Zero If the Student Coauthored with theAdvisor (Full Results)
Dependent
Adjacent Period(1) α̂
(2) β̂
∧(3) Lower bound of σd2

Sample Size
Total
After matching

Credit Share Weighted

First-authored-paper Based[DD∆outcome]2[DD∆outcome]2

τ =3
τ =4
τ =5
(1)
(2)
(3)
-0.03160.00530.0436(0.0298) (0.0171) (0.0218)
0.59290.57310.5303(0.4475) (0.2777) (0.2292)
0.1482 * 0.1433 ** 0.1326 **[0.0926] [0.0195] [0.0103]

894
27

1154
52

1381
85

70

τ =3
(4)
-0.1398(0.1937)
3.6030(2.8352)0.9007[0.1019]

894
27

τ =4
τ =5
(5)
(6)
0.07850.1862(0.1079) (0.0760)
3.52052.8782(1.7587) (1.3485)
0.8801 ** 0.7196 **[0.0227] [0.0164]

1154
52

1381
85

Table E.8: Estimation Results: the Double-Diﬀerence Measure in Levels Is Used as the
Dependent Variable (Full Results)
Dependent
Adjacent Period(1) α̂
(2) β̂
Sample Size
Total
After matching

Credit Share Weighted

First-authored-paper Based[DD∆outcome][DD∆outcome]

τ =3
(1)
-0.0462(0.0237)
0.1439(0.1746)τ =4(2)-0.0186(0.0225)0.1480(0.1204)τ =5(3)
0.0362 ***(0.0134)
0.0310(0.0985)τ =3(4)-0.0655(0.0961)0.2655(2.1883)τ =4(5)-0.0693(0.0856)
0.3968(1.5374)τ =5(6)0.0963(0.0893)0.2276(1.3783)

925
104

1202
186

1446
271

925
104

1202
186

1446
271

Notes: The dependent variable is the squared double-diﬀerenced average student researchoutcome growth. The total sample size is given by the number of observations for eachtuple of (a, c, c′ ) for any advisor a in A and cohort c, c′ such that 0 < c − c′ ≤ τ where τ isthe period over which the diﬀerence is taken. To make the sample balanced, a propensity
score matching method is used. A logit model is used to estimate the propensity scores. The
after-matching sample size is the sum of the numbers of observations for the treatment groupwhere turnover occurred and the corresponding control group that are matched throughthe propensity score method. The standard errors that are computed by the subsampling
method of Politis and Romano (1994) are in parentheses. The numbers in square brackets
are p-values for the one-sided tests such that Lower bound of σd2 = 0 against the alternative∧∧

Lower bound of σd2 > 0.
*** Significant at the 1 percent level.
** Significant at the 5 percent level.
* Significant at the 10 percent level.

71

Table E.9: Estimation Results: Eﬀect of Non-Advisor Turnover on Student Research Outcome Growth at the Doctoral Level (Full Results)
Dependent
Adjacent Period

Credit Share Weighted

First-authored-paper Based[DD∆outcome]2[DD∆outcome]2

τ =3
τ =4
τ =5
(1)
(2)
(3)
0.0434 *** 0.0356 *** 0.0335 ***(0.0175) (0.0056) (0.0060)-0.0230
0.02740.0527(0.0374) (0.0320) (0.0336)

τ =3
τ =4
τ =5
(4)
(5)
(6)
0.0988 *** 0.0863 *** 0.0755 ***(0.0358) (0.0094) (0.0136)
0.07640.10070.1768 *(0.1192) (0.0924) (0.0924)(3) π̂ 2 = β̂ind /β̂dir-0.0682

0.10300.26940.03310.04720.1078Sample Size
Total
After matching

858
145

1105
282

1317
288

858
145

1105
282

1317
288

(1) α̂ind(2) β̂indNotes: The dependent variable is the squared double-diﬀerenced average student researchoutcome growth. The total sample size is given by the number of observations for eachtuple of (a, c, c′ ) for any advisor a in A and cohort c, c′ such that 0 < c − c′ ≤ τ where τ isthe period over which the diﬀerence is taken. To make the sample balanced, a propensity
score matching method is used. A logit model is used to estimate the propensity scores. The
after-matching sample size is the sum of the numbers of observations for the treatment groupwhere turnover occurred and the corresponding control group that are matched throughthe propensity score method. The standard errors that are computed by the subsampling
method of Politis and Romano (1994) are in parentheses. The numbers in square brackets
are p-values for the one-sided tests such that Lower bound of σd2 = 0 against the alternative∧∧

Lower bound of σd2 > 0.
*** Significant at the 1 percent level.
** Significant at the 5 percent level.
* Significant at the 10 percent level.

72