Under The Influence? Intellectual Exchange in Political Science∗

March 18, 2007

Abstract
We study the performance of political science journals in terms of their contribution to intellectual exchange in the discipline. Relying on the interplay of citation patterns, our method is
simple, cheap, objective and captures the influence of journals in a meaningful way. We findthat the American Political Science Review , World Politics and International Organization
lead the field. In contrast to previous subjective studies, we also find that economics is a key
influence on political science, much more so than sociology.

1Introduction

Intellectual exchange is central to progress in any discipline, including political science. The transfer of knowledge, ideas and techniques takes place in many forums (e.g. advisor-student meetings,
conferences, department lounges) and it is no simple task to systematically identify or quantify
this interchange. In general though, the fruition of a successful or insightful idea is a publishedjournal article or book. The way in which the author(s) of a published piece of work acknowledges
previous or contemporary work that contributed to its development is via references, or citations.
Thus, while we cannot easily keep track of the entire process of intellectual exchange that leads topublication, citations inform us of other (usually published) work that influenced and contributed
to the articles and books that make up the research output of the field.

∗

This version: March 18, 2007. We thank two anonymous referees for helpful comments on content and structure.

1

Peer-reviewed journals are far and away the most prestigious place for scholars to publish theirresearch.1 Almost all cutting edge research in political science that becomes highly influential ispublished, in some version, in the top peer-reviewed journals such as the American Political Science Review, American Journal of Political Science, International Organization, Comparative
Politics, and the Journal of Politics. Understanding the role and importance of different journals
in communicating knowledge is thus a crucial part of understanding how the field progresses and atask we undertake here.

As we explain below, we are hardly the first scholars to evaluate journals and attempt to quantify
their contributions to the discipline. Current techniques and measures typically utilize raw countsof citations which, we contend, are poor proxies for the true variables of interest. Else, they rely onpolitical scientists’ assessments of various journals which are, by definition, subjective and relatively
expensive to obtain. By contrast, in this paper, using commonly available citations data and theway that journals cross-reference each other, we show a way to systematically assess their contribution to intellectual exchange. As we show, this simple approach that uses information about the
total pattern of citations between journals, first used in statistics by Stigler (1994), greatly expands
our understanding of how the field communicates and advances.

2

Influence in the Discipline

Political scientists are evidently concerned with influence in the discipline. For instance, the November 2006 edition of the American Political Science Review was solely devoted to the evolution ofpolitical science and listed the twenty most cited articles in the history of the journal. While thesetwenty articles are undoubtedly very influential, a list of raw counts still does not tell us muchabout the exchange of knowledge throughout the discipline. And, certainly, a list of raw counts
such as the one published the APSR does not tell us which journal had the most influence on the1

Obviously, this statement does not apply to book manuscripts. We focus on peer-reviewed journals and leave theinfluence of books and book publishers to another study. For political scientists’ evaluations of various publishers,
see Goodson, Dillman and Hira (1999).

2

discipline in 2004 or 2005.2

Citation counts are used in a variety of ways to evaluate the quality of journals. Christenson
and Sigelman (1985) utilize citation counts in an early study of journal quality. Additionally, Journal Citation Reports , which is available from the ISI Web of Knowledge, calculates a variety ofmeasures such as the “impact factor”, “immediacy index”, as well as reporting the raw number ofcitations for each year. Both the “impact factor” as well as the “immediacy factor” measure howfrequently a journal’s “average” article is cited, with the former using “recent” citations (i.e., fromthe previous two years) and the latter using only citations from the current year. The common
problem with measures based upon raw citation counts like this is that a citation is treated as astandard ‘unit’ regardless of the parties involved. Hence, in terms of ‘impact’, there is no difference
between Journal A’s citation by a highly esteemed and well-read Journal B, and A’s citation by athird-rate Journal C that has a small and easily (intellectually) pleased audience. Yet most politicalscientists would consider B’s citation of A more prestigious. For similar reasons, problems may becaused by journals that self-cite a great deal.

Most work by political scientists that evaluates the relative quality of journals relies on rankings
provided by political scientists themselves through surveys. Studies by Giles and Wright (1975)
and Garand and Giles (1989) are examples. Noting that field journals with a narrow readership
obtained “surprisingly” high rankings in scholars’ subjective evaluations, Garand (1990) pioneered
an approach also used by Crewe and Norris (1991) and Garand and Giles (2003), that includes anadditional measure to the subjective quality ranking to account for the discipline-wide familiarityof the journal.3 To repeat, this method is nonetheless subjective. Moreover, they are relatively
expensive: surveys are typically time consuming to dissipate, fill and assess.

Thus the potential utility of a simple, cheap, systematic method that estimates the contribu2

See Stigler (1994) for a number of other problems specific to the examination of data on citations of individual
authors.
3
Their final impact score is calculated as follows:
Journal Impact=Journal Evaluation + (Journal
Evaluation×Journal Familiarity)3

tion of journals to the exchange of knowledge in the discipline. This would be a compliment tocurrent practices, and allow us to see whether, for example, political scientists’ subjective analysis
corresponds closely to the actual exchange of ideas among the top journals in the discipline.

3

Data

We utilize data published by Journal Citation Reports and available online from ISI Web of Knowledge from 2003–2005. The data includes, for 84 political science journals, how many total citations
each journal had in the given year, as well as a variety of measures based upon citation counts.
Key to our enterprize, the data records for a journal such as the American Political Science Review
how many times any other journal (such as the Journal of Politics or American Journal of Political
Science ) cited it in each given year4 as well as how many times articles published in the Journalof Politics or American Journal of Political Science were cited by the American Political Science
Review in each given year.5

In order to both keep the below analysis meaningful and tractable, we examine the pattern ofintellectual exchange among sixteen top journals from 2003–2005. We chose journals thatare believed to be among the field’s top journals by scholars. Thus, we chose the top
sixteen journals reported by Garand and Giles (2003).6 This provides us with a nice baseline forcomparison while also giving us a list of prestigious journals that are comparable forums for intellectual exchange.7 One of the more interesting findings of the recent survey of political scientists
by Garand and Giles is that three of the top nine general journals (American Economic Review ,American Sociological Review , and American Journal of Sociology ) are not even political sciencejournals. Garand and Giles (2003, 298–299) posit that these three journals fare so well because
4

The articles cited can be published in any year.
Any given article can only be cited once in a paper, regardless of how many times it is referred to in the text.
6
American Economic Review (AER),American Journal of Political Science (AJPS), American Journal of Sociology(AJS), American Political Science Review (APSR), American Sociological Review (ASR), British Journal of Political Science (BJPS), Journal of Politics (JOP), Political Research Quarterly (PRQ), Political Science and Politics(PS), International Studies Quarterly (ISQ), Public Opinion Quarterly (POQ), World Politics (WP), International
Organization (IO), Comparative Politics (CP), Comparative Political Studies (CPS), and Political Science Quarterly(PSQ): see also Table 1.
7
Our analysis would not be of much interest if we included a list of journals that do not exchange any citations.
54political scientists recognize them as “the flagships of their respective disciplines”, not because they
are familiar or contribute in a meaningful way to intellectual exchange in the discipline. In thefollowing section, we assess this claim as well as some widely held beliefs about journal quality withactual data on intellectual exchange.

4

Method: The Intellectual Exchange Model

In Table 1, we present the citation counts for the journals in our study broken down by the journalthat cited them for 2005. Clearly, there is much variation in these counts, and which journals
cite which journals, and our concern here is to summarize this data in a straightforward way that
captures the diffusion of ideas, information and influence across journals. As noted above, weconceive of each citation as an intellectual exchange between journals: when Journal B cites(an article in) Journal A, we say that A ‘influences’ B; when the opposite pattern occurs, we saythat B ‘influences’ A. One could quibble with the term ‘influences’ but we believe the central idea
is a sound one: if a journal makes reference to another it is clear that the journal being cited has
made an impact on the citer. The citer may be extending the work of an article that appearedin the cited journal, making reference for further reading or completeness, or even attempting towholly repudiate the cited findings. The nature of the case makes no difference to our argument;
as we explain, what does matter is who takes part in the intellectual exchanges and with whom. Of
course, the unit of analysis is the journal, rather than the article, and we assume that, in general,journals are influential because they publish influential articles.

To construct our measure, suppose that each journal has a ‘power to influence’ which, though
it exists and affects its relations with other journals, cannot be directly observed. In this sense,
it is no different to a citizen’s propensity to vote, or a nations ‘resolve’ in war crisis negotiations.
One way to measure this power is to use what we call the ‘Intellectual Exchange’ model. The idea
is to estimate a score for each journal i, denoted Ei which has the property that for any citation
involving two journals as featured in Table 1, the (log) odds that it is A influencing B, rather than5

Cited→
AER

AER
—

AJPS
13

AJS
3

APSR
38

ASR
0

BJPS
3

CP
0

CPS
0

IO
2

ISQ
0

JOP
2

POQ
0

PRQ
0

PS
0

PSQ
0

WP
0

AJPS

16

—

12

194

8

25

9

27

24

20

84

34

23

9

10

28

AJS

23

15

—

36

212

2

6

5

15

4

10

10

4

0

0

13

APSR

12

86

6

—

2

7

2

8

53

9

25

9

3

3

13

20

ASR

31

19

332

43

—

8

0

2

17

8

11

14

0

0

3

15

BJPS

6

84

8

115

8

—

4

21

16

12

49

14

5

0

5

12

CP

0

7

3

20

3

2

—

9

9

5

0

2

0

2

0

18

CPS

6

54

7

76

8

19

15

—

41

7

17

5

4

0

0

29

IO

19

32

10

66

9

8

10

23

—

40

15

0

2

2

3

42

ISQ

2

52

3

68

7

7

3

11

69

—

19

5

7

0

4

30

JOP

10

188

10

198

0

18

2

18

18

21

—

37

12

16

6

7

POQ

0

48

7

44

9

9

0

0

0

0

18

—

0

2

4

0

PRQ

0

121

4

139

4

13

0

0

0

5

92

16

—

4

9

2

PS

2

16

2

24

0

2

0

0

0

0

15

9

7

—

4

0

PSQ

0

7

0

11

0

2

0

0

7

0

4

0

0

2

—

9

WP
↑
Citing

2

9

0

29

2

0

5

8

25

6

3

0

0

0

0

—

Table 1: Cross-citations in political science for selected journals, 2005. Rows are the citing journal; columns are
the journal being cited : e.g. APSR is cited by AJPS 194 times in 2005.

6

B influencing A, is given by EA − EB . More formally:ln

Pr(A influencing B)= EA − EB
Pr(B influencing A)or, equivalently,
Pr(A influencing B) =exp(EA ).
exp(EA ) + exp(EB )The interpretation is very simple: the larger the relative value of Ei for each journal, the more influential it is. Once we have the scores, we can gauge literally how much more influential a journal
is than another by computing the probabilities above for any pair we wish. Since it may not be
obvious from these equations, to understand the strength of this approach, suppose as above thatjournal A is cited by both journal B and journal C. Suppose also that journal B is very well citeditself, but C is not. Then in our model, B’s citation of A will contribute more to A’s score thanC’s citation of A will contribute. Otherwise put: it is not simply the quantity of citations thatmatters, but the quality.

This approach is known as the Bradley-Terry (Bradley and Terry, 1952) model in statistics and has
seen numerous uses in that discipline— one of which is extremely close to the current application(Stigler, 1994). Such a model can be fit in any number of software packages and we chose R (RDevelopment Core Team, 2006) using the BradleyTerry library (Firth, 2005).

5

Findings

In Table 2 we report the influence scores for each of the journals in our study for the year 2005.
We include the ranking for each individual year in the sample as well as the overall
ranking across the three years. Notice that, because the power to influence is relative to otherjournals, we assign one of the publications as the ‘baseline’ or ‘reference’ score of zero. We choose
Political Science and Politics (PS) for this purpose in all models. Additionally, in the fifth
column we provide the rankings reported by Garand and Giles (2003) for comparison.

7

2003
Journal Score

2004
Journal Score

2005
Journal Score

Overall
Journal Score

Garand & Giles
Journal Rank

AER

3.61

AER

3.70

APSR

1.99

AER

2.75

APSR

1

APSR

2.68

AJS

2.75

WP

1.72

APSR

2.34

AJPS

2

WP

2.08

APSR

2.49

AER

1.66

WP

2.00

JOP

3

PSQ

2.04

WP

2.40

PSQ

1.27

IO

1.45

WP

4

AJS

197

ASR

2.38

AJPS

0.99

ASR

1.39

IO

5

POQ

1.73

IO

1.99

IO

0.88

AJS

1.35

BJPS

6

IO

1.64

AJPS

1.50

POQ

0.73

AJPS

1.34

ASR

7

AJPS

1.63

CP

1.19

CP

0.47

POQ

1.01

AER

8

CP

1.27

JOP

0.79

JOP

0.28

CP

0.97

CP

9

JOP

0.52

POQ

0.74

AJS

0.26

PSQ

0.91

AJS

10

ISQ

0.52

ISQ

0.54

ISQ

0.10

JOP

0.52

CPS

11

BJPS

0.43

CPS

0.34

CPS

0.05

ISQ

0.37

PS

12

CPS

0.28

BJPS

0.20

PS

0.00

CPS

0.23

PRQ

13

PS

0.00

PS

0.00

ASR

-0.18

BJPS

0.09

ISQ

14

PRQ

-0.93

PSQ

-0.17

BJPS

-0.29

PS

0.00

PSQ

15

—

—

PRQ

-0.93

PRQ

-0.99

PRQ

-0.99

POQ

16

Table 2: Rank order of scores for journals from the ‘intellectual exchange’ model: a higher score
implies a more influential journal. Columns, from left, refer to 2003, 2004, 2005 and overall (allyears aggregated). Final column on far right is rank of top 16 journals as presented by Garand &Giles (2003). Political Science and Politics is set to zero for the intellectual exchange model, andemboldened in table. American Sociological Review data not available for 2003.

8

From the overall results (including all years) in Table 2 we conclude that the American
Economic Review is the most influential journal (of the 16 in the sample), followedby the American Political Science Review and World Politics . Examination of theindividual years demonstrates that this pattern is generally stable, although the exact
rankings fluctuate slightly. The biggest surprise from the aggregate (and individualyear) results is the poor performance of the AJPS and JOP relative to the rankings
provided by Garand and Giles (2003). Although these two journals are regarded as twoof the top three by scholars in the discipline, the results in Table 2 indicate that journals such as World Politics and International Organization are more influential. The
influence score of Public Opinion Quarterly also consistently ranks higher than in thesurvey performed by Garand and Giles (2003), while the British Journal of Political
Science does considerably worse. Political Science Quarterly is also surprisingly influential, although its performance is inconsistent across the three years in the sample.

As noted above, the scores in Table 2 have a direct interpretation outside of the rank order.
Consider, for example the probability that any particular citation involving the American Political
Science Review and Political Research Quarterly has the APSR influencing PRQ. This is:Pr(APSR influences PRQ) =exp(2.34)= 0.965
exp(2.34) + exp(−0.99)

which is a near certainty. By contrast, the British Journal of Political Science and theComparative Political Studies are much more evenly matched in terms of influence.
Indeed the probability that an intellectual exchange between them has the BJPS
influencing the CPS, rather than the other way round, is:Pr(BJPS influences ASR) =exp(0.09)= 0.465
exp(0.09) + exp(0.23)

which is close to a coin toss.

9

Journal

Score

APSR

2.29

WP

2.02

IO

1.44

AJPS

1.32

POQ

0.97

CP

0.97

PSQ

0.86

JOP

0.47

ISQ

0.36

CPS

0.26

BJPS

0.05

PS

0.00

PRQ

-1.01

Table 3: Rank order of scores for political science journals from the ‘intellectual exchange’ model:
a higher score implies a more influential journal.
An objection here might be the inclusion of journals that are usually considered part of economics
or sociology and hence outside of the discipline per se.8 In response, we report our model finding
just for political science journals in Table 3, in which it is readily apparent that the rank order ofTable 2 is unperturbed by reducing the sample of journals.

A potentially more interesting issue concerns the effect of a journals’ origins and general audience
8

Notice that we assume authors in all disciplines (and all journals) behave similarly in only citing work that is
actually germane to their endeavor. From Table 1 we see no reason to suppose otherwise.

10

EstimateStd. Error

Economics

1.33

0.11∗∗∗

Sociology

0.21

0.07∗∗∗

Table 4: Effect of political science, sociology and economics focus on journal’s influence. Asterisked
coefficients imply p < 0.01 (***)
in terms of its influence. That is, we could suppose that a journal’s influence in political science
is in part determined by the discipline it serves. In Table 4 we re-estimate our model making thescores a (linear) function of the discipline that a journal predominately operates in. Hence, wedenote the American Economic Review as an economics journal, while the American Journal of
Sociology and American Sociological Review are from sociology.

The interpretation of coefficients in Table 4 is not unlike that for a (multinomial) logistic regression, relative to a base category of political science. In particular, economics journals
have an estimated advantage in influence of 1.33 over political science journals, while
the influence of sociology journals in political science is much smaller at 0.21. Bothcoefficients are statistically significant. This suggests that, for our sample years, for
our sample journals, economic titles are much more influential than sociology journals
in the discipline of political science, and, in fact, more influential than political sciencejournals broadly construed. This is quite a contrast to the argument put forth by
Garand and Giles (2003).

6

Discussion

As Riker (1982, 753) observed, “[political] science involves the accumulation of knowledge.” A partof this accumulation is the regular interchanges and diffusion of findings, ideas, innovations, models
and techniques. In this paper, we set out to measure this tendency as it applies to journals inthe discipline. Breaking away from simple unitary ‘counts’ of citations or expensive and subjective
11

surveys, we showed a way to compare journals based on their performance as exchangers of information. The method was straightforward to compute and yielded what we believe to be a valid andobjectively justifiable rank ordering with the American Political Science Review , the American
Economic Review , and World Politics as the top performing political science publications.
We went on to show that, though sociology journals are clearly part of the exchange process inpolitical science, they are not outperforming economics journals as a source of ideas or influence.
More tentatively, one might argue that this is evidence that the discipline increasingly adheres to the “Rochester School”, which emphasizes rational choice modelingof human political behavior in a style strongly reminiscent of economics (Amadae andBueno de Mesquita, 1999). Of course, the temporal domain of our study only includes
three years and we look forward to publishing updates to our findings as they occur.

References
Amadae, S. M. and Bruce Bueno de Mesquita. 1999. “The Rochester School: The Origins of
Positive Political Theory.” Annual Review of Political Science 2:269–295.
Bradley, R A and M E Terry. 1952. “Rank Analysis of Incomplete Block Designs I: The Methodof Paired Comparisons.” Biometrika 39:324–345.
Christenson, James and Lee Sigelman. 1985. “Accrediting Knowledge: Journal Stature and Citation
Impact in Social Science.” Social Science Quarterly 66:964–975.
Crewe, Ivor and Pippa Norris. 1991. “British and American Journal Evaluation: Divergence orConvergence?” PS: Political Science and Politics 24:524–531.
Firth, David. 2005. “Bradley-Terry Models in R.” Journal of Statistical Software 12.
Garand, James C. 1990. “An Alternative Interpretation of Recent Political Science Journal Evaluations.” PS: Political Science and Politics 23:448–451.
Garand, James C. and Micheal W. Giles. 1989. “Political Scientists’ Journal Evaluation Revisited.”
PS: Political Science and Politics 22:613–617.
12

Garand, James C. and Micheal W. Giles. 2003. “Journals in the Discipline: A Report on a New
Survey of American Political Scientists.” PS: Political Science and Politics 36:293–308.
Giles, Micheal W. and Gerald Wright. 1975. “Political Scientists’ Evaluations of Sixty-Three Journals.” PS: Political Science and Politics 8:254–257.
Goodson, Larry, Bradford Dillman and Anil Hira. 1999. “Ranking the Presses: Political Scientists
Evaluations of Publisher Quality.” PS: Political Science and Politics 32:257–262.
R Development Core Team. 2006. R: A Language and Environment for Statistical Computing.
Vienna, Austria: R Foundation for Statistical Computing. ISBN 3-900051-07-0.
URL: http://www.R-project.org
Riker, William H. 1982. “The Two-party System and Duverger’s Law: An Essay on the History of
Political Science.” American Political Science Review 76:753–766.
Stigler, S. 1994. “Citation Patterns in the Journals of Statistics and Probability.” Statistical Science
9:94–108.

13