Dimensions of Diplomacy
Understanding Private Information in International Relations Using
the WikiLeaks Cable Disclosure∗
Michael Gill†

Arthur Spirling‡

Abstract
We consider the role of private information in international relations—a key concept in‘rationalist’ models of interstate relations—from a unique empirical perspective. Noting that little systematic observational data exists regarding the contemporary private
information available to state actors, we analyze 163,958 United States diplomatic cables for the period 2005 to 2010—as disclosed by the Wikileaks organization—in orderto speak to several aspects of contemporary international relations theory. In this analysis, we show that diplomatic secrecy consists of two distinct ‘dimensions’: substantive
and procedural. The former deals with secrets per se as they relate to specific politicalissue areas, the publication of which would actively damage U.S. interests, especiallyin terms of revealing the resolve or capabilities of the state. Procedural secrecy, meanwhile, deals with the diplomatic norm of confidentiality in meetings—regardless of thesubstantive content of any single cable. We relate these two dimensions of diplomacy todifferent concepts of secrecy in the theoretical IR literature, and demonstrate that bothplay an important role in dictating the classification decisions of the U.S. State Department. In uncovering these substantive points, our paper presents novel methodologicaltools of general interest to scholars in the field.

∗

First version: February 1, 2014. This version: March 6, 2015. Erin Baggott, Amber Boydstun, Andy
Hall, Robert Schub, and Anne Sartori provided very helpful comments on an earlier draft. We thankaudiences at the Harvard Political Economy Workshop, MIT, Princeton University, the New Directions in
Text Analysis conference, and the Midwest Political Science Association meeting for feedback.
†Department of Government, Harvard University. mzgill@fas.harvard.edu‡
Department of Government, Harvard University. aspirling@gov.harvard.edu

11Introduction

State secrets and the private information possessed by leaders are nearly impossible to observe in practice. Yet such quantities—and the general difficulty of communicating that
information in the form of a state’s capabilities and resolve—are at the core of the gametheoretic approach to the study of conflict (Frieden and Lake, 2005; Lake, 2010). In thatparadigm, private information is a crucial ingredient of understanding why disputes may
arise (see, e.g., Powell, 1999, 2002; Tarar and Leventoglu, 2009) and escalate (e.g., Fearon,
1994a), how they might end (e.g., Goemens, 2000), their duration (e.g., Slantchev, 2004)
and whether they might be avoided in the first place (see, e.g., Fey and Ramsay, 2011).
This central place for information, secrecy, and beliefs also holds in theories of international
relations that do not invoke the tenants of bargaining directly, including those that rely on
ideational motivations (e.g., Wendt, 1999) or the pursuit of material resources and state
security (e.g., Waltz, 1979), more broadly. Perhaps unsurprisingly given the importance ofcommunication and information control, recent theoretical research has turned particularly
to the use and practice of diplomacy in the international system (see, e.g., Sartori, 2002;
Smith and Stam, 2004; Sartori, 2005; Kurizaki, 2007; Trager, 2010; Ramsay, 2011). Thereit joins a now well-established empirical literature that either explores the plausibility of thefundamental tenants of the rationalist approach (e.g., Fearon, 1994b; Partell and Palmer,
1999; Werner, 1999; Reed, 2003; Ramsay, 2008; Reiter, 2009; Potter and Baum, 2014), or
assuming that those assumptions are correct, gives methodological advice on how to fit statistical models consistent with them (see, e.g., Signorino, 1999).

Despite the obvious progress made on creating and examining models that do a better jobof describing the reality of crises as observed by international relations researchers, we still
understand relatively little about the dynamics of information transmission and protection2

in non-crisis situations—yet these surely constitute the great majority of all international
interactions. Accordingly, outside of some specific policy areas relating to conflict (on military operations, see, e.g., Keohane and Nye, 1977) we have a dearth of knowledge regarding
the more general diplomatic behavior of leaders and bureaucrats. For example, we know very
little, in an empirical sense, of how ‘military capabilities’ are actually conceived of by important agents, how such information is withheld or protected relative to other international
political issues, and how that conception affects what is promoted, concealed or communicated to foreign and domestic actors. This situation is unsurprising, but unsatisfactory. It is
unsurprising—given limited resources, including researcher time—the literature has focusedon crises and war, which have very clear welfare consequences. Assuming we did want tostudy more general interactive practices, data limitations are prohibitive in any case: secret
information is, by definition, closely guarded by states and even when their files are declassified and actors willing to give interviews, they do so in an obviously selective way (see,e.g,
Shapiro and Siegel, 2010, for discussion). Furthermore, were such data possible to obtain,
it is not in an obviously usable form. In particular, quantitative scholars have tended tohone their techniques for observational data in which each ‘row’ represents the incidenceof a particular phenomenon of interest (e.g., Ghosn, Palmer and Bremer, 2004), whereas
information pertaining to diplomacy is mostly in terms of documents (primarily cables) sent
between embassies and bureaucratic departments and ministries. In such a world, what
constitutes an individual observation is quite unclear.

This state of affairs is unsatisfactory for more obvious reasons: put very crudely, as a discipline, we do not how secrecy ‘works’ in contemporary diplomacy, despite the fact that it isa vital part of our theories. Our ignorance in international relations regarding such a corecomponent of our theories may be compared with a much more favorable situation in relatedareas of political science which have new and fine-grained data to assess the plausibility of3their models, and to expand their understanding of the processes therein. As an example,
consider the now voluminous literature on psychology in international relations (starting atleast with Jervis, 1976), in which scholars have used new measures of biological responses
to assess the effects of emotion on political opinions (e.g., Renshon, Lee and Tingley, Forthcoming).

This paper introduces new data and methods to get precisely at these fundamental issues
for United States foreign policy: that is, we characterize diplomacy in terms of what is
kept secret, and provide explanations as to why. Our data are the more than one hundred
thousand diplomatic cables released to the WikiLeaks organization, dealing with the period
between 2005 and 2010—an era in which coverage is relatively dense, and during which theUnited States had several ongoing military operations in the Middle East. The cables—
which are essentially secure emails sent between the Department of State and its embassies
and missions in hundreds of cities around the world—include large numbers of documents
never intended for public consumption. As such, they include thousands of (officially) secretand confidential missives, and thus allow us extremely rare access to the world of private
information in IR unfettered by state censorship. Our primary theoretical contribution is toargue that private information is as much about ‘procedure’ as it is about ‘substance’. Put
differently, though the United States certainly does not wish other state actors to discovercertain facts about its material capabilities (especially regarding matters of ‘high politics’ inthe sense of Keohane and Nye, 1977), it also wishes to obfuscate the way that it allocatesintelligence gathering resources. That is, the United States seeks to hide the information
it gathers, from whom it garners it, and what it chooses to disseminate. We call this latter dimension of diplomacy ‘procedural’ and contend that it refers not to actions regarding
specific objects (such as arms, services, plans or strategies) but rather to a method of behavior in general, regardless of political issue areas. In keeping with a pre-existing literature4

on diplomacy (particularly Sartori, 2002), we provide corroborative evidence for the notionthat developing and maintaining a reputation for confidentiality matters to diplomats andtheir staffs. We go beyond current accounts, however, in demonstrating that patterns of
information protection hold somewhat independently of the policy areas discussed.

While the purpose of this paper is not to ‘test the assumptions’ of bargaining models per se,
our hope is that our inspection will aid researchers interested in the empirical implicationsof such theoretical models. In particular, studying the way that information is shielded fromglobal public view on an everyday basis provides a resource for those interested in ‘audiencecosts’ (e.g., Fearon, 1994a; Weeks, 2008)—which apply to crises bargaining situations, specifically. Furthermore, since we show that both procedure and substance matters for secrecy,
we believe our empirical efforts provide an impetus for theory development in contemporaryinternational relations research. This is in part because our sample is much broader in substantive terms than has been available for previous studies, for which we have theoreticalwork already. But it is also partly because the cables we use have ‘official’ designations interms of their classification status and the justification for that status. That is, our work
relies on the officially structured indexing of cables and their topical designations, for whichstate actors have made conscious decisions. Aside from data cleaning, we do relatively little
to restructure the standardized form of the metadata observed on each diplomatic cable, assuch a procedure could induce error in the variables we chose to measure. As a result, ourinferences are less dependent on somewhat idiosyncratic or arbitrary research rules, and thus
sharper than previously possible; we believe this makes them especially ripe for theorizing(for further testing thereafter), as our topical policy codings are identical to the internal
indexing structure used for information management at the State Department.

In undertaking the study, our paper contributes methodologically and suggests new ways
5

of working with ‘texts-as-data’ (Grimmer and Stewart, 2013). In particular, we use machinelearning techniques—such as random forests (Breiman, 2001) and the ‘lasso’ (Tibshirani,1994)—in tandem with matched sampling designs to identify how ‘important’ terms discriminate between restricted and unrestricted documents. We provide novel ways of comparing
texts, based on matching on the metadata of each document, such that political scientists
may think sensibly about the (marginal) influence of secrecy on a document’s content.

This paper proceeds as follows. In the next section we describe some literature, and orient the reader to the debates at hand. Then, we introduce the data and provide somesummaries of it. We then discuss the methodological contributions. First, we consider an
uncovering of the substantive dimension of diplomacy. This boils down to a regression-based
approach with internal State Department subject ‘tags’ as independent variables predicting the classification status of a document (with multi-way fixed effects between location oforigin and time). This is followed by an investigation of procedural secrecy, which sees us
exactly match on official State Department subject tags, and using popular machine learning techniques to estimate the difference between restricted and unrestricted access cables(i.e., estimating the textual differences between more restricted and less restricted cables,
on a sample of cables that are comparable given their meta data). We then provide somediscussion of our results and conclude.

2

Theories of Private Information Disclosure

To get much of their purchase on the world, the analytic conclusions drawn from rationalist
studies of international dynamics often hinge on differences between private and public information, and the degree of information overlap or ‘common knowledge’ shared by actors in6

a non-cooperative environment (see Powell, 2002, for an overview). Somewhat surprisingly,
researchers modeling in this tradition rarely define exactly what constitutes information ofeither type. If they do, they do not study the difference in detail. Primarily this is a data
issue: understanding what states know, what they do not, and what they are keeping secretcannot be determined deductively insofar as official refusal to answer queries about particular issues does not allow scholars to become much more informed about the true state of
the world. While it is correct that documents pertaining to international decision making
are routinely declassified, this process tends to be slow, somewhat haphazard, and obviouslycase selective (see Allen and Connelly, 2015, for an overview of U.S. protocols).

Given the obvious data limitations for recent historical periods, scholars interested in examining the plausibility of theories that rely on information—for example, those that utilize‘audience costs’—tend to pursue one of three avenues. First, they use conflict (or survey)
observational data in a regression context (e.g., Weeks, 2008). This has the obvious benefit
of being straightforwardly replicable, but arguably lacks the kind of internal validity thatwould be convincing for skeptics (see, e.g., Trachtenberg, 2012, for recent discussion). Second, they enter the archives of governments and produce historically rich case studies (e.g.,
Schultz, 2001; Snyder and Borghard, 2011), which are necessarily limited to specific times
and places. Third, they undertake field or survey experiments on political actors or on lessrepresentative samples (e.g., Tomz, 2007). In all cases, it is obvious that if researchers had
broader access to cases or incidences of censored versus public materials, they might be able
to draw sharper conclusions—better in terms of both internal and external validity—thancurrent approaches allow.

Whatever the research approach, a key insight of the audience cost literature is that increasing the publicity of some political matters may be useful, particularly if the issue at7

hand is one of signaling a commitment credibly—and we might expect variation in a nation’s willingness to publicize issues given the strategic importance of the issues at hand(e.g., Fearon, 1994a). Of course, strategically making information public is helpful beyond
the signaling case: citizens sometimes need to be warned to stay away from certain areas,just as investors may be encouraged by the public announcement of tax breaks or shifting
interest rates. Along these lines it is not hard to think of ‘substantive’ facts, especiallyconnected to capabilities, that states may want to conceal—for example, where they locate
their nuclear warheads, or how many they have and how easy they are to launch. It would
directly hurt a state were any of this information known, since it compromises defense plans,decreases combat effectiveness, and broadly provides potential enemies with additional bargaining power.

Inasmuch as nations have incentives to disclose some information publicly while keeping
other information private, one can imagine scenarios in which governments prefer to conceal
what they are ‘trying to know’ or how they acquire and share information independent of
any substantive issue at hand. In brief, there may be incentives to practice discretion, on
average, independent of a particular policy issue being discussed diplomatically. Suppose, forexample, the United States had a rule such that it publicized its diplomatic communications
on some policy areas but not on others; competing nations (or public actors) could reasonably update their beliefs on what is likely being negotiated or communicated in private given
the absence of that policy in being revealed in public summaries. In such a world, one cansee how a practice of selective information disclosure could in fact advantage international
competitors relative to a more general norm of secrecy and discretion. In an extension of thislogic, nations may have incentives to protect or conceal information concerning the meetingsof leaders or public officials irrespective of the topics or issues discussed. Taking this further,
suppose it was common knowledge that a group of international leaders had scheduled a8

meeting on a certain day to discuss an unknown international issue; if an institutional rule
existed such that the topic of the meeting would be made public if and only if it concerneda certain set of political issue areas (e.g., the topic of the meeting is made public if about
environmental politics or maritime disputes, but kept private if about military capabilities,
terrorism, or territorial disputes), then the absence of a public summary of the meeting
would nevertheless reveal some information to the public about the likely issues raised inthat private communication—a fact that could make the institutional rule of selective disclosure, at least on the margins, strategically self-defeating. It would be more difficult forthe public to update its beliefs on the likely content of a meeting if no such (disclosure) rule
were to exist, by contrast.

Although the discussion above concerns the practice of disclosure in international diplomacy, the intuition that selective (i.e., topic-specific) information disclosure may be welfare
reducing is at the core of a much broader class of political, bureaucratic, and economic phenomena. There is a vast microeconomic literature on the public dissemination of private
information in macroeconomic environments with strategic complementarities—an area of
economic research which tends to focus on the conditions under which information disclosure
may be welfare improving or influence a system’s volatility. Common settings for such formal reasoning are studies of central bank communication and global financial systems, where
decision makers may wish on the one hand to disseminate some information to public audiences but avoid being so transparent as to induce financial crises or speculative attacks on
the other. In the determination of interest rates and deliberation of monetary policy, central
bankers face decision problems that closely resemble problems faced by international diplomats, although this bureaucratic comparison is not specifically discussed in this literature.
When making a public statement, social planners may be concerned both about the precisionof a public signal (i.e., how closely a message will map to real world outcomes) in addition9

to the degree of its publicity (i.e., how many individuals observe a signal). Cornand andHeinemann (2008) provide a careful discussion as to how precision and publicity may both
independently and jointly shape the decision to disclose private information. The authors
argue, in a refrain largely in step with much of the scholarly theories related to this problem,“The optimal degree of publicity depends on the precision of announcements” (718)—more
specifically, that if the precision of a public signal is not guaranteed to be sufficiently high(e.g., if the event or issue described in a communication is not sufficiently likely to occur
in reality), it may be dominant for bankers to avoid private information dissemination to
the public sphere altogether. Arguments more favorable to the prospect of transparency’s
welfare improvement can be found in Cukierman (2001) and Angeletos and Pavan (2004),
under the requirement that the quality of a public signals is sufficiently high. The model
presented in Woodford (2005) provides a sufficient condition for when increased transparencyof central banking should lead to a welfare reduction in expectation.

2.1

Implications of Motivating Theories

In international relations, meetings and communications between diplomats and state officials may be in service of multiple ends. Diplomats may wish to signal their intentions
about U.S. foreign policy to other leaders, share sensitive information freely and openly totrusted confederates, and—in perhaps an ideal case—update the quality of their private information about political issues that might otherwise be difficult to observe through othermeans. With that said, there is no perfect way to observe all private information available
to diplomatic actors by simply observing records of their communications and their associated handling statuses. Even if our data contained records of all top-secret communications
between relevant officials, our sample would still be unable to sensibly measure all officials’
private information per se.
10

What can be approximated, however, is the degree to which particular policy issues andlanguage-based features map to higher levels of political protection within our sample. Common sense dictates that the relationship between political issues and handling status ought
to be strongly correlated with political actors’ preferences over the sensitivity of a politicalissue area, or more broadly, the incentives to shift information disclosure to the public on a
specific political issue. If we are to observe that some political issues are systemically morepredictive of document secrecy than others, a fortiori this provides evidence of diplomatic
preferences over the sensitivity of political issues in the international system.

Formal theories of strategic information disclosure suggest diplomats would have incentives
to withhold information as a function of the presumed cooperativeness of a decision environment. All else equal, as incentives between nations to coordinate on policy issues decrease(e.g., on national security discussions or information on nations’ relative capabilities), wewould expect communications on such issues to be more protected in our sample on average than policy areas with more of a cooperative or ‘common-pool’ character (such asenvironmental concerns). In terms of their rank ordering, a natural prediction concerningthe ‘substantive’ secrecy of our sample would suggest a monotonic decrease in the in-sample
estimates of cable classification as one moves from traditionally ‘non-cooperative’ games on
one end to ‘cooperative’ issue areas on the other.

To assess the plausibility procedural dynamics as a driver of information restriction, ouraim is to extract features of language that are predictable of cable secrecy after adjustingfor the topical focus of diplomatic communications. If formal theories on information dissemination focus on minimizing enemies’ abilities to anticipate foreign decisions, one might
expect particular textual features of diplomatic cables to be predictive of secrecy conditional11

on subject matter. In particular, if meetings between diplomats, ambassadors, or leading
political officials serve an information-gathering purpose, one might expect references tosuch individuals, all else equal, to be positively associated with a document’s probability ofrestriction.

3

Data

The WikiLeaks cables disclosed in the Manning leak of 2010 are the 251,237 diplomatic secure messages sent by the U.S. State Department to U.S. embassies and missions. The date
range for the original data is from 1966 to 2010, and in Figure 1 we plot the total numberof cables per month from that time period. In our work here, we focus on all cables written
and sent between January 1, 2005 and the end of the data, the first few months of 2010. We
do this for two reasons: first, because as the figure suggests coverage prior to the year 2000
is somewhat sparse and inconsistent. Second, because we were concerned about changes tosecurity procedure (particularly regarding requirements to copy in embassies and missions
on particular types of messages) after the terrorist attacks of September 11, 2001. All told,
we are left with around 163,958 documents from which to draw inferences. Additional details on the likely representativeness of this sample relative to all communications between
embassies over this period can be found in Gill and Spirling (Forthcoming).

Technically speaking, cables may be classified into one of three categories, depending on the
degree of damage to national security that “the unauthorized disclosure of which reasonably
could be expected to cause.”1 Furthermore, any classified document must pertain to at least
one of a series of topics which inter alia include military plans, intelligence, foreign relationsof the United States, nuclear programs, weapons of mass destruction and vulnerabilities in
1

As described in Executive Order 13526, 2009.

12

Diplomatic Cables per Month in Sample
6000
●
●
●

5000

●
●●
●
● ●
●
●
●● ●
● ●●
●●●●
●
●● ●●
●
●
●●
●
● ●●● ●
● ● ● ●
● ●●● ● ●●●●

Total Cables

4000

3000
●
●
●
●
●●
●●
●●
●
●

2000

1000

●

●
●● ● ●
●

● ●

●●

●
●
●

1966
1967
1968
1969
1970
1971
1972
1973
1974
1975
1976
1977
1978
1979
1980
1981
1982
1983
1984
1985
1986
1987
1988
1989
1990
1991
1992
1993
1994
1995
1996
1997
1998
1999
2000
2001
2002
2003
2004
2005
2006
2007
2008
2009
2010

0

●
●
● ●
●●
●●
●
●
●
●
●●● ●
●● ●
●●●●
●
●
●
●●
●
●
●●●
●
●●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●

Time

Figure 1: Number of cables per month, 1966–2010. Note that post-2001 period has much
more dense coverage.
national security. In descending order of the purported balefulness of unauthorized release,
these categories are ‘Top Secret’, ‘Secret’ and ‘Confidential’. If a cable does not meet thecriteria for such restricted access, it is deemed ‘Unclassified’. In our particular data, we havethe following distribution: zero Top Secret, 10,195 Secret cables, 87,270 Confidential cables,and 66,493 Unclassified. There are, in addition, some extra classifications that appear lessfrequently in the data, such as ‘Confidential and Not For Foreign Distribution’ , ‘Unclassified for official use only’ , and ‘Secret and Not for Foreign Distribution’; we ignore thesecategories for our current efforts.

For our purposes below, we divide the categories into ‘restricted’ (R), which includes Secretand Confidential communications, and ‘unrestricted’ (U), which includes the unclassified
documents only. The central idea here is to separate documents into more ‘private’ andmore ‘public’ information, respectively. This measure is somewhat crude, but given thattheories in International Relations use similarly binary demarcations we think this is reason13

able. To be clear, the fact that a cable is unrestricted does not mean that it is automaticallymade public: it is still a government document rather than a press release. But unclassified
documents—so long as they are not ‘For Official Use Only’—do make their way into the
public domain, and are eligible for release under Freedom of Information Act requests. Putotherwise, our unrestricted case covers documents that the public (anyone without specificsecurity clearances) could access; our restricted cables are those that are not released orreleasable to the public.

Any given document has a series of subject matter ‘tags’ assigned to it by its authors,with guidelines for this process provided by the State Department.2 From our perspective,these tags contribute meta-data that communicates the topic of the content therein, and areassigned to each document following its completion. After cables are written and subject
tags are assigned, each cable is given its overall classification status.3 Examples of subject
tags in our data include ‘ADCO’ which refers to ‘Diplomatic Courier Operations’, ‘PTER’ whichrefers to ‘Terrorists and Terrorism’, ‘SMIG’ which pertains to ‘Migration’ and so on. There atotal of 97 tags in our data, though their use varies widely in relative frequency terms. The
full list can be seen in Appendix C. The variety in tag number per document can be seen in
Figure 2; inspection suggests that the modal number of tags is two or three, though there
are 14,451 unique combinations of subject tags (ignoring each cable’s location of origin) thatappear in our post-2005 sample at least once.

In Figure 3 we report the structure of the data in terms of the way that tags co-occur across
2

These are literally geopolitical ‘TAGS’, an acronym for ‘Traffic Analysis by Geography and Subject’,implemented for diplomatic communication in its modern form by an executive order (number 11652) inJune 1974. Their justification was to “[p]ermit more rapid and discriminating distribution of messages”,
and to “[p]rovide statistics to both offices and posts on what is being communicated in the Department-fieldsystem”; they were to “[s]erve as headings for clustering the terms used by professional indexers to identify
the content of substantive messages.”3
Additional details on the origins and formal procedures of cable creation can be found in Appendix A.

14

Number of Subject Tags per Cable
50000
Full Sample
Matched Sample

Frequency

40000

30000

20000

10000

0
0

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

# Tags on Cable

Figure 2: Distribution of tag numbers by document, matched and unmatched sample (defined
below)
cables. Areas of darkness in that plot are places where tags coincide. Our main observationis that tags in section ‘P’ (which denote ‘Political’ issues) and, to a lesser extent, tags insection ‘E’ (denoting ‘Economic’ matters) tend to coexist heavily with other subject indicators, suggesting that these issues play an important organizing role in the U.S. diplomatic
service. Machine readable versions of the documents themselves are available at various
websites for download, though some pre-processing is then required prior to any analysis.
In particular, the tag information must be captured and removed, and some other cleaning
performed. Much of what follows involves operations on the ‘document-term matrix’ (DTM)of the texts, which was ‘stemmed’ (meaning that words were pruned back to their ‘roots’
where possible, using the Porter (1980) algorithm), ‘stopped’ (meaning that function words
which are thought to contain little discriminating information were removed), and subject
to a ‘sparsity’ condition of 99-percent (i.e., only words that occur in more than one-percent
of all documents but in no more than 99-percent of all documents are included). Such data
cleaning is common in text analysis in the social sciences (e.g., Grimmer and Stewart, 2013).
The resulting DTM for analysis is matrix with dimensions 163, 958 × 3, 755.
15

Conditional Probability of Subject TAG Co−occurrence

A

B
C

E

M

O

P

S

Automated Data Processing
Buildings and Grounds
Budget Services and Financial Systems
Communication Operations and Administration
COMSEC Key Management
Diplomatic Courier Operations
Diplomatic Pouch and Mail
Emergency Planning and Evacuation
Financial Management
Foreign Service InstituteForeign Service National Personnel
General Accounting OfficeInformation Management Services
INR Program Administration
Internet Administration
Allowances
Medical Services
Management Operations
Telecommunications Equipment MaintenanceCommunications, Circuits, and Networks
Employees Abroad
Office Management Specialist Issues
International Organizations and Conferences
Personal Computers
Personnel
U.S. Sponsored Schools
Security
Inspector General Activities
Business Services Reporting
Trade Expansion and Promotion
FCS Management Operations
Trade and Investment Opportunities
Assistance to Citizens
Federal Agency Services
Judicial Assistance and Notarial Services
Visa Lookout
Consular Administration and Management
Passport and Citizenship
Visas
Agriculture and Forestry
Foreign Economic Assistance
Civil Aviation
Economic Conditions
Communications and Postal Systems
Financial and Monetary Affairs
Commercial Fishing and Fish Processing
Industry and Manufacturing
Economic and Commercial Internet
Foreign Investments
Labor Sector Affairs
Land Transportation
Minerals and Metals
Energy and PowerPetroleum and Natural Gas
Foreign Trade
Trade and Technology Controls
Waterborne Transportation
Military and Defense Arrangements
Military Assistance and Sales
Military Capabilities
Military Nuclear Applications
Military Operations
U.S. Diplomatic Representation
Educational and Cultural Exchange Operations
Foreign Diplomats and Foreign Missions
International Information Programs
Diplomatic Correspondence
Public Relations and Correspondence
U.S. Congressional Travel
Science Grants
TravelVisits and Travel of Prominent Individuals and Leaders
Arms Controls and Disarmament
National Boundaries, Territories, and Sovereignty
Internal Governmental Affairs
High Seas Affairs
Human Rights
Intelligence
National Security
National Independence
Refugees
External Political Relations
Propaganda and Psychological Operations
Terrorists and Terrorism
Cultural Affairs
Environmental Affairs
Migration
Narcotics
Social Conditions
Biological and Medical Science
Internet Technology
Engineering Research and Development
Physical Sciences
Energy Technology
Space Activities
Science and Technology Policy
AADP
ABLD
ABUD
ACKM
ACOA
ADCO
ADPM
AEMR
AFIN
AFSI
AFSN
AGAO
AINF
AINR
AINT
ALOW
AMED
AMGT
AMTC
ANET
AODE
AOMS
AORC
APCS
APER
ASCH
ASEC
ASIG
BBSR
BEXP
BMGT
BTIO
CASC
CFED
CJAN
CLOK
CMGT
CPAS
CVIS
EAGR
EAID
EAIR
ECON
ECPS
EFIN
EFIS
EIND
EINT
EINV
ELAB
ELTN
EMIN
ENRG
EPET
ETRD
ETTC
EWWT
MARR
MASS
MCAP
MNUC
MOPS
ODIP
OEXC
OFDP
OIIP
OPDC
OPRC
OREP
OSCI
OTRA
OVIP
PARM
PBTS
PGOV
PHSA
PHUM
PINR
PINS
PNAT
PREF
PREL
PROP
PTER
SCUL
SENV
SMIG
SNAR
SOCI
TBIO
TINT
TNGD
TPHY
TRGY
TSPA
TSPL

T

AADP
ABLD
ABUD
ACKM
ACOA
ADCO
ADPM
AEMR
AFIN
AFSI
AFSN
AGAO
AINF
AINR
AINT
ALOW
AMED
AMGT
AMTC
ANET
AODE
AOMS
AORC
APCS
APER
ASCH
ASEC
ASIG
BBSR
BEXP
BMGT
BTIO
CASC
CFED
CJAN
CLOK
CMGT
CPAS
CVIS
EAGR
EAID
EAIR
ECON
ECPS
EFIN
EFIS
EIND
EINT
EINV
ELAB
ELTN
EMIN
ENRG
EPET
ETRD
ETTC
EWWT
MARR
MASS
MCAP
MNUC
MOPS
ODIP
OEXC
OFDP
OIIP
OPDC
OPRC
OREP
OSCI
OTRA
OVIP
PARM
PBTS
PGOV
PHSA
PHUM
PINR
PINS
PNAT
PREF
PREL
PROP
PTER
SCUL
SENV
SMIG
SNAR
SOCI
TBIO
TINT
TNGD
TPHY
TRGY
TSPA
TSPL

A

B

C

E

M

O

P

S

T

Figure 3: This figure shows the empirical conditional probability of U.S. State Department
subject TAG co-occurrences in the post-2005 sample (n = 163, 958). Subject tags are arepresented in alphabetical order with their official U.S. State Department meanings listed in
the righthand column. Each cell in the figure represents the conditional probability thata column subject will be tagged given that a row subject has already been tagged. Darker
shaded cells indicate a higher conditional probability.

16

4

Methods

Our claim above is that the secrecy endemic to diplomacy comes in at least two separable
varieties: substantive secrecy—the notion that certain information about a policy area is
to be kept confidential because it would be per se damaging to security were it released—
and procedural secrecy, which is concerned with the notion that secrecy protects foreign ordomestic agents from outside consequences of their actions. To assess the evidence for theseseparate ideas, some care is required in terms of methods. Here we explain our approaches.

4.1

Substantive Secrecy

We first examine the question of substantive secrecy—i.e., how a State Department topic
or substance of a diplomatic communication, all else equal, influences its probability ofrestriction. The objective is to quantify both the magnitude and direction of how the presenceof official U.S. State Department communication subject tags influence cable secrecy. In suit,
we regress each cable’s observed restriction status on its subject tags and location of origin.
This fixed-effects least squares equation can be written as follows:

Ri = α +X

βt T agit + γj + εij(1)where Ri is a dummy variable for cable i that takes the value of 1 if the cable is restricted
and 0 if unrestricted, T agit is a subject tag dummy variable for cable i for each tag t, γj isthe fixed effect for embassy j, while α and εij are the constant and error terms, respectively.
Given that each covariate in this regression is binary, each regression coefficient β̂t is a sample
estimate of the difference between two conditional expectations: the conditional probability
a document will be restricted given the presence of a subject tag minus the conditional

17

probability of restriction without that subject tag present.4

4.2

Procedural Secrecy

Recall that procedural secrecy concerns the diplomatic norms of confidentiality in meetings.
If it exists as a quantity that can be identified in our data, then it should emerge as a key discriminator between restricted and unrestricted cables. However, if there is indeed a subject
tag imbalance between restricted and unrestricted cables (as suggested above), this implies
that a simple comparison of word frequencies between restricted and unrestricted documents
is unlikely to isolate how text varies on the margins a function of secrecy status alone, sinceobserved differences are likely to arise directly from ex ante differences in subject matter.

Thus, the question we ask in this section is: having adjusted for cable subject matter (given
an observed sequence of subject tags on a document) and locations of origin, all else equal,
can restricted diplomatic communications be distinguished from unrestricted communications? This question may be thought of as estimating the marginal effect of secrecy on
the content of a restricted communication. More precisely, given two documents indexedwith identical subject tags and originating from the same source, are there specific textual
features that systematically distinguish restricted cables from unrestricted cables? If suchtextual features exist, is there anything substantively unifying about these features? In
particular, does whatever differentiates these communications be considered ‘procedural’ innature?
4

That is, β̂t = P r(Ri = \1|T agt = 1, X) − P r(Ri = \1|T agt = 0, X). If standard assumptions hold—e.g.,
unconfoundedness, overlap, or “selection on observables” (Rosenbaum and Rubin, 1983; Heckman and Robb,
1985)—β̂t tells us on average how much document restriction varies by each subject tag in our sample.
Although the outcome of interest is binary, OLS is appropriate when the conditional expectation function(CEF) of each regressor with respect to the outcome exhibits is linear (see, e.g., Angrist and Pischke, 2009,
Chapter 3). OLS suitably estimates whereby sample average effect of restriction on each subject tag in thecontext of our data, as each regression coefficient represents a conditional mean.

18

4.2.1

Exact Matching on Subject Tags and Origin

To assess whether secrecy, on the margin, is associated with differences in document composition, we restrict our sample to exactly matched subsets of cables within each embassy in
our sample. More precisely, for each embassy (i.e., each cable’s location of origin), we implement the algorithm outlined in Figure 11 in Appendix F to construct datasets of cable pairsthat are exactly matched on official U.S. State Department subject tags and their embassies
of origin, but differ on their restriction level. The objective of this matching procedure isto restrict the full sample such that there is perfect subject overlap on cables in our study.
As a result of the matching procedure, within each embassy, for each restricted cable there
will exist an unrestricted cable that has an identical subject tag pattern. We rely only on
the State Department’s official subject tags for this procedure. If two or more unrestricted
matches are found for a single restricted cable, we select the match that is written mostclosely in time to the restricted cable’s date of authorship. For the results presented in thisstudy, matching is performed without replacement, and datasets are stored and analyzed at
the embassy level (although pooled analyses are also appropriate with the resulting data).

Since we wish to make inferences about textual differences between restricted and unrestricted cables on the margin—i.e., once cable subject tags have been accounted for—the
within-embassy matched sampling design has intuitive appeal. The sampling design allows
for a meaningful examination of procedural secrecy. Adjusting the sample directly for differences in subject matter and controlling for embassy-level effects, the design allows us toisolate differences in textual composition that are likely to arise from a document’s handlingstatus alone. Intuitively, the aim of our exactly-matched sampling design is to “control” forsubstantive differences in cables that may be present in the unmatched sample—differencesthat may arise from hypothetical variation in reporting rules, document disclosure standards,
authorship style, or political priorities at the embassy level. If systematic textual differences19remain between restricted and unrestricted cables after subject and location have been accounted for, these differences are likely to arise from procedural rules that are separate fromsubject-specific handling rules.

The formal appeal of exact matching is that it is nonparametric and approximates the actof “blocking” in randomized experiments (Cox, 1958; Imai, King and Stuart, 2008).5 Exact
matching is often untenable in applied research, however, since in many cases the sampling
procedure can dramatically reduce a researcher’s final sample size, and the procedure tends
to rely on initially large sample sizes. Unsurprisingly, this was a concern for our modelingattempts, along with the possible danger that many documents dealing with sensitive substantive areas would be jettisoned from the final analysis because no match could be found
for them. Further, we were concerned that certain ‘important’ embassies would be, relative
to the original dataset, heavily under-represented.

Neither of these concerns appear to be true of the matched sample. In Figure 4 we report the reduction in subject tag imbalance of the exactly matched sample, in addition to
information on which subject tags remain present. In the exactly-matched sample, we seeboth embassy-level and aggregate level subject imbalances have been eliminated. The upperright plot shows the within-embassy subject proportions are identical between unrestricted
5
In the context of parametric adjustment, Rosenbaum and Rubin (1983, 1984) show that matching ona correctly specified propensity score (i.e., a unit’s conditional probability of being assuaged to treatment,
given its covariates) is sufficient to allow for the unbiased of the average effect of a treatment for a given
population of interest, i.e., eliminate confounding. But in settings with observational data, a researcher rarely
knows whether an appropriate functional relation has been specified in model-based matching procedures(Rosenbaum, 2002). The appeal of an exact covariate matching procedure is that if the appropriate set of
conditioning measures has been identified, the unobserved functional relation between between covariates
and the assignment to treatment is ignorable due to perfect balance on conditioning variables. Under generalconditions, exact matching procedures are both equal percent bias reducing (Rubin, 1976) and monotone
imbalance bounding (Iacus, King and Porro, 2011). These traits are not generally true for most distancebased or model-based (parametric adjustment) matching methods, which has led several scholars to conclude
that exact matching is close to an “ideal” matching procedure in observational settings (e.g., Stuart, 2010;
Imai, King and Stuart, 2008).

20and restricted cables, which is why the colored lines appear purple (due to perfect overlap
between the red and blue lines). The exact subject balance is also true for the exactly
matched sample average. The lower-right plot demonstrates this point in an extreme form:
there exist no embassy-level or aggregate level subject imbalances in the exactly-matchedsample. The analysis gives us confidence that inferences following from the matched sample will be appropriate to a broad class of diplomatic communications. The distribution ofsubject tags in the matched sample map to substantively meaningful political issue areas.
The majority of cables in the matched samples have to do with Economic Affairs, Militaryand Defense Affairs, and Political Affairs—each topic within foreign policy that are closely
related to formal theories of rational diplomacy. On the other hand, the exact-sampling
design is less capable of making credible counterfactual statements about Administrative
Affairs and Outreach.6

In Figure 5 we report the embassies, and their relative prevalence, in our matched data. Importantly, we note that ‘larger’ embassies—including the U.S. State Department itself—aremost represented; in particular, Ankara, Baghdad, Paris, Cairo and Moscow (all centers ofactivity in the original data) appear at higher rates in the matched sample. Taken alongside
the results of Figure 4, this presents strong evidence that the matched sampling proceduredoes not leave the general patterns of the whole sample too far behind, and is due to thefact that there are sufficiently high within-embassy subject tag correlations. The diplomatic
locations contained in the study sample are represented in a manner proportionate to their
6

The reason subject tag overlap is important is because exact matching will allow us to inspect textual
differences akin to treatment effects on the treated. Treatment effects on the treated are not the same as theaverage effect of treatment unconditionally, nor are they the average effect of treatment in the sample. Moreprecisely, they concern how much potential outcomes would differ for a set of treated units in the sample ifthey were instead to become untreated. In the present study, therefore, with our exactly-matched sample,
the design allows us to estimate answers questions like the following: If a set of treated restricted documents
like those in our sample were instead to become unrestricted, on what textual dimensions would we expect
those collections of documents to vary?

21

overall representativeness in the full sample.

4.2.2

Supervised Learning and Penalized Regression

For each of the matched samples described in Section 4.2.1, we implement a set of supervised learning models to identify which words are most important to (i.e., predictive of)cable secrecy. The matrix of words used in this classification setting is taken from the fullpost-2005 document-term matrix described before, but now only includes rows that satisfy
the within-embassy, exactly-matched sampling design. On the ‘left hand side’ we have the(binary) restriction status of a given document which we intend to predict with the words
within that document. Quantitatively, we observe how within-sample classification error
rates vary as a function of which words are included in the model; qualitatively, we wish to
make statements about how a document’s restriction status would likely change if particular
words within these documents were to vary. Two supervised learning methods are applied tothese data: the “random forest” (hereafter RF) algorithm (Breiman, 2001), and the “lasso”(Tibshirani, 1994). Results from both procedures are used alongside the topic model estimates described below to make statements both at the world-level and topic-level about how
secrecy, on the margin, influences the content of diplomatic communications. More details
on the RF and lasso procedures can be found in Appendix B.

With both RF and the lasso, we obtain embassy-level estimates of word-level dependencies todocument restriction. In the context of RF, each exactly-matched dataset for embassy j has
a corresponding vector of word importances, where importance is defined as an estimate ofeach variable’s in-sample average marginal error reduction. In the context of the lasso, eachembassy has a corresponding vector of penalized partial regression coefficients. For both the
RF and the lasso procedures, we refer to this collection of embassy importance vectors as theembassy importance matrix. Each row in this matrix represents a given embassy, and each
22

Figure 4: In the upper-left subplot, the thin-transparent lines in red (restricted) and blue
(unrestricted) correspond to embassy-level averages of individual tag frequencies in unrestricted and restricted cables. The thicker vertical colored lines in the foreground denotesample averages. The lower-left plot provides much of the same information but in slightly
different form: background lines correspond to embassy-level imbalances (subject tag differences in means between restricted and unrestricted cables within embassy) whereas the
thicker bar plot in the foreground is the sample level difference in means. These two subplots demonstrate there is subject imbalance between unrestricted and restricted cables bothon aggregate levels and, generally speaking, at individual embassy levels. In the post-2005
sample, “A – Administrative Affairs” tend to be more public, “B – Business Services” tend
to be more public, “C – Consular Affairs” tend to be more public, “E – Economic Affairs”
tend to be more public, “M – Military and Defense Affairs” tend to be more private, “O –Outreach” tends to be more public , “P – Political Affairs” tend to be more private, “S –Social Affairs” tend to be more public, and “T – Technology and Science” issues are slightly
more public on average. The overall subject distribution in the matched sample tends to beweighted more towards “Political Affairs,” “Economic Affairs,” and “Military Affairs.”
23

Number of Cables by Location in Matched Sample
Secretary of StateUSUN New York
Embassy Ankara
Embassy Bogota
Embassy Beijing
Embassy Kuwait
Embassy Jakarta
Embassy Baghdad
Consulate Adana
Embassy New Delhi
Embassy Kinshasa
Embassy Paris
Embassy Port Au Prince
American Institute Taiwan Taipei
Embassy Nairobi
Embassy Tokyo
Embassy Kabul
Embassy Vienna
Embassy Amman
Embassy Zagreb
Embassy The Hague
Embassy Cairo
Embassy Manila
Embassy Dhaka
Embassy Moscow
Embassy Seoul
Consulate Lagos
Embassy Abuja
Embassy Madrid
Embassy Asuncion
Embassy Khartoum
Embassy Colombo
Embassy Bratislava
Embassy Tbilisi
Embassy San Salvador
Embassy Minsk
Embassy Rome
Embassy Bangkok
Embassy Lima
Embassy Vilnius
Embassy Tegucigalpa
Embassy Ottawa
Embassy Sanaa
Embassy Tel Aviv
Embassy Islamabad
Embassy Athens
Embassy Dushanbe
Embassy Santo Domingo
Embassy Quito
Embassy Prague
Embassy Abu Dhabi
Embassy Riga
Embassy Nicosia
Embassy Ashgabat
Embassy Sofia
Embassy Pretoria
Embassy Buenos Aires
Embassy Manama
Embassy Caracas
US Mission Geneva
Embassy Yerevan
Embassy Rabat
Embassy Lisbon
Embassy Dublin
Embassy Stockholm
Embassy Riyadh
Embassy Brasilia
Embassy Accra
Embassy Dar Es Salaam
Embassy Brussels
Consulate Istanbul
Embassy Ndjamena
Embassy Ljubljana
Embassy Warsaw
Embassy Beirut
Embassy Lilongwe
Embassy Skopje
Embassy Tunis
Embassy La Paz
Embassy Santiago
Embassy Maseru
Embassy Kathmandu
Embassy Muscat
Embassy Addis Ababa
Embassy Mexico
Embassy Guatemala
Embassy Freetown
Embassy BelgradeEmbassy Astana
Embassy Panama
Embassy Berlin
Embassy Helsinki
Embassy Bucharest
Embassy Tashkent
Embassy Tallinn
Embassy Canberra
Consulate Jerusalem
Embassy Maputo
Embassy Kyiv
Embassy Kingston
Embassy Budapest
Embassy Algiers
Embassy San Jose
Embassy Oslo
Embassy Djibouti
USEU Brussels
Embassy Sarajevo
Embassy Baku
Embassy Kuala Lumpur
Embassy Doha
Embassy Bridgetown
Embassy Nassau
Embassy Conakry
Embassy Wellington
Embassy Tirana
Embassy Montevideo
Embassy Hanoi
Embassy DiliEmbassy Harare
UNVIE
Embassy Managua
Consulate Hong Kong
Embassy London
Embassy Kigali
Embassy Bishkek
Embassy Abidjan
Mission USNATO
Embassy Damascus
Embassy Singapore
Embassy Bamako
Embassy Pristina
Embassy Phnom Penh
US Office Almaty
Embassy Dakar

0

200

400

600
800
Total Cables

1000

1200

Figure 5: Counts of cables by embassy in the matched sample.
24

1400

column is a measure of a word’s relative importance to prediction accuracy in the embassy’s
matched sample. Each cell entry is then the RF importance measure for that term in thatembassy. To obtain sample-average estimates of word-level importances to prediction, weweight weight the results of each embassy-level importance vector by its relative share of all
cables in the exactly matched sample. The prevalence of any given embassy in the matchedsample, therefore, proportionately weights the importance terms associated with that embassy (thus, for example, we will up-weight the importance terms associated with the State
Department itself and other embassies near the top of Figure 5). Using the sample-weighted
results of the RF within-embassy, exactly-matched classification procedure, we then took
the top 30 of these terms (recall that they are all positively signed, regardless of their actual
signed effect on classification), and recorded their corresponding coefficients from the lasso
regressions at the embassy level. The lasso regression coefficients are similarly weighted as
sample averages in proportion to each embassy’s representation in the matched sample.

4.2.3

Supplementary Analysis: Topics

Some supplementary analyses are performed to address what differentiates more restricted
documents from less restricted documents on the margins. In particular, we topic model
our sample of n = 163, 958 cables, using the most common probabilistic topic model incontemporary text analysis research, Latent Dirichlet Allocation, henceforth referred to as
LDA (Blei et al, 2003). Information on our topic modeling procedure is outlined in Appendix
E. Results of the topic modeling procedure are used as an illustrative aid to categorize thewords we find to be predictive of document restriction.

25

5

Results

We first interpret our tag regressions in terms of the nature of the substantive secrecy they
reveal, before considering the evidence for our procedural secrecy hypothesis above.

5.1

Substantive Secrecy: High vs. Low Politics

Recall that testing for substantive secrecy boils down to testing whether or not the probability a diplomatic cable is withheld from the public is measurably predicted by the subjectof the cable communication, adjusting for the cable’s location of origin and other factors.
Figure 6 presents this analysis, where each point corresponds to an estimate of the sample
average effect of a subject TAG on the probability of the cable’s restriction. Around eachestimate is the 95-percent confidence interval. In terms of coefficient direction, note that thebroken line in the center of the plot denotes a point estimate of zero ‘effect’: tags to the rightof this line are generally associated with restricted documents (on average); the presence oftags to the left, generally predict an unrestricted status for the cables. Tags highlighted in
red indicate coefficients that are statistically differentiable from zero. Our first observationis that there are a large number of statistically significant predictors: almost every subject
matter tag is associated with increasing or decreasing the probability that a particular cable
is restricted. Second, we note that the direction of the effects are somewhat in line with ourpriors. Thus we see that cables concerning “Terrorists and Terrorism”, “Military Capabilities”, “Intelligence,” and “National Independence,” for example, are more likely to be kept
private than cables concerning “Migration” “Narcotics,” “Personnel,” or “Environmental
Affairs.” In particular, we see that dispatches dealing with ‘core’ state secrets, especiallypertaining to information, capabilities and threats are restricted. We note that such subject
matter accords with notions of ‘high politics’—specifically, state security and survival—asdescribed by Keohane and Nye (1977). On the other hand, cables that discuss more ‘public

26

Marginal Effects of U.S. State Department Subject Tags on Probability of Restriction
Visa Lookout●
Internal Governmental Affairs
External Political Relations
National Independence
Diplomatic Courier Operations
National Boundaries, Territories, and Sovereignty●
Arms Controls and Disarmament●
Petroleum and Natural Gas●
Allowances●
Diplomatic Pouch and Mail●
Terrorists and Terrorism●
Military Capabilities●
INR Program Administration●
Propaganda and Psychological Operations●
Refugees●
Internet Administration●
Intelligence●
U.S. Sponsored Schools●
Military Operations●
Minerals and Metals●
Medical Services●
Energy Technology●
Space Activities●
Military and Defense Arrangements●
Visits and Travel of Prominent Individuals and Leaders●
Energy and Power●
Security●
Automated Data Processing●
Science and Technology Policy●
Engineering Research and Development●
Financial and Monetary Affairs●
Social Conditions●
Business Services Reporting●
Assistance to Citizens●
Foreign Investments●
U.S. Diplomatic Representation●
Military Nuclear Applications●
Communications, Circuits, and Networks●
Buildings and Grounds●
Foreign Trade●
Passport and Citizenship●Economic Conditions●
Economic and Commercial Internet●
Trade and Investment Opportunities●
Land Transportation●
National Security
●
Visas
●
Civil Aviation●
Inspector General Activities●
Agriculture and Forestry●● Human Rights
Personal Computers●
Foreign Economic Assistance●
Judicial Assistance and Notarial Services●
Commercial Fishing and Fish Processing●
Industry and Manufacturing●
Communications and Postal Systems●
Trade and Technology Controls●
Cultural Affairs●
Waterborne Transportation●
Communication Operations and Administration●
High Seas Affairs●
Internet Technology●
Foreign Service National Personnel●
Physical Sciences●
Management Operations●
U.S. Congressional Travel●
COMSEC Key Management●
Emergency Planning and Evacuation●
Personnel●
Environmental Affairs●
Budget Services and Financial Systems●
Foreign Diplomats and Foreign Missions●
Telecommunications Equipment Maintenance●Federal Agency Services●
Military Assistance and Sales●
Labor Sector Affairs●
General Accounting Office●
Narcotics●
FCS Management Operations●
Biological and Medical Science●
Employees Abroad●
International Organizations and Conferences●
Educational and Cultural Exchange Operations●
Migration●
Science Grants●
Foreign Service Institute●
Office Management Specialist Issues●
Financial Management●
Trade Expansion and Promotion●
Information Management Services●
Consular Administration and Management
●

●
●
●

●

TravelDiplomatic CorrespondenceInternational Information Programs●
Public Relations and Correspondence
●

●

●

−0.6

−0.4

−0.2

0.0
βtag

0.2

0.4

0.6

Figure 6: Substantive content as a predictor of secrecy status: estimates in red are statistically distinguishable from zero following the “Holm-Bonferroni” correction for multiple
comparisons (Holm, 1979). 95% CIs around each estimate. All cables written in and after2005 used in estimates, along with cable-origin fixed effects. The central line corresponds toa β̂t of zero.
27

good’ orientated matters—wherein we can imagine that sharing information may not bedamaging, and may in fact be optimal—tend to be unrestricted. In this latter category
are tags that seem to require or be synonymous with publicity and the dissemination of information: “International Information Programs”, “Public Relations and Correspondence”,“International Organizations and Conferences”, “Educational and Cultural Exchange Operations” and so on. With respect to the work of Keohane and Nye (1977), we might see suchmatters as ‘low politics’: issues of more domestic or economic concern.

The fact that cable substance drives at least some part of diplomatic secrecy should notcome as a surprise to theorists of rational diplomacy. As noted above, most contemporary
theoretical treatments of crisis diplomacy concern agents’ incentives to misrepresent their resolve, capabilities, or information in bargaining settings: our results here suggest the UnitedStates. acts in a way compatible with that logic.

5.2

Matched Sample Results: Procedural Secrecy

In terms of procedural secrecy, an overview of our main results may be found in Figure
7. Recall that we used the RF algorithm to identify the thirty ‘most important’ tokens interms of their ability to discriminate between the unrestricted and restricted cables statusof a document. In the second column of the plot, these are clearly seen and include words
such as ‘said’, ‘told’, ‘ambassador’, ‘want’, ‘note’, ‘meet’, ‘want’, ‘ask’, ‘discuss’, ‘concern’,‘state’, ‘agre[e]’ ‘support’, ‘however’, ‘thank’, ‘request’, ‘possibl[e]’, ‘like’ and so on. Our
immediate observation is that in stark contrast to our tag regressions, these words do notconnote substantive state secrets per se; rather, they refer to the holding of meetings and thegeneral protocols of diplomatic exchange with foreign nationals. Related to this idea, notethe presence of terms such as ‘poloff’ (the Embassy’s Political Officer), ‘usg’ (United States
Government) and ‘minist’ (minister): actors who we expect to be involved in daily embassy
28

interactions. On the left of the figure, we report the lasso (point) estimate associated withthe terms. When these points are to the right of the vertical line, the use of that word (onaverage) increases the probability that a document is restricted; when the points are to theleft, this suggests that the word is associated (on average) with a decrease in probability
that a document is restricted. Examining this part of our results, we note that terms suchas ‘said’ and ‘told’, ‘request’, ‘like’ are used disproportionately more in restricted cables. To
us, this is evidence that once one controls for substantive area, secrecy is mostly about keeping meetings private and confidential, regardless of whether anything intrinsically ‘secret’ isbeing discussed.

To evaluate this intuition, we recorded the modal topic—i.e., for each word, the topic withthe highest posterior probability from the topic model described earlier—in which our mostinfluential words appeared. If we are correct that secrecy is partly about a norm of discretion
rather than content, we would expect to see most of the terms mapping to a single (or perhaps a few) ‘administrative’ topic(s), rather than topics pertaining to matters of substantive
import. On the right-hand side of the plot, we see this is almost entirely the case. There,
the solid lines lead from each word to the topic it most likely belongs; the dashed lines arefrom each word to second most likely topic. We see first that with a few exceptions, allof the words ‘belong’ to the first, second, or third topics. Inspecting those more closely,
we note that those topics generally consist of administrative nouns and verbs, rather thansubjects of interest: thus, we find “said” in the first, second, and third topic as a leading
word, while ‘will’ appears in the fourth topic. Importantly, the words that we have identified as discriminating between unrestricted and restricted cables do not appear alongsideobviously substantive subject matter such as pertains to the Middle East (topic six or topic
seven), the Pacific rim (topic 8), nuclear proliferation (topic 16) or Russian aggression (topic
14). Of course, we do see that some terms are likely to appear within certain substantive
29

Words with Highest Predictive Importance in Matched Sample (Procedural Secrecy)
Avg. Lasso Coeff.

(RF Rank) Word

●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●

−0.01

0

0.01

Modal Topic(1) said(2) told(3) ambassador(4) note(5) meet(6) text(7) point(8) want(9) ask(10) poloff(11) minist(12) will(13) discuss(14) concern(15) state(16) usg(17) unit(18) background(19) reftel(20) ani(21) posit(22) agre(23) support(24) mfa(25) however(26) thank(27) request(28) possibl(29) like(30) demarch

LDA Topic
said, note, meet, mfa, foreign, will, discuss, request
ambassador, said, presid, visit, note, minist, meet, also
minist, said, will, govern, presid, prime, new, parliament
will, support, secur, need, work, plan, reform, internstate, committe, propos, unit, text, deleg, provid, articl
said, lebanon, syria, sudan, egypt, arab, syrian, darfur
governor, muslim, said, provinci, leader, member, council, religi
japan, north, korea, will, minist, japanes, govern, prime
inform, control, train, drug, law, border, custom, enforc
will, compani, project, oper, said, one, port, plan
travel, embassi, secur, visitor, post, offic, inform, willreport, polic, attack, secur, forc, protest, press, twooil, gas, energi, uae, compani, will, agreement, project
court, case, prison, investig, law, legal, right, judg
program, particip, develop, univers, student, educ, organ, includ
russia, russian, georgia, said, moscow, nato, czech, ukrain
iran, nuclear, iranian, said, azerbaijan, german, sanction, bahrainright, human, south, french, africa, african, cuba, francchina, taiwan, chines, will, unit, state, chen, presid
health, assist, food, provid, program, water, will, refuge
militari, pakistan, afghanistan, defens, forc, india, nato, afghan
invest, law, trade, foreign, govern, busi, compani, industri
burma, thai, thailand, post, gob, somalia, asean, applicparti, elect, polit, will, vote, opposit, support, candid
israel, palestinian, isra, will, report, gaza, hama, jordan
turkey, turkish, pkk, got, ankara, will, said, greek
labor, traffick, women, child, children, govern, work, victim
percent, bank, million, increas, econom, budget, billion, year
iraq, iraqi, baghdad, kuwait, goi, secur, will, maliki
name, rank, father, birth, dob, date, pob, unit

0.02

Note: LDA topics ordered from top−to−bottom by cosine similarity to first topic. Each topic is labeled by its most−indicative word stems, in rank order.

Figure 7: Procedural secrecy: matched sample results. For each of the words listed on
the lefthand side of the plot, a solid line maps that word to its most likely topic (given
estimates from the LDA model described in Appendix E). A dotted line maps each word toits second most likely topic. The topics listed on the righthand side of the graph are ordered
in a specific manner: the uppermost topic is the mode of the modal topic assignments (i.e.,
the topic that is most frequently the modal topic assignment for the top RF terms), while
subsequent topics are presented in descending order according to their similarity to the first
topic. Topical similarity determined by the cosine similarity between topic vectors. The plot
reveals remarkable concordance on the following: words that are most predictive of secrecy
tend to be used in similar topics, and those topics tend to concern the official business offoreign leaders, their meetings, and words relating to information exchange. In general, thesewords also have positive positive lasso regression coefficient estimates, which implies theiruse on average is positively associated with cable restriction.

30

topics (such as ‘meet’, which appears in a ‘Burma’ topic and ‘demarch[e]’ which appears in
an Israel topic towards the bottom of the plot). Such occurrences are not the norm, however.

In terms of the theories presented earlier, our finding here seems most closely compatible with the work of Sartori (2002) and Kurizaki (2007) insofar as privacy seems to be
intrinsically valued by diplomats, rather than because it allows per se information exchange.

5.3

Share of Secrecy: Substance vs. Procedure

Above, we made the claim that while some of observed diplomatic censorship is a consequence of the need to protect state secrets, at least part of it results from the need to keep
meetings confidential as a procedural requirement, regardless of what is to be discussed. In
our final set of results, we attempt to estimate the relative contribution that these two separate elements make to the practice of restricting information from public view. In Figure 8
we report a comparison of models with this in mind. Here, ‘Tags’ refers to the tag covariates
we noted earlier, ‘Embassy’ are simply embassy fixed effects, and ‘Words’ are the top 30
words selected by the Random Forest procedure above. In all cases, the numbers to theright of the bars refer to the percent correctly predicted (unrestricted and restricted) by agiven (logit) model in the entire sample of 163,958 documents.

Unsurprisingly, we see that a model with tags, the word information, and the embassy
fixed effects does best in terms of the proportion of documents it can classify correctly, ataround 92%. The null model, the sample proportion of restricted cables is 59%, and clearly
the statistical model improves substantially upon this. More interesting from our perspective
is a comparison of the second and third bar (‘Words + Embassy’ and ‘Tags + Embassy’), andthe fifth and sixth (‘Words’ and ‘Tags’) since the performance of the models using the RF
words and tags are so similar. That is, it seems that whether we use the substantive topics
31Classification Performance of Logistic Regression Models
92.09

Tags + Words + Embassy
Tags + Embassy

90

Words + Embassy

89.59

Tags + Words

88.26

Embassy

85.83

Tags

84.52

Words

83.63
59.45

'Null' Model

0

20

40
60
Percent Correctly Classified

80

100

Figure 8: Comparison of classification performance for logistic regression models with 10fold cross validation on the full post-2005 sample (n = 163, 958). ‘Tags’ denote presence of
subject tags, ‘Words’ the top 30 RF words, and ‘Embassy’ denotes inclusion of embassy IDs.
The value of the ‘null model’ represents the proportion of the modal restriction status in thedata set (i.e., restricted).

32

alone, or the words that we identified as connoting secret meetings rather than substance,
our model performs similarly. This suggests, at the very least, that both substantive andprocedural secrecy matter for diplomatic communication, and that both the audience costtheories and more recent work on communication have some support in the data.

6

Discussion

Conflict and bargaining have always been at the core of international relations and its study(see, e.g., Thucydides, 1910/431BCE; von Clausewitz, 1832/1989). In recent times, the discipline has amassed an impressive array of theoretical models that make use of, or provide
findings for, ‘information’ and its dissemination between actors. This paper opened by noting that, despite this voluminous literature, there is little systematic statistical work on thesubject, and that this is hardly surprising given that secrets—by definition—are difficult toresearch. In this paper, we made use of the WikiLeaks ‘Cablegate’ disclosure of diplomaticcommunications, a new and contemporary dataset that has an unusually large amount of‘uncensored’ content (that was not systematically edited), to examine the empirical support
for various conceptions of secrecy and communication. We argued that diplomatic confidentiality, i.e. information actively kept from the public, is used in at least two scenariosor ‘dimensions’: first, in a way pertaining to substance and second, pertaining to procedure.
In the former case, documents that deal with issues that could damage U.S. capabilities
were they available to others, are disproportionately kept secret. Meanwhile, in cases where
publicity is helpful to the U.S. government are made available either for direct public consumption or for distribution to those who will have the opportunity to influence opinion.
More speculatively, and untested here, information may be released because it creates a
useful ‘audience cost’ and encourages commitment to a costly path of action. In the second
case, that of the procedural dimension, diplomats ensure that the circumstances and process33

of meetings in general—regardless of their actual subject content—are not disclosed. To beclear, we found evidence of both dimensions in our data, and were able to characterize theircontent and nature. In this way, both the recent literature that emphasizes the importanceof diplomacy (e.g., Sartori, 2002; Kurizaki, 2007; Trager, 2010), and the earlier ‘rationalist’
literature that has it playing little role, finds some support here.

Apart from the preliminary analysis our paper provided, it also contributed methodologically to a growing area of political science: that of text analysis. In particular, we were
faced with a situation in which documents had to be compared within particular subject
areas, such that their discriminatory terms could be uncovered. We used an exact matching algorithm to get at our textual quantities of interest. In our case, the subject matter
was determined by the U.S. State department (via the TAGS system), but the problem isobviously more general than this. For example, one might be interested in the success (orotherwise) of different bills in Congress or the public opinion reception of speeches fromprimary candidates. Clearly, the subject matter between documents differs and needs to be‘controlled’ for in some sense. We provided one way of proceeding in such situations.

Of course, analytically, we have only scratched the surface here. Though we have documented the nature and structure of secrecy and the cables themselves, there is much more
to do. First, while we argue that the ‘more secret’ topics in the TAGS system seem to deal
more fully with capabilities than the ‘least secret’, we are necessarily vague on the details.
We would like to know more about why exactly some subjects are kept from public view,
and whether such decisions accord with IR theory in the area: for example, is topic secrecy
actually dictated by a desire to avoid revealing capabilities on a particular subject, or is
it more connected to notions of resolve, or even just the information dispensing machinery
itself? Second, although we do not directly engage the plausibility of the audience cost lit34

erature and its critics (see Slantchev, 2012, for a review), our findings are at least minimallyconsistent with both sides of that debate, insofar as we find some evidence that the U.S.
attempts to make more public its views (and thus possibly create such ‘audience costs’)
where helpful, but not always. That is, it seems to preserve ‘room for maneuver’ in someareas. Subsequent analysis might weigh in more helpfully on this debate by considering
the constraints that U.S. officials face in the various areas of international relations withwhich it deals: for example, we might be interested to know whether, in fact, issues that theUnited States is seemingly ‘open’ about with the public are simply those where it cannot
be otherwise given commonly held knowledge about the U.S. position (or it weaknesses) in
the wider world. This is ultimately a call to incorporate more topic-specific covariates andcircumstances in the analysis. Finally, while we have emphasized the importance of private
diplomatic meetings as part of the arsenal of U.S. international relations practice, we havedone little to explain how or why they are used. That is, we are not much the wiser as towhich of the various theories (Sartori, 2002; Kurizaki, 2007; Trager, 2010, e.g.) of diplomatic
exchange is correct, if any. A continued digging into the WikiLeaks corpus might allow for
more direct tests of these models. We leave such questions for future work.

35

A

How Cables are Written and Classified

U.S. State Department cables are official communications between Foreign Service officials,
embassies, consulates, and international organizations (such as the United Nations Headquarters in New York City, The Hague) around the world. These communications serve atleast two purposes. First, they exist to share information regarding the daily proceedings ofan embassy or nation with other institutions around the world, on topics broadly related tothe overall interests of the United States. Second, they exist to create official records (i.e.,a database) of political information relevant to particular foreign outposts over time. An
early articulation of these twinned objectives can be found in Department of State (1974,
http://aad.archives.gov/aad/content/aad_docs/rg59_state_dept_tags_74.pdf).
When written, diplomatic cables take a standardized form. For example, the State Department provides glossaries of suggested language for officials to use in discussions of specificpolitical issues in official communications. The “Termdex” chapter of 5 FAH-3 TAGS Terms
Handbook serves this role, which “is an alphabetic list of words and phrases frequently foundin Departmental communications”(1), requests officials use the terms “Border Incident” instead of “Border Violation” when discussing territorial disputes in official communications(5 FAH-3 H-810, pg. 9), or to use “Liberation Front” instead of “Liberation Movement” (5FAH-3 H-810, pg. 40). So too, the Termdex recommends that particular phrases, if used,
be used in tandem with specific subject TAGS—e.g., cables that discuss “Collective Bargaining” be tagged with the political tag “ELAB: Labor Sector Affairs”, and discussions of“Collective Security” be tagged with “MARR: Military and Defense Arrangements” (13).

After a cable is written and political subject TAGS are marked in metadata, cables are
assigned an overall restriction status. Generally speaking, for any single cable in our sample
it is impossible to the full set of individuals who may be involved in a single communication’s

36restriction status, just as it is impossible to know if esoteric operational practices exist across
U.S. embassies in the sample. That said, there is sufficient reason to believe a cable’s restriction status is determined after a cable’s text is complete, and generally this determination
is made by a person of authority at the cable’s location of origin (e.g., an Ambassador, ora Deputy Assistant Secretary, as articulated in “Original Classification Authority”, 5 FAH3 H-714.1, pg. 2). The Foreign Affairs Manual recommends a communication’s “overall
classification level is determined by the highest classification level of any of the portions”
of its text. In other words, after a communication is written, an embassy’s classification
authority reviews all portions of a communication to check for the sensitivity of all portionsof a message, where a “portion is ordinarily defined as a paragraph but also includes subject
lines, titles, subheadings, tables, maps, photographs, graphs, and any other inserts withintext” (“Classification Level”, 5 FAH-3 H-713, pg. 2). This acknowledges that while not all
portions of a cable may be equally sensitive, a cable’s overall restriction status is set once alldetails of its contents have been reviewed. It is each cable’s overall restriction status that
we use for analysis in this study.

37

B

Additional Details on Lasso and Random Forest

The RF and lasso procedures require brief explanation as they are not widely used in political research, although inevitably many technical details will be left for readers consult in theworks cited. Both are widely used in “small n, large p” settings: cases in which there may
be there may be a greater number of possible parameters than observations in the sample.

The RF algorithm is a decision tree and resampling-based classification procedure which
relies on repeatedly dividing the observed sample of data into random bootstrapped training
datasets and fitting decision trees to each random training set, then aggregating the classification results over all independent training sets. In the ‘statistical learning’ literature,
this procedure is commonly referred to as bootstrapped aggregation (i.e., “bagging”), andcan be widely applied to improve the classification precision of various models, regression
included. A RF algorithm procedure deviates from bagging alone by also randomly sampling the parameter space included in each iteration of this bagging procedure (e.g., Ho,
1998). One result of procedures like RF is it allows researchers to think about the relative
variable importance of predictors in a classification setting. Due to the fact that at eachbagged iteration of the procedure there are random subsets of the feature space included in
the decision-trees, not all predicting variables (i.e., “words” in our context) are likely to be
included as predictors at each stage of the algorithm. Overall, a predictor’s variable importance can be thought of as a result of this process: an estimate of the marginal reduction inclassification error that results from a single word’s inclusion to the classification procedure
overall, given the random inclusion of other predictor variables from the sample.

The lasso is a form of penalized regression, similar to ridge regression, whereby regression coefficients are weighted by “shrinkage factors” such that regression coefficients are38

weighted towards zero (Tibshirani, 1994; Hastie, Tibshirani and Friedman, 2009). The lasso
is commonly used for feature selection in high-dimensional learning problems to decrease thevariance of a particular classifier. In our context, the procedure is similar to an ordinary leastsquares regression procedure in which the best-model is determined by that which minimizes
the in-sample sum of squared residuals, except regression coefficients are penalized according
to prior rules (i.e., the shrinkage factor and tuning factor) on the minimum coefficient sizea variable is allowed to have to be included in the final classification model. The estimatespresented in Figure 7 are obtained from taking the average lasso coefficient for each word
over a representative range of shrinkage factors, where each model is estimated at the embassy level using its exactly matched data subset.

39

C

Cable Tags

Tag

Meaning

Meaning

1) AADP

Automated Data Processing51) ELTN

Land Transportation

2) ABLD

Buildings and Grounds

52) EMIN

Minerals and Metals

3) ABUD

Budget Services and Financial Systems

53) ENRG

Energy and Power

4) ACOA

Communication Operations and Administration

54) EPET

Petroleum and Natural Gas

5) ACKM

COMSEC Key Management55) ETRD

Foreign Trade

6) ADCO

Diplomatic Courier Operations56) ETTC

Trade and Technology Controls

7) ADPM

Diplomatic Pouch and Mail

57) EWWT

Waterborne Transportation

8) AEMR

Emergency Planning and Evacuation58) MARR

Military and Defense Arrangements

9) AFIN

Financial Management59) MASS

Military Assistance and Sales

10) AFSI

Foreign Service Institute60) MCAP

Military Capabilities

11) AFSN

Foreign Service National Personnel61) MNUC

Military Nuclear Applications

12) AGAO

General Accounting Office62) MOPS

Military Operations

13) AINF

Information Management Services63) ODIP

U.S. Diplomatic Representation

14) AINR

INR Program Administration64) OEXC

Educational and Cultural Exchange Operations

15) AINTInternet Administration65) OFDP

Foreign Diplomats and Foreign Missions16) ALOW

Allowances

66) OIIP

International Information Programs17) AMED

Medical Services67) OPDC

Diplomatic Correspondence18) AMGTManagement Operations68) OPRC

Public Relations and Correspondence19) AMTC

Telecommunications Equipment Maintenance69) OREP

U.S. Congressional Travel20) ANET

Communications, Circuits, and Networks70) OSCI

Science Grants21) AODE

Employees Abroad71) OTRA

Travel22) AOMS

Office Management Specialist Issues72) OVIP

Visits and Travel of Prominent Individuals and Leaders

23) AORC

International Organizations and Conferences

73) PARM

Arms Controls and Disarmament24) APCS

Personal Computers74) PBTS

National Boundaries, Territories, and Sovereignty

25) APER

Personnel75) PGOVInternal Governmental Affairs

26) ASCH

U.S. Sponsored Schools76) PHSAHigh Seas Affairs27) ASEC

Security77) PHUM

Human Rights

28) ASIG

Inspector General Activities

78) PINR

Intelligence29) BBSR

Business Services Reporting79) PINS

National Security30) BEXP

Trade Expansion and Promotion80) PNAT

National Independence31) BMGTFCS Management Operations81) PREF

Refugees32) BTIO

Trade and Investment Opportunities

82) PREL

External Political Relations

33) CASC

Assistance to Citizens

83) PROP

Propaganda and Psychological Operations34) CFED

Federal Agency Services84) PTER

Terrorists and Terrorism

35) CJAN

Judicial Assistance and Notarial Services85) SCUL

Cultural Affairs36) CLOK

Visa Lookout86) SENV

Environmental Affairs37) CMGT

Consular Administration and Management87) SMIG

Migration38) CPAS

Passport and Citizenship88) SNAR

Narcotics39) CVIS

Visas89) SOCI

Social Conditions40) EAGR

Agriculture and Forestry90) TBIO

Biological and Medical Science41) EAID

Foreign Economic Assistance91) TINTInternet Technology42) EAIR

Civil Aviation

92) TNGDEngineering Research and Development43) ECON

Economic Conditions93) TPHY

Physical Sciences

44) ECPS

Communications and Postal Systems

94) TRGY

Energy Technology45) EFIN

Financial and Monetary Affairs

95) TSPASpace Activities

46) EFIS

Commercial Fishing and Fish Processing96) TSPL

Science and Technology Policy47) EIND

Industry and Manufacturing48) EINT

Economic and Commercial Internet

49) EINV

Foreign Investments50) ELAB

Labor Sector Affairs

Figure 9: List of State Department “Subject TAGS” observed in sample and their meanings.
Meanings taken from “ Subject TAGS” in the U.S. Department of State Foreign Affairs
Manual Volume 5, Handbook 3.
40

D

Average Cable Restrictiveness by Embassy
Cable Totals and Restriction Frequency by Place of Origin: post−2005
Origin#U1) Embassy Baghdad
2) Secretary of State
3) Embassy Tokyo
4) Embassy Ankara
5) American Institute Taiwan, Taipei6) Embassy Paris
7) Embassy Moscow8) Embassy Tel Aviv
9) Embassy Beijing
10) Embassy Madrid11) USUN New York12) Embassy Bangkok
13) Embassy New Delhi
14) Embassy Jakarta
15) Embassy Kuwait16) Embassy Cairo
17) Embassy Beirut18) Embassy Kabul
19) Embassy Amman
20) Consulate Jerusalem21) Embassy Caracas22) Embassy Seoul23) Embassy Dhaka24) Embassy Bogota
25) Embassy The Hague
26) Embassy Islamabad27) Embassy Mexico28) Embassy Colombo29) Embassy Buenos Aires
30) Embassy Khartoum31) Embassy Abuja
32) Embassy Ashgabat33) Embassy Baku
34) Embassy Kathmandu35) Embassy Vienna36) Embassy Nairobi37) Embassy Berlin38) Embassy Damascus39) Embassy Manila40) Embassy Kinshasa41) Embassy Rome
42) Embassy Rangoon
43) Embassy Manama44) Embassy Santiago45) Embassy Muscat46) Embassy Abu Dhabi47) Embassy Bridgetown48) Embassy Pretoria49) Embassy Tashkent50) Consulate Lagos
51) Embassy Harare52) Embassy Tegucigalpa
53) Embassy Asuncion54) Embassy Yerevan55) Embassy Brasilia56) Embassy Santo Domingo57) Embassy Tbilisi58) Embassy Athens59) Embassy La Paz
60) Embassy San Salvador61) Embassy Port Au Prince62) Embassy Riyadh63) Embassy Managua64) Embassy Sanaa
65) Embassy Addis Ababa66) Embassy Kyiv
67) Embassy Ottawa
68) Embassy Rabat69) Embassy Kingston70) Embassy Quito71) Embassy Zagreb
72) Embassy Prague73) Embassy Minsk
74) Embassy Bishkek75) Embassy Lima76) Embassy Warsaw77) Embassy Bratislava78) Embassy Kuala Lumpur79) Embassy Vilnius
80) Embassy Sofia
81) Embassy Tunis
82) Embassy Sarajevo83) Embassy Algiers84) Consulate Hong Kong
85) Embassy Nicosia86) Embassy Astana87) Embassy Dushanbe88) Embassy Ljubljana
89) Embassy Dublin

4970 823 0.14
1772 3456 0.66
1845 3151 0.63
2689 1787 0.40
1431 1502 0.51
1685 1248 0.432172 449 0.171245 1280 0.511737 787 0.31
747 1705 0.701068 1364 0.561380 985 0.421546 773 0.33
1498 615 0.29
1037 1058 0.51
1574 413 0.211746 144 0.08
1072 703 0.401113 642 0.37
1232 519 0.30
1563 162 0.09
1066 640 0.38
778 913 0.54
1032 636 0.38
702 942 0.57
1089 514 0.32
308 1281 0.811081 453 0.30
636 881 0.58
1093 411 0.27
903 431 0.32
1023 305 0.23
1177 145 0.11
1062 253 0.19
503 812 0.62
673 627 0.48
936 331 0.26
810 415 0.34
776 432 0.36
702 497 0.41
807 359 0.31
1059 820.07
967 171 0.15
274 861 0.76
585 526 0.47
700 394 0.36
271 778 0.74
426 608 0.59
853 169 0.17
707 313 0.31
750 268 0.26
616 399 0.39
391 623 0.61
775 230 0.23
374 626 0.63
229 756 0.77
722 255 0.26
662 310 0.32
770 181 0.19
354 578 0.62
475 433 0.48
714 192 0.21
581 313 0.35
593 298 0.33
607 282 0.32
761 122 0.14
473 404 0.46
590 282 0.32
257 599 0.70
598 256 0.30
363 480 0.57
566 268 0.32
509 312 0.38
637 154 0.19
376 381 0.50
586 170 0.22
482 266 0.36
517 209 0.29
426 295 0.41
465 240 0.34
520 182 0.26
542 138 0.20
560 118 0.17
308 361 0.54
459 210 0.31
291 374 0.56
350 307 0.47
420 216 0.34
348 286 0.45

#R

%R

Origin#U#R%R

Origin#U#R%R90) Embassy Wellington91) Embassy Maputo
92) Embassy Brussels
93) Embassy Guatemala94) Embassy Budapest95) Embassy Stockholm96) Embassy Bucharest97) Embassy Lisbon98) Embassy Panama99) Embassy Oslo100) Embassy Conakry101) Embassy Djibouti102) Embassy San Jose103) USEU Brussels104) Consulate Sao Paulo105) Embassy Hanoi
106) Embassy Riga
107) Embassy Doha108) Embassy Maseru
109) Embassy Dar Es Salaam110) Embassy London
111) US Mission Geneva
112) Mission USNATO
113) Embassy Pristina114) Embassy Accra
115) US Interests Section Havana116) Embassy Asmara117) Embassy Ndjamena118) Embassy Abidjan
119) Consulate Adana
120) Embassy Nouakchott121) Embassy Singapore
122) Consulate Istanbul123) Embassy Tripoli
124) Embassy Helsinki125) Embassy Kigali
126) Embassy Freetown
127) Embassy Belgrade
128) Mission Geneva
129) Embassy Dakar
130) Embassy Tallinn131) UNVIE132) Embassy Suva
133) Consulate Jeddah134) Embassy Canberra135) Embassy Lilongwe
136) Embassy Niamey137) Embassy Skopje138) Embassy Vatican139) Embassy Bamako140) Embassy Paramaribo141) Embassy Montevideo142) Embassy Phnom Penh143) REO Basrah144) Embassy Nassau145) Embassy Georgetown
146) Embassy Tirana
147) Consulate Shanghai148) Embassy Dili
149) Embassy Antananarivo150) Embassy Chisinau
151) REO Hillah152) US Office Almaty153) Embassy Yaounde154) Embassy Port Of Spain155) Embassy Cotonou
156) Embassy Kampala157) Consulate Chiang Mai
158) Embassy Vientiane159) Embassy Libreville160) Embassy Lusaka
161) Embassy Banjul
162) Embassy Gaborone
163) Embassy Luanda
164) Embassy Ulaanbaatar
165) Consulate Dubai
166) Consulate Guangzhou167) Embassy Bern
168) Embassy Copenhagen169) Iran RPO Dubai
170) Consulate Peshawar171) Embassy Belmopan172) Consulate Chengdu
173) Embassy Reykjavik174) Embassy Port Louis175) Embassy Monrovia
176) Embassy Bandar Seri Begawan177) Consulate Casablanca178) Embassy Ouagadougou

219
183
386
186
440
300
434
243
345
403
394
290
138
337
30
186
251
403
107
223
309
224
363
324
211
411
406
241
318
0
391
267
287
352
224
274
209
144
354
255
173
271
280
298
184
108
82
191
271
216
60
169
127
288
154
92
153
247
127
155
212
203
121
141
80
30
118
114
127
141
114
163
74
119
91
149
92
144
132
163
158
41
144
78
74
73
91
70
72

410
442
224
392
135
274
137
322
217
134
136
237
385
183
489
332
258
102
389
270
172
256
109
136
247
45
49
212
133
450
55
156
128
62
189
128
192
254
43
138
209
103
67
47
161
231
252
143
63
115
268
152
194
23
145
196
125
26
128
96
33
39
117
87
140
189
96
98
72
57
82
26
112
63
91
27
84
29
39
2
3
119
8
73
74
74
48
66
63

0.65
0.71
0.37
0.68
0.23
0.48
0.24
0.57
0.39
0.25
0.26
0.45
0.74
0.35
0.94
0.64
0.51
0.20
0.78
0.55
0.36
0.53
0.23
0.30
0.54
0.10
0.11
0.47
0.29
1
0.12
0.37
0.31
0.15
0.46
0.32
0.48
0.64
0.11
0.35
0.55
0.28
0.19
0.14
0.47
0.68
0.75
0.43
0.19
0.35
0.82
0.47
0.60
0.07
0.48
0.68
0.45
0.10
0.50
0.38
0.13
0.16
0.49
0.38
0.64
0.86
0.45
0.46
0.36
0.29
0.42
0.14
0.60
0.35
0.50
0.15
0.48
0.17
0.23
0.01
0.02
0.74
0.05
0.48
0.50
0.50
0.35
0.49
0.47

179) Mission USOSCE
108180) REO Kirkuk
116181) Embassy Bujumbura
54182) Embassy Luxembourg62183) Consulate Frankfurt
17184) Consulate Shenyang
97185) Consulate Ho Chi Minh City
68186) Embassy Lome
18187) Embassy Grenada
0188) Consulate Kolkata
13189) Embassy Windhoek
49190) Consulate Rio De Janeiro
7191) Consulate Vladivostok
0192) Consulate Monterrey
46193) Consulate Karachi
75194) Consulate Nogales
0195) Embassy Mbabane
12196) Consulate Halifax
0197) Embassy Valletta
31198) Consulate Lahore
60199) Mission UNESCO20200) Consulate Chennai
16201) Embassy Port Moresby
39202) Consulate Mumbai
22203) Consulate Cape Town
14204) Consulate Munich
23205) Embassy Brazzaville3206) Consulate Dusseldorf
0207) REO Mosul
49208) Consulate Hamburg
8209) Consulate St Petersburg
0210) Consulate Surabaya
0211) Embassy Praia
5212) Consulate Toronto
9213) US Delegation, Secretary30214) Consulate Johannesburg
2215) Embassy Podgorica
0216) Consulate Montreal
8217) Embassy Kolonia
20218) Consulate Quebec
14219) Consulate Guadalajara
2220) Consulate Naha
25221) Embassy Bangui2222) Consulate Milan
18223) Consulate Thessaloniki
0224) Consulate Guayaquil
18225) Consulate Strasbourg16226) Consulate Vancouver
5227) UN Rome0228) Consulate Yekaterinburg
0229) Consulate Durban
11230) Consulate Dhahran
12231) US Mission CD Geneva
16232) Consulate Curacao3233) Consulate Tijuana
4234) Consulate Calgary
0235) Consulate Naples
8236) Consulate Barcelona
2237) Consulate Auckland
0238) Consulate Ciudad Juarez
0239) Consulate Florence2240) Consulate Recife
1241) Embassy Majuro
2242) US Delegation FEST TWO
5243) Consulate Fukuoka
1244) Consulate Sydney
4245) Consulate Leipzig
0246) Consulate Hermosillo
0247) Consulate Melbourne
5248) Consulate Perth4249) Embassy Koror
0250) Consulate Belfast
4251) Consulate Hamilton
0252) Consulate Marseille1253) Consulate Sapporo
1254) Embassy Apia
0255) Consulate Matamoros
0256) Embassy Malabo0257) Consulate Nuevo Laredo
0258) Consulate Osaka Kobe
2259) Consulate Amsterdam
0260) Consulate Merida
0261) ** Dhahran
1262) American Consulate Hyderabad 0263) Consulate Krakow
0264) Department of State0265) DIR FSINFATC
0266) US Office FSC Charleston
0

16
5
56
48
88
7
32
81
97
78
39
79
84
36
4
78
60
71
39
7
47
47
23
38
45
35
51
51
2
41
45
42
35
29
6
33
35
23
11
16
27
4
24
6
24
4
6
17
21
20
7
5
1
13
10
13
5
10
11
9
7
8
7
4
7
4
7
6
1
2
6
1
5
4
4
5
4
4
3
1
2
2
0
1
1
1
1
1

0.13
0.04
0.51
0.44
0.84
0.07
0.32
0.82
1
0.86
0.44
0.92
1
0.44
0.05
1
0.83
1
0.56
0.10
0.70
0.75
0.37
0.63
0.76
0.60
0.94
1
0.04
0.84
1
1
0.88
0.76
0.17
0.94
1
0.74
0.35
0.53
0.93
0.14
0.92
0.25
1
0.18
0.27
0.77
1
1
0.39
0.29
0.06
0.81
0.71
1
0.38
0.83
1
1
0.78
0.89
0.78
0.44
0.88
0.50
1
1
0.17
0.33
1
0.20
1
0.80
0.80
1
1
1
1
0.33
1
1
0
1
1
1
1
1

Note: locations in red indicate cable 'senders' with at least 100 restricted and 100 unrestricted cables in sample.

Figure 10: Frequency of restricted versus unrestricted cables by place of origin, for all cables
post-2005.
41

E

Probabilistic Topic Model

The field of quantitative text analysis has grown substantially in recent years. In this literature, applied researches extensively use Latent Dirichlet Allocation (Blei, Ng and Jordan,
2003) as a generative model to extract “themes” or “topics” from a collection of documents.
See, for example, Blei (2012) for an overview, and Quinn et al. (2010) for recent politicalscience applications of topic models. The model assumes documents are composed of latent
topics that are chosen with probabilities following a Dirichlet distribution, and multinomial
choice probabilities for word choice conditional on a topic. More precisely, the frameworkfrom Blei, Ng and Jordan (2003) has the number of words N in a document be Poisson(ξ),
the latent topic probabilities θ be Dirichlet(α), the topics zn be Multinomial(θ), and thewords wn be Multinomial(β), conditional on zn . Then, with M documents, they have that

Q
Q R
Nd P
p(z
|θ
)p(w
|z
,
β)
dθd . Computational difficulp(θ|α)
p(C|α, β) = M
dn d
dn dn
d
n=1
zdn
d=1
ties arise in this setting, but there are ways to deal with them (e.g., Hoffman et al., 2013).

When a researcher estimates an LDA model, the topics returned are characterized by themultinomial probabilities for all words within each topic, as well as the posterior distributionof topics conditional on a certain word. In practice, the researcher selects the number of
topics a priori, although recent efforts have been made to assess how an approximate numberof topics may be present in a sample of data (see, e.g., Hoffman et al., 2013). We used theseposterior estimates to generate the topical ordering in Figure 7: for each topic, we took thecosine similarity (e.g., Manning, Raghavan and Schütze, 2008) between all pairs of topics,
where similarity between topic vectors is determined by the posterior weights placed on eachword in each topic.

42

F

Exact Matching Algorithm
1. Let Nj be the set of cables from embassy j that occur during or after the year 2005
in the sample, where |Nj | is the number of cables originating from location j.
2. For each of the |Nj | documents in the sample, record the subject tags present oneach diplomatic cable.
3. For all restricted cables in Nj , find all unrestricted cables in Nj that exactly matchon subject tags and year of creation.
4. From the subset of restricted cables in Nj with at least one unrestricted exact
match,(a) Randomly draw a restricted cable and find the unrestricted, exact-matchedcable that is written most closely in time (i.e., the cable that minimizes theabsolute value between the difference in release days). Each cable may bematched with or without replacement.
(b) Continue this process until there are no-more restricted and unrestricted cables to pair together.
5. Record the list of exactly-matched pairs of cables, if applicable.
Figure 11: Outline of Exact Matching Algorithm

43References
Allen, David and Matthew Connelly. 2015. Diplomatic History After the Big Bang: Using
Computational Methods to Explore the Infinite Archive. In Explaining the History ofAmerican Foreign Relations, ed. Frank Costigliola and Michael Hogan. 3rd ed. New York:
Cambridge University Press.
Angeletos, George-Marios and Alessandro Pavan. 2004. “Transparency of Information and
Coordination in Economies with Investment Complementarities.” The American Economic
Review 94(2):91–98.
Angrist, Joshua D. and Jörn-Steffan Pischke. 2009. Mostly Harmless Econometrics: An
Empiricists Companion. Princeton University Press.
Blei, David M. 2012. “Probabilistic Topic Models.” Communications of the ACM 55(4):77–
84.
Blei, David M., Andrew Y. Ng and Michael I Jordan. 2003. “Latent Dirichlet Allocation.”
Journal of Machine Learning Research 2(4-5):993—1022.
Breiman, Leo. 2001. “Random Forests.” Machine Learning 45(1):5–32.
Cornand, Camille and Frank Heinemann. 2008. “Optimal Degree of Public Information
Dissemination.” The Economic Journal 118:718–742.
Cox, David R. 1958. Planning of Experiments. Wiley Publications in Statistics.
Cukierman, Alex. 2001. Accountability, credibility, transparency, and stabilization policy in
the Eurosystem. In The Impact of the EMU on Europe and Developing Countries, ed. C.
Wyplosz. Oxford University Press.
Fearon, James. 1994a. “Domestic Political Audiences and the Escalation of International
Disputes.” American Political Science Review 88:577–92.
Fearon, James. 1994b. “Signaling Versus the Balance of Power and Interests: An Empirical
Test of a Crisis Bargaining Model.” Journal of Conflict Resolution 38(2):236–69.
Fey, Mark and Kristopher Ramsay. 2011. “Uncertainty and Incentives in Crisis Bargaining: Game-Free Analysis of International Conflict.” American Journal of Political Science55(1):149–169.
Frieden, Jeffry and David Lake. 2005. “International Relations as a Social Science: Rigor andRelevance.” Annals of the American Academy of Political and Social Science 600(1):136–
156.
Ghosn, Faten, Glenn Palmer and Stuart Bremer. 2004. “The MID3 Data Set, 1993–2001:
Procedures, Coding Rules, and Description.” Conflict Management and Peace Science21(2):133–154.
44

Gill, Michael and Arthur Spirling. Forthcoming. “Estimating the Severity of the Wikileaks
United States Diplomatic Cables Disclosure.” Political Analysis .
Goemens, Hein. 2000. War and Punishment: The Causes of War Termination and the FirstWorld War. Princeton, NJ: Princeton University Press.
Grimmer, Justin and Brandon Stewart. 2013. “Text as Data: The Promise and Pitfalls ofAutomatic Content Analysis Methods for Political Texts.” Political Analysis 21(3):267–27.
Hastie, Trevor, Robert Tibshirani and Jerome Friedman. 2009. The Elements of Statistical
Learning: Data Mining, Inference, and Prediction. 2nd ed. Springer.
Heckman, James J. and Richard Robb. 1985. “Alternative Methods for Evaluating the
Impact of Interventions.” Journal of Econometrics 30(1):239–267.
Ho, Tin. 1998. “The Random Subspace Method for Constructing Decision Forests.” IEEE
Transactions on Pattern Analysis and Machine Intelligence 20(8):832–844.
Hoffman, Matthew D., David M. Blei, Chong Wang and John Paisley. 2013. “Stochastic
Variational Inference.” Journal of Machine Learning Research 14:1303–1347.
Holm, Sture. 1979. “A simple sequentially rejective multiple test procedure.” Scandinavian
Journal of Statistics 6(2):65–70.
Iacus, Stefano M., Gary King and Giuseppe Porro. 2011. “Multivariate Matching Methods
That Are Monotonic Imbalance Bounding.” Journal of the American Statistical Association 106(493):345–361.
Imai, Kosuke, Gary King and Elizabeth A. Stuart. 2008. “Misunderstandings between experimentalists and observationalists about causal inference.” Journal of the Royal StatisticalSociety, Series A 171(2):481–502.
Jervis, Robert. 1976. Perception and Misperception in International Politics. Princeton,
N.J.: Princeton University Press.
Keohane, Robert and Joseph Nye. 1977. Power and Interdependence: World Politics inTransition. Boston: Little, Brown.
Kurizaki, Shuhei. 2007. “Efficient Secrecy: Public versus Private Threats in Crisis Diplomacy.” American Political Science Review 101(3).
Lake, David. 2010. “Two Cheers for Bargaining Theory: Assessing Rationalist Explanationsof the Iraq War.” International Security 35(3):7–52.
Manning, Christopher D., Prabhakar Raghavan and Hinrich Schütze. 2008. An Introduction
to Information Retrieval. Cambridge University Press.

45

Partell, Peter and Glenn Palmer. 1999. “Audience Costs and Interstate Crises: An Empirical Assessment of Fearon’s Model of Dispute Outcomes.” International Studies Quarterly
43(2):389–405.
Porter, Martin F. 1980. “An Algorithm for Suffix Stripping.” Program 14(3):130–137.
Potter, Philip and Matthew Baum. 2014. “Looking for Audience Costs in all the Wrong
Places: Electoral Institutions, Media Access, and Democratic Constraint.” The Journalof Politics 76:167–181.
Powell, Robert. 1999. In the Shadow of Power,. Princeton, NJ: Princeton University Press.
Powell, Robert. 2002. “Bargaining Theory and International Conflict.” Annual Review of
Political Science 5:1–30.
Quinn, Kevin, Burt Monroe, Michael Colaresi, Michael H. Crespin and Dragomir Radev.
2010. “How to Analyze Political Attention with Minimal Assumptions and Costs.” American Journal of Political Science 54:209–228.
Ramsay, Kristopher. 2008. “Settling It on the Field: Battlefield Events and War Termination.” Journal of Conflict Resolution 52(6):850–879.
Ramsay, Kristopher. 2011. “Cheap Talk Diplomacy, Voluntary Negotiations, and Variable
Bargaining Power.” International Studies Quarterly 55:1003–1023.
Reed, William. 2003. “Information and Economic Interdependence.” Journal of Conflict
Resolution 47(1):54–71.
Reiter, Dan. 2009. How Wars End. Princeton: Princeton University Press.
Renshon, Jonathan, Jooa Julia Lee and Dustin Tingley. Forthcoming. “Physiological Arousal
and Political Beliefs.” Political Psychology .
Rosenbaum, Paul R. 2002. Observational Studies. 2nd ed. Springer.
Rosenbaum, Paul R. and Donald B. Rubin. 1983. “The Central Role of the Propensity Score
in Observational Studies for Causal Effects.” Biometrika 70(1):41–55.
Rosenbaum, Paul R. and Donald B. Rubin. 1984. “Reducing Bias in Observational Studies Using Subclassification on the Propensity Score.” Journal of the American Statistical
Association 79(387):516–524.
Rubin, Donald B. 1976. “Multivariate Matching Methods That Are Equal Percent Bias
Reducing, I: Some Examples.” Biometrics 32(1):109–120.
Sartori, Anne. 2002. “The Might of the Pen: A Reputational Theory of Communication inInternational Disputes.” International Organization 56(1):121–149.
46

Sartori, Anne. 2005. Deterrence by Diplomacy. Princeton: Princeton University Press.
Schultz, Kenneth. 2001. “Looking for Audience Costs.” Journal of Conflict Resolution 45:32–
60.
Shapiro, Jacob and David Siegel. 2010. “Is this Paper Dangerous? Balancing Secrecy andOpenness in Counterterrorism.” Security Studies 19(1):66–98.
Signorino, Curtis. 1999. “Strategic Interaction and the Statistical Analysis of International
Conflict.” American Political Science Review 93(2):279–297.
Slantchev, Branislav. 2004. “How Initiators End Their Wars: The Duration of Warfare andthe Terms of Peace.” American Journal of Political Science 48(4):813–829.
Slantchev, Branislav. 2012. “Audience Cost Theory and Its Critics.” Security Studies 21:376–
382.
Smith, Alistair and Allan Stam. 2004. “Bargaining and the Nature of War.” Journal ofConflict Resolution 48(6).
Snyder, Jack and Erica D. Borghard. 2011. “The Cost of Empty Threats: A Penny, Not a
Pound.” American Political Science Review 105(3).
Stuart, Elizabeth A. 2010. “Matching methods for causal inference: A review and a lookforward.” Statistical Science 25(1):1–21.
Tarar, Ahmer and Behar Leventoglu. 2009. “Public Commitment in Crisis Bargaining.”
International Studies Quarterly 53:817–839.
Thucydides. 1910/431BCE. The Peloponnesian War. London: J. M. Dent.
Tibshirani, Robert. 1994. “Regression Shrinkage and Selection Via the Lasso.” Journal of
the Royal Statistical Society, Series B 58:267–288.
Tomz, Michael. 2007. “Domestic Audience Costs in International Relations: An Experimental Approach.” International Organization 61:821–40.
Trachtenberg, Marc. 2012. “Audience Costs: An Historical Analysis.” Security Studies
21(1):3–42.
Trager, Robert. 2010. “Diplomatic Calculus in Anarchy: How Communication Matters.”
American Political Science Review 104(2).
von Clausewitz, Carl. 1832/1989. On War. Princeton: Princeton University Press.
Waltz, Kenneth. 1979. Theory of International Politics. New York: McGraw-Hill.
Weeks, Jessica. 2008. “Autocratic Audience Costs: Regime Type and Signaling Resolve.”
International Organization 62(1).
47

Wendt, Alexander. 1999. Social Theory of International Politics. New York: Cambridge
University Press.
Werner, Suzanne. 1999. “The Precarious Nature of Peace: Resolving the Issues, Enforcing
the Settlement, and Renegotiating the Terms.” American Journal of Political Science
43(3):912–934.
Woodford, Michael. 2005. “Central Bank Communication and Policy Effectiveness.” NBER
Working Paper No. 11898 .

48