Abstract

1
Introduction
Informal institutions are "socially shared rules, usually unwritten, that are created, commu- nicated, and enforced outside of officially sanctioned channels" (Helmke and Levitsky, 2004, 727; see also Lauth, 2000). Our interest here is in the evolution of one particular informal institution, the study of which yields general methodological lessons for scholars of Compar- ative Politics: the rule that in Westminster systems, the 'Shadow Cabinet'—the group of frontbench spokespersons from the Official Opposition—forms the executive when the party currently in opposition next enters government. This relationship is at the core of Westmin- ster democracy for reasons that are obvious from any textbook account of those systems (e.g. Lijphart, 1999). Yet it has never been part of statute law, and was not always the case in practice: in the 18th and early 19th century, leaders of governments (Prime Ministers) were implicitly or explicitly selected directly by the Crown (Marriott, 1925), and later via some decision-making process within the majority party (see, e.g, Bagehot, 1873/2011; Jenkins, 1996). Prior to modern times, the presence of competing formal and informal institutions meant that conflicts over exactly which set of rules and actors had precedence was common (see, e.g., Erskine May, 1864/1986, on the 'bedchamber crisis'). In modern Westminster systems, where partisan voting is the norm (e.g. Butler and Stokes, 1969; Heath et al., 1991; Clarke et al., 2004)¹ and majoritarian electoral systems deliver disproportionate government numerical superiority in parliament (Bogdanor and Butler, 1983), along with disciplined backbenchers (Cowley, 2002; Kam, 2009), the leadership of the winning party can expect comparatively long durations in government, and the ability to propose and enact legislation close to its ideal point (Powell, 2000).  Thus, the identity of the 'government-in-waiting', and the fact that it will become the executive once in office, has profound implications for almost all actors in the system, including voters and legisla- tors. This is quite apart from other significant roles that the shadow cabinet plays: inter alia , organizing opposition to the government's legislative plans in the division lobbies (see Potter, 1965; Brazier, 1999; Dewan and Spirling, 2011); holding ministers to account in de- bates (Chester and Bowring, 1962; Franklin and Norton, 1993); and providing a formal link between the parliamentary party and its grassroots.²  Yet in stark contrast to the cabinet (e.g. Alt, 1975; Cox, 1987; King, 1994; Jenkins, 1996; Jenks, 1903; Kam and Indridason, 2005; Berlinski, Dewan and Dowding, 2007; Dewan and Myatt, 2010), and with exceptions (e.g. Lowell, 1908; Turner, 1969; Punnett, 1973; Johnson, 1997), there has been little work on the opposition per se.  This is especially true in terms of literature on the origins and development of the shadow cabinet and the informal institution to which it is vital. While competition for votes provides the incentive for the teams of current and would-be ministers to organize themselves in coherent party groups today, it is far from obvious that this was the original motivation for their formation.  Indeed, the dominant explanation for the origins of the cabinet—that of Cox (1987)—focusses on the specifically procedural problems that the organization emerged to counteract: in particular, the resolving of a parliamentary 'tragedy of the commons' after the First Reform Act. Thus, one plausible explanation for the evolution of the shadow cabinet is that it corrected some functional misfiring at the center of Westminster life, and did so in a way that improved the efficiency of the institution as a whole. In this sense, the emergence of the 'government-in-waiting' is an endogenous process, and a response to other forces in parliament—rather than as a product of some exogenous shock directly (see, e.g., Mahoney and Thelen, 2010, for a general discussion of this idea). In contrast to such a theoretical position, scholars have been quick to recognize the importance of the Second Reform Act as ushering in a more modern period characterized by a "triumph of partisan politics" (Jenkins, 1996, ch6), in which parties began to lay out more coherent ideological positions, and turned to more disciplined, hierarchical organiza- tional forms both inside (see Cox, 1992; Rush, 2001; Eggers and Spirling, Forthcoming b ) and outside (see Ostrogorski, 1902/64; Hanham, 1978) parliament itself. From this perspective, the shadow cabinet might be seen as the product of electoral forces unleashed after franchise extension: a strategy—or the consequences of a strategy—by which (opposition) MPs could win working class votes at the ballot box. That is, the shadow cabinet is the product of a competition problem, rather than one of procedure. At base, determining which of these theoretical positions is most plausible—or in what combination—requires that we accurately time the emergence and the nature of the emer- gence of the shadow cabinet. If it came into being in something approximating its modern form around the time of the rationalizations that Cox (1987) describes—i.e. shortly after the 1830s—we have evidence in line with a 'procedural' theory.  In stark contrast, if its emergence is clearly post-Second Reform Act (and to boot, relatively shortly after), we are on firmer ground with a 'competition' story. Of course, the reality of historical change stretched out over a hundred year period of reform means that the researcher is rarely in a position to speak in absolutes: that one account is 'correct' while the other is 'wrong'. A more sensible strategy is thus to weigh the relative heft that might be given to the differing accounts. With such an approach in mind, an immediate problem is that executing any large-n stud- ies of informal institutions is extremely difficult (although see e.g. Desposato, 2006; Stokes, 2006), not least because, almost by definition, they leave less of a 'paper trail' of official documentation. In the specific case of the shadow cabinet, only in very recent times has its membership or activities been recorded for outside observers. 3 The result is that researchers must make more uncertain inferences about who, exactly, constitutes the body itself and what it is doing. This problem is compounded in Westminster systems by the fact that the opposition per se is procedurally weak and hard to observe 'in action': usual metrics for examining the strength of opposition organization—like 'roll rates' (Cox and McCubbins, 2005) or strategic use of committee control (e.g. Krehbiel, 1992) in the US Congress are either very consistently zero or simply non-existent. Put more succinctly, since oppositions almost always lose against governments—in terms of what gets on to the legislative agenda and what becomes law—there is seemingly little variation in legislative output to explain or explore over time. 4  Consequently, studying the opposition and its role in informal institu- tions is extremely challenging. Here, we attempt to improve matters by describing informal institutions in a way famil- iar to political methodologists in Comparative Politics; in particular, via a latent variable representation in which an observed variable for a given unit (speech contents) may be used to make inferences about an unobserved one (shadow cabinet management) and its relation- ship with an outcome of interest (cabinet membership) (see, e.g, Jackman and Treier, 2008; Pemstein, Meserve and Melton, 2010; Svolik, 2014, for applications of this type of approach). As suggested by this strategy, a second contribution below is to provide a text-as-data mea- surement strategy (see, e.g. Slapin and Proksch, 2008; Quinn et al., 2010; Grimmer and Stewart, 2013) using the almost one million utterances between the approximate dates of the First and Fourth Reform Acts (1832–1918) in which the relevant informal institution first emerged and then evolved. We model these speeches using a measure that considers the 'burstiness' (Kleinberg, 2002) of different (government and opposition) actors over time: specifically, we introduce a validated method for scoring individuals via their spoken con- tributions to debate in the House of Commons. This metric relies on the relative 'spike' in activity around particular terms that members of parliament (MPs) use, in order to measure members' latent agenda-setting abilities. By studying the estimated abilities of opposition members relative to each other, we can then infer when and how the shadow cabinet emerged and thus contribute to the debate about its origins sketched above. We find that, put bluntly, while the First Reform Act made the cabinet, the Second Re- form Act made the shadow cabinet. More subtly, we provide theory and evidence to suggest that the 1867 Second Reform Act, and its associated introduction of a "party orientated electorate" (in the sense of Cox, 1987) was crucial for the establishment of a hierarchial opposition leadership, with small numbers of senior individuals increasingly dominating ex- changes from the 1870s and 1880s onwards. We show that after the 1870s, (a) the opposition as a whole was able to wrestle back some noticeable control of the agenda from the cabi- net; (b) a small group of opposition individuals emerged who, relative to their co-partisan colleagues, increasingly dominated debates; (c) the relationship between being one of these individuals and taking a role in the 'next' cabinet controlled by their party was increas- ingly strong. In providing these findings, we expend some effort validating our measure, and demonstrating its strength over potentially misleading alternatives. More generally, a con- sequence of our efforts is that the literature on British political development—including the work on the export of its governance arrangements (e.g. Rhodes and Weller, 2005; Rhodes, Wanna and Weller, 2009)—becomes more evenly balanced, with appropriate focus on both government and its alternative.
2
Shadow Cabinet: Literature and Orientation
Students of British Political Development typically focus on the period of profound trans- formation between the First (1832) and Fourth (1918) Reform Act, and our study does the same. Discussion of changes in parliament during this time can be found in many sources (e.g. Redlich, 1908; Fraser, 1960; Cox, 1987; Rush, 2001), but several key developments are as follows: first, as per Cox (1987), the cabinet as agenda-setter emerged in the 1830s as an attempt to solve a common resource problem—of too many MPs taking up too much time with self-promoting minutiae—in the aftermath of the Great Reform Act. Second, as the century progressed, parties showed increasing cohesion in their division voting (Lowell, 1902; Aydelotte, 1963; Berrington, 1968), with the period of the 'Corn Laws' (McLean, 2001; Schonhardt-Bailey, 2003) and their immediate aftermath one of unusual disunity and party realignment.  Executive dominance was arguably complete in its modern sense with the coming of the 'Railway Timetable', introduced by Prime Minister Balfour in 1902 (Richards, 1988, 145), giving governments clear precedence to introduce (and pass) their legislation with little opportunity for an opposition to overturn such plans on the floor or division lobbies in the House of Commons. Not least because it plays a larger role in policy making, and has done for a longer period, the cabinet has attracted much more scholarly attention than its opposition counterpart. As noted, the Cox (1987) account dominates and suggests that the emergence of the cabinet as an organizational force, whatever its later expanded role in public policy, occurred as a solu- tion to a procedural problem. A puzzle that arises from this accepted assertion is the timing and precise form of the shadow cabinet's emergence as a de facto organization. On the one hand, we might expect it to (begin to) arise fairly soon after this, motivated perhaps by the sudden threat of institutional dominance by a powerful executive or, more charitably, stimu- lated by some other potential efficiency gain. This might include the requirement to provide swift and acceptable transitions of power after elections, 5 or to safeguard backbencher rights in general. Certainly, scholars of other Westminster institutional developments—like the ad- vent of (aggressive) parliamentary questions—have made the case that they arose relatively quickly from the need of non-cabinet members to keep the executive in check (see Chester and Bowring, 1962). Similarly, certain institutional behaviors, such as cohesive division vot- ing against the government's legislation and the commensurate use of government whipping to make executive bills into parliamentary acts (Cox, 1992) started not long after the ratio- nalization noted by Cox (1987). On the other hand, historians suggest that the notion of the informal institution of 'government- in-waiting' did not emerge until much later:  at least until after the Second Reform Act (1867). This suffrage expansion and the "triumph of partisan politics" (Jenkins, 1996) it induced was commensurate with a decline of personalised, patronage appeals (Camp, Dixit and Stokes, 2014). An alternative argument then, is that the shadow cabinet emerged as a collection of opposition leaders with specific designs on governmental roles, putting forth a more unified policy-based appeal than had been previously used in elections. Here, analysts have pointed to early 'campaigning' efforts from the likes of Liberal leader William Gladstone in the late 1870s as the start of this process (see Kelley, 1960; Matthew, 1997). The observational im- plication of this position is that we should not see the evolution of an opposition leadership until after suffrage expanded. While qualitative scholars have documented changes in how opposition leaders acted and, more broadly, how they strategized, they have not provided a systematic assessment of such claims. In particular, they have been unable—mostly due to data limitations—to link par- liamentary activity to both opposition organization and subsequent government formation. Though we will shortly discuss our data driven attempt to resolve the debate noted, two points of circumspection are worth making here. First, our 'testing' of the procedural vs competition theory relies on relative timing after 1832. This is justifiable insofar as the most well-cited and widely accepted account—that of Cox (1987)—portrays the Great Re- form Act as the key impetus for the formation of the cabinet, and that all other sources we have seen claim this institution predated the shadow cabinet. 6  A second, related issue, is that we assume that changes in the legislative and electoral environment take at least a little time to take effect on MP behavior. Naturally, such caveats mean that sharp inference is difficult, a position that we believe is philosophically in line with our comments above about the importance of a measured approach to weighing the evidence for the theories.
A Methodological Problem
2.1
As with much of Westminster constitution-making, formal de jure recognition of entities with political power and importance has traditionally come much later (if at all) than their de facto existence as a force. Thus, the informal practice by which a parliamentary opposition critiques the government has a long history: it was well underway by the 1720s, with the present day term of the 'His Majesty's loyal Opposition' first appearing in debate in 1826 (Johnson, 1997, 488–490). Turning to statute to characterize the hierarchical structure of the opposition is of little help—indeed, as with many other informal institutions at West- minster, it is only with respect to salary commitments that the Leader of the Opposition is mentioned in law. 7  The term 'shadow cabinet' was used as early as the 1880s, though not with any legal basis, and it initially referred to a set of ex-ministers, now out of office as their party was no longer in government (see Brazier, 1997, Ch 3). Initial meetings of the shadow cabinet were more informal than modern practice (and records of them are scant), but in the post-Second World War period in Britain, opposition parties gave chosen senior MPs specific policy responsibilities and titles with the expectation that they would fulfill a similar ministerial role should their party win a subsequent general election. 8  All told, there are few 'official' records of opposition that we can call upon to answer our question of interest. This problem is made harder still because, as alluded to above, oppositions—and shadow cabinets—are weak in procedural terms: Westminster governments are typically single-party (Lijphart, 1999), and face few serious institutional impediments to imposing their will (Pow- ell, 2000.  A consequence is that the opposition rarely achieves legislative 'victories', and thus one cannot usefully measure outcomes that would be seen in other parliaments, such as 'roll rates', successful legislation sponsoring (Volden, Wiseman and Wittmer, 2013) or negative agenda control (e.g. Cox and McCubbins, 2005; Wawro and Schickler, 2006). Since these measures take a value of (near) zero at Westminster, they cannot tell us much about who is organizing opposition to the government. Yet this is a key element of exploring the particular informal institution of interest here. What we do have is speeches, and we return to their use below. Thinking in a more general way, the reason that informal institutions are difficult to analyze is that they are rules which connote probabilistic statistical relationships between inputs ( X ) and outcomes ( Y ) but for which X and/or Y are not directly observable. 9 Typically, when faced with a latent variable that is an important part of some data generating process as X is here, political methodologists in Comparative Politics attempt to infer its values from other, observed variables (see, for example, Jackman and Treier, 2008; Pemstein, Meserve and Melton, 2010; Svolik, 2014). To keep matters simple, suppose that there is one such observed variable, denoted Z , and that for the units in the study, their (latent) value of X probabilistically determines their (observed) value of Z, but that Z does not have any direct relationship with Y itself. In what follows, we will take precisely the route implied by our comments about X , Y and Z . For us, Y is the status of an MP as part of the Cabinet in a particular time period. This is observable. Meanwhile, X is his status (or not) as a member of the Shadow Cabinet in the prior parliament, something that occurs prior to Y . For our period, this is latent, and cannot be directly observed. Below, Z will be an MP's agenda- setting ability, derived from observational data on his speeches via a particular metric that we will define in some detail. Thus, an MP’s speeches will help us infer whether he was a member of the Shadow Cabinet or not (along with other details on the Shadow Cabinet's evolution), and we will then study the relationship between this Shadow status and pro- motion to the Cabinet once his party is in power. In this way, we can assess the changing nature of an informal institution vital to the functioning of Westminster democracy.
3
Data
The data we use are described in Eggers and Spirling (Forthcoming a ) but the essence is this: between 1832 and 1915. 10 They have been disambiguated in terms of speaker, which in turn has been matched to a unique MP identity.  Other information pertaining to these MPs includes their party affiliation in any given parliament, along with their ministerial service record. The speech records are machine-readable, and can be processed using software tools discussed below. For our purposes, the speeches are organized by 'parliamentary session', a period with a mean length of around 200 days. The period between two general elections (usually com- prising several sessions, each starting about a year apart) is referred to as a 'parliament'. We obtained dates for the sessions from the usual sources for the period: Cook and Keith (1975) and Butler and Butler (1994). Thus for any given day, we know the identity of the government and opposition parties, and thus any contemporary MPs. In what follows, we will limit our analysis to MPs running in general elections under either a Conservative or Liberal label (as originally demarcated by Craig (1989), Craig (1974) and Walker (1978)), the two parties who actually held ministerial positions during this time and thus for whom the concept of 'Shadow Cabinet' makes most sense.11 As a quick overview of the general trends in our data, consider Figure 1 and Figure 2. In Figure 1 we report the number of government (top panel) and opposition (bottom panel) MPs in our data over time (note that the total membership of the Commons is essentially constant:  around 650–670 MPs).  The solid line in both plots reports (on the right y-axis) the number of speeches in total given by the government and opposition. It can be seen that the number of government MPs is approximately constant over time (at around 350), while the total number of opposition MPs falls slightly (in part owing to other—i.e. non-Conservative, non-Liberal—opposition parties entering the fray).  Meanwhile, the total number of speeches by government MPs rises after around 1880, reaching a peak towards the end of the data of around 16000 speeches for the session. The total number of opposition speeches also in- creases very slightly over time, similarly peaking at the turn of the 20th century. Figure 2 reports (on the left axis) the mean number of words per speech over time, while the right y -axis gives the mean number of speeches per MP during the period under study. We see a secular fall in the average length of speeches from a high of around 600 words to around 200. By contrast the number of the speeches given per MP rises both in mean and variance terms, and reaches a peak at the end of the data. The patterns seen in the plots are evidence of a general tightening of executive control over proceedings in the House of Commons, and the expectation that more speeches would be questions or answers to those questions (Chester and Bowring, 1962). This trend of a focus on government business was joined by powers given to the House in the 1880s (in the face of Irish obstructionism) allowing it to 'close' debate, and Speaker rulings that gave that officer a new ability to control overly long or irrelevant statements in the House. In the face of these changing control structures and totals, our approach below avoids absolute measures of opposition performance, instead comparing within oppositions and then in relative terms with respect to governments.12
Methods
4
While our X , membership of the Shadow Cabinet, is latent, we can observe members making speeches which can inform us about X via our observed variable Z. Each MP also has an observable set of covariates pertaining to their current role in the government (i.e. Y), if they are part of the governing party. Our central concern is understanding which MPs 'lead' debate in parliament. Our strategy trades on the idea that influential individuals will raise concerns, terms, topics and issues which MPs will subsequently talk about in that debate and ones that follow. 13
4.1
Concept and Measurement
One way to approach this measurement problem is to see speeches in the House of Com- mons as analogous to a stream of arriving data the contents of which requires modeling. In computer science, a popular way to examine such streams is to consider their 'bursti- ness', in the sense of Kleinberg (2002). The idea is to model the arrival times at which certain words—considered as a type of event—appear. Words that surge in use suddenly are said to "burst" or to be "bursty", which in practice means that the 'gaps' between seeing the word are becoming shorter and shorter. Depending on the nature of the stream pro- cess, there are different statistical models that may be fit to the data to determine burstiness. When data arrives as a continuous process—rather than as, say, batches every year— Kleinberg (2002) suggests an 'infinite-state model' in which bursts are state transitions in a hidden Markov process. For a given term, we begin with a ‘base rate’ calculated as票, where n is the number of speeches using a particular word and T is the total number of speeches in the session. Thus, if there were a hundred mentions of the term 'boundary', and 10000 speeches, the base rate is α 0 = 1000 = 0.01, corresponding to a mean wait time of σφ = 100 speeches. 14 With the base rate in mind, we ask how the gaps between occurrences of the relevant term are changing as the session unfolds. In particular, the Markov process assumes that when in state i , gap times, x , are exponentially distributed with pdf f(x) = αie-αix where α ; is the rate, such that larger values of α imply smaller expected values on the wait time ( ) until the next event occurs. Setting the estimation problem up continues by making α i = π s i , meaning that the rate is proportional to a quantity s + . In keeping with the original Kleinberg (2002) presentation, s will be fixed at 2 for our purposes but i will be estimated as an integer greater than or equal to one, and will be different at different times for the same word (depending on the state of the system). Put very crudely, the idea is to observe the series of gaps between uses of a term, and then to find values of i that when plugged into the previous formula will fit the data, with respect to the (exponentially distributed) wait times that were seen in practice. To see how this might work, and remembering that s is fixed, suppose we saw wait times of 20, 20, 10, 5, 10 . . .  We can see that the third wait time (10) is half what the second one was (20), implying an increased value of i . Similarly, the fourth wait time (5) is half the previous one, implying that i has increased even further. The fifth wait time (10) suggests that i has declined, since the wait time has doubled. We will ultimately have a se- ries of 'states' that describe our data, which is simply the vector of i values that we estimated. to s¹ to s² to s³ requires at least a halving of the gap (and more than a halving if s were chosen to be larger than 2). Clearly, there will be a great many terms that never exhibit any bursts (they are 'not bursty') because their arrival rate is simply too uniform. Thus if the term 'bill' occurs (exactly) every 100 speeches, then obviously the gaps between observa- tions are not changing. As a result, the term exhibits no bursts in use. This logic potentially extends to any word, no matter how common: e.g. the word 'the' might be used uniformly in every speech, and thus will demonstrate no bursts. In this case, the base rate, which is very high, will be a perfectly adequate model for the data. This process has a second component: a cost term, denoted as γ , which essentially im- poses a penalty when the system seems to move 'up' in intensity in terms of the underlying rate—no cost is imposed for the system to move down in intensity. 15  A larger γ is associated with relatively few upwards transitions. Meanwhile, the exponential component (determined by s) encourages the fitting of a model to the data that reflects the actual sequence of gaps observed. The resulting minimization problem takes both parts into account, and thus at- tempts to fit the data with as few transitions as possible. Note that the bursts in this model are nested: that is, bursts of higher intensity occur within periods of lower intensity activity. As per the original presentation, γ is set to 1 for our work here. 16 Conceived in the usual way, burstiness is a property of streams of events—with one ex- 15More formally, in the original setup, the optimization problem requires finding a sequences of states
ample being words in speeches (or indeed, MPs in debates as we'll explain below). We can, for example, examine the burst pertaining to the word "Ireland" or "boundary", and in Figure 3 we picture the latter of these terms for the 1884 session in which the Redistribution of Seats Act—dealing specifically with the redrawing of districts—was discussed: note the levels (literally, the states of the Markov process—the exponential distributions delivering the gap times—where 0 is the base rate) of the bursts that the word went through, and the varying lengths of the those bursty episodes. In principle, we can do this for every term and every session. Our innovation is now to use these burstiness estimates to compare MPs with each other. To do that, we need a metric that allows us to compute a score for each member taking into account the relative burstiness of their contributions. For us, this is a weighted sum. For each MP, a burst that begins with a speech made by him is scored as the length of that burst (literally, the number of speeches that occur while at the particular value of i ) multiplied by its intensity. All such bursts are then summed and a total score produced. As an example, consider an MP making 100 speeches. Suppose that a word from one of his speeches launches a burst of intensity level 2 for a time period of 30. A different word from the same speech launches a burst of intensity level 3 for a time period of 4. Meanwhile a word from another speech launches a burst of intensity level 3 for a time period of 5. His overall burstiness is thus calculated as (2 × 30) + (3 × 4) + (3 × 5) = 87. Note that bursts are hierarchical: a burst of level n can only occur within a burst of level m , where m < n . A consequence is that MPs cannot be given 'credit' for decreasing the intensity with which a particular word is used rel- ative to the current period in which they are speaking. Note further, that if one MP boosts a term's use to, say, level 2, while a second MP then boosts it further to level 3, the first MP receives 'credit' only for the level 2 burst, while the second receives credit only for the level 3. This is simply a measurement strategy that accords with our notions of MPs building on the points of others, which we believe requires certain oratory skill and which our metric rewards. In terms of preprocessing, we do nothing to our texts except remove punctuation and convert everything to lower case. In particular, we do not remove stop words since their use, if they are indeed stop words in the usual sense, should remain relatively uniform over time and will not be bursty. Nor do we stem the terms, the idea being that we wish to observe particular uses of terms rather than generic concepts that can be spoken in several ways. In Supp Info A, we give some pseudo-code to clarify the algorithm we used.
Validation
4.2
The claim is that our burstiness metric captures some notion of 'agenda setting' by MPs, and 'agenda content' in terms of the words that come up in debate. We now validate our approach by demonstrating that (a) during given periods, the 'right' words are bursty; that (b) for given words, the 'right' sessions show them to be bursty at that time; and that (c) the 'right' individual MPs are bursty at the 'right' times. By "right" in the foregoing sentence, we mean 'in ways that are congruent with our expectations and knowledge of the period'. Beginning with our first validation exercise, consider Table 1. We report three particular sessions—in 1846, 1866 and 1885—and terms that appeared near the top of the burstiness rank order for those periods. We see immediately from the first column that MPs were discussing (in a bursty way) 'wheat' (ranked 3) and 'grain' (ranked 5), during a period when the Corn Laws were under serious discussion. Similarly, just prior to the Second Reform Act of 1867, they raised issues pertaining to the franchise and the earlier 1832 Great Reform Act. In 1885, the time of the controversial Government of Ireland Bill that would have delivered Home Rule to Ireland, we see surges of terms like 'irishmen' and their leader '[P]arnell' along with other terms specific to such discussion. It is worth contrasting the exercise that produced Table 1 relative to the use of 'topic models' in political science (Quinn et al., 2010; Grimmer and Stewart, 2013). In our approach, terms are rewarded if they 'suddenly' appear with relative intensity; in this way, a specific term used consistently in every session such as 'budget' or 'trade' or 'education', would not nec- essarily be bursty. By contrast, a topic model would almost certainly have a topic allocated to, or defined by, such concepts. That is, topic models do a good job of summarizing 'what' was discussed in some general way, while burstiness captures dynamics in which terms were
intensely discussed and that dominated the agenda for spurts of time. 17 Moving on with our validation, we want to see that certain terms are bursty when we expect them to be. Consider Figure 4. There we report four terms with distinct burstiness 'signa- tures' over time. In each case, the y-axis is the burstiness of the word, calculated as its levels multiplied by the durations of those levels.18  This is then rescaled, or standardized, between 0 and 1 within a given session. Thus, as terms approach a burstiness value of 'one' they are the most bursty term that session , the second most bursty term would typically have a score of just shy of one (e.g. 0.98), the third most term just below that and so on. The x -axis labels correspond to the beginnings of the various parliaments (generally following general elections) over the period. The [pink] dots are the transformed scores per session, and the solid [red] lines are lowess curves. In the first panel, we consider the term 'tariff', which was used repeatedly and intensively in two different periods: first, during the 'corn laws' debates of the 1840s, and then at the start of the twentieth century, when Joseph Chamberlain in particular argued for a system of 'imperial preference' for Empire goods (see, e.g, Howe, 1998, for discussion). The term 'zulu' appears high on the parliamentary agenda in the early and mid-1880s—during the exact period that the British were at war with this group—and then disappears. The word 'ireland' is bursty throughout the entire Victorian era, and this seems entirely reasonable given that the 'Irish Question', and Irish MPs, were a constant concern during this time. Finally, in the last panel, we note that the word 'gentlemen' is similarly constantly on the agenda though its burstiness is very low—implying that its use is not especially intense. This makes sense for a generally procedural word that is used fairly consistently over time. As the third part of our validation exercise, we considered the burstiness profile of members of parliament, reporting our results in Figure 5.  William Gladstone and Benjamin Dis- raeli are both bursty during periods that they dominated the Commons (including as Prime Minister). The Irish parliamentary leader and strategist, Charles Parnell appears especially bursty during the 1880s, as expected. Finally, Samuel Plimsoll (MP for Derby) has a small but marked impact after the 1868 election, when he was responsible for pressuring the then government to introduce legislation to mandate waterlines on merchant ships.
4.3
Alternative measures: ‘speechiness’
One concern readers may have is that burstiness is simply a stand-in (i.e.  a proxy) for 'speechiness': that is, our metric measures nothing more than the varying ability or willing- ness of MPs to make speeches. This is not the case: the correlation between the burstiness of opposition MPs and the number of speeches they make varies between 0.19 (1892) and 0.90 (1847) over the sessions as a whole, with a mean of 0.63. These variables are not mea- suring the same thing: while it is true that an MP may be non-bursty because he makes no o 1900 1906 1910 Figure 4: The burstiness profile of different terms over time. The y -axis of the plots is the 'standardized' burstiness of the term, a rescaled metric where a value of 1 corresponds to the most bursty term that session, while a value of 0 refers to the least bursty term. The x-axis labels correspond to the beginnings of the various parliaments over the period. The [pink] dots are the actual standardized scores, and the solid [red] lines are lowess curves. speeches, making lots of speeches is no guarantee of being bursty. In particular, an MP who makes (perhaps thousands of) speeches that are simple responses, or contain terms that are not picked up by others, will not be bursty. 19 More philosophically, another reason to eschew speechiness as a measure is that it over- rewards procedural responsibility and opportunity relative to actual agenda-setting ability. To see this, consider the Speakers of the House of Commons. With power over recognition and organization of debates in the chamber, Speakers make many speeches. On the other hand, since they are non-partisan figures without incentives or opportunities to introduce or discuss policy proposals, we would not expect them to be more bursty than leaders of the government and opposition. 20 To see the problems that insufficient attention to such a dis- tinction can produce, consider Figure 6; there, we compare the speechiness and burstiness of the seven Speakers in our data, in chronological order. On the y -axis we report the quantile position of the Speaker's mean scores—in terms of the number of speeches he made, and his burstiness—relative to all other MPs serving for the same sessions who made at least one speech. We see immediately (top line and points), that every Speaker is close to the maximum of the empirical CDF in terms of speeches, occupying somewhere around the 97th percentile on this measure. By contrast, the burstiness metric (bottom line and points) has the Speaker rightfully downplayed in terms of score—around the 80th percentile, on average. Clearly then, relative to speechiness, burstiness avoids directly reproducing procedural power in favor of a more genuine measure of policy agenda-setting. It is thus a better measurement strategy for our current work, especially for opposition members who have few de jure powers. that, in fact, the existence of the shadow cabinet can be discerned from an inspection of speech records in the House of Commons. One concern might be that perhaps, prior to more widespread democratization, the shadow cabinet made its presence felt in other ways—for example, via electioneering in the constituencies.  We acknowledge this, but would claim that at the very least, our study is of the shadow cabinet in a sense similar to that as in the modern period—that is, as a legislative force.
Results
5
We have established a metric for measuring the agenda-setting ability of individual MPs.  Ul- timately, we want to use it to explore the ways in which the informal institution of interest— i.e. that shadow cabinet members become cabinet members—evolved over time. This re- quires three interrelated steps: first, we need to show how and when the opposition as a whole organized, and collectively paid more attention to agenda control. Second, within that opposition, we need to explore the ways that agenda-power became concentrated in a 'leadership' group. That is, we need to assess whether and when a shadow cabinet could have been said to emerge. Third, given that we have established that the opposition orga- nized, and that they did so under a shadow cabinet, we need to show that the latter became ministers at the exchange of power and that this relationship was non-constant over time.
5.1
Opposition Burstiness over Time
We begin by considering the agenda-setting ability of the opposition, and the way that this changes over time.  Of course, our metric above is 'absolute': it calculates a raw number pertaining to individuals, or groups of individuals, and their ability to raise issues which draw attention in parliament. In practice, this means that burstiness may be generally higher under two conditions: first, when (exogenously) there are more things to be bursty about—e.g. a war occurs, or a famine, or some other event of note; second, when there are more opportunities to talk, since this lengthens the period (in speech terms) when bursts may come to exist.  Given these facts, we consider the burstiness of the opposition relative to the cabinet. In particular, we begin this section by taking the ratio of mean opposition burstiness to mean cabinet burstiness for every session in our data. In the upper panel of Figure 7 we plot that quantity: it appears as the [black] jagged line, that peaks and troughs, moving left to right, reaching its zenith around 1857 (when the cabinet was about 40 times more bursty), and its nadir around 1885 (when the cabinet was about 5 times more bursty). Note that for clarity, we demarcate the x -axis using general election dates for the period. The first observation from the upper panel of Figure 7 is that the cabinet was always more Figure 7: Ratio of (mean) burstiness: cabinet to opposition, cabinet to (Government party) backbenchers.  One change point was found in the opposition ratio time series, marked on the plot with the broken line and mean of ratio given on either side.
Cabinet vs Opposition
bursty than the opposition, on average: notice that the line is never below one. Given the dominance of the cabinet over procedure from the 1830s onwards, this is not per se surpris- ing: ministers have more opportunities to be bursty, and presumably by the very nature of their jobs have more ready access to information that can become bursty (e.g. reports of officials figures or policies). However, moving left to right, we see a generally decreasing ratio: the smooth [red] lowess line makes the point very clear. Put otherwise, the opposition becomes more bursty relative to the cabinet, with the key change apparently happening between about 1870 and 1885. To place this result on a more sound statistical footing, we conducted structural break tests (in the sense of Bai and Perron, 2003). 21  We found one break in the ratio data, dating to the third session of the parliament beginning after the 1880 General Election: in the figure, we present this point as a broken line and note that the mean ratio dropped by over 50%, from 16.91 to 7.25 after the change point. Though we will give a more detailed analysis below, note that this aggregate realignment of agenda power is commensurate with our claim that a set of opposition leaders—i.e. a shadow cabinet—is emerging and making its influence known after suffrage extension. An obvious concern on seeing such a result is that there is nothing 'special' about the op- position: perhaps the cabinet's agenda-setting ability was in secular decline from the 1870s onwards? We can go some way to refuting this suggestion by studying the lower panel of Figure 7, where we consider the (mean) ratio of the cabinet to government backbenchers. Notice that both the underlying ratio, and the smoothed lowess, are essentially constant. We find no breakpoints here using the usual formal tests. Ultimately then, we can conclude that the change in the ratio for the opposition is something specific to that side of the House of Commons, and not a general artefact of changing cabinet roles or priorities at the time.
5.2
Opposition Outliers as a ‘Front Bench’
Having established that the opposition was increasingly aggressive in its agenda-setting just after the Second Reform Act, we next seek the precise mechanics of that change.  That is, we wish to understand exactly how the opposition asserted its control. Recall that one possibility is that it increasingly mimicked the government party's authority structure by establishing an 'executive' core of frontbenchers to set policy and rebuff the cabinet, while a pliant majority of opposition backbenchers formed up behind them. In Figure 8 we examine the evidence for such a claim. In the upper portion of Figure 8, we report boxplots of the burstiness of opposition parties (specifically, the Conservatives and Liberals) over time. The points (circles and squares) de- note outliers, defined in the usual way as points above (and below) 1.5 times the interquartile range of the given session. Note immediately that, in practice, all outliers are in the right tails of their distributions: that is, the median opposition member has a very low burstiness for the entire period (and, indeed, it is close to zero on this measure). Second, we see a surge in the magnitude of the outliers around 1880: indeed, some of the largest burstiness scores are recorded between 1880 and 1892. Formal time-series tests on the means of each session show that there is one break point, demarcated by a broken line during the third session of the parliament meeting in 1880. The standard deviation of the burstiness yields an almost identical finding, albeit the change point corresponds to the second session of 1880. Finally, we report the changing means and standard deviations themselves: prior to the break, we have a mean of 211688.28, while the standard deviation is around seven million.  By contrast, the latter part of the time series has a mean and standard deviation an order of magnitude higher. We conclude that the 'average' burstiness of the opposition was increasing, while simultaneously showing more variance. Given that the floor value of the metric is zero, the implication of the top panel is that some individuals are increasingly 'pulling away' from To make this point clearer, consider the middle panel where we have 'standardized' the measure by session, meaning all MPs fall between zero and a burstiness of one. There, the flat dashes at the bottom of the figure are the medians, while the points are the positions of the 90th percentile members of the opposition (which typically represents an MP ranked around 20th most bursty in his party). The solid [red] line represents a lowess smoothed curve through those points and it can be readily seen that its slope increases in magnitude around the time of the Second Reform Act. Notice that, while there is some noise, the slope is clearly upwards and by the turn of the century the 90th percentile has pulled away from the medians (on average) substantially. To test our intuitions more precisely, the bottom panel of Figure 8 reports the number of opposition outliers over time. Clearly, there is a downward trend: beginning around 70 outliers, the opposition has around 50 outliers by the 1870s, and fewer than 20 by the end of the period. Again, we use a formal structural break test which in this case revealed two breaks: one in the last session of the 1865 parliament, and the second in the first session of the parliament meeting after the 1886 election. In both cases, the mean is reduced. Im- portantly for our purposes, the average number of outliers is reduced to the approximate size—below 20—that we would expect for a 'shadow cabinet' of spokesmen on various issues of governance. To reiterate: here we find that the date of the Second Reform Act (1867) was a crucial transition point for the emergence of a small(er) set of bursty individuals on the opposition benches, congruent with the existence of a cadre of senior MPs in leadership roles. On seeing these plots, readers may be skeptical about our claims regarding the importance of the Second Reform Act given that subsequent sessions—e.g. 1880 and 1902 in the top panel, and the mid-1880s in the bottom panel—are also associated with surges and changes to the opposition's relative burstiness.²²  Two comments are in order: first and foremost, these dates obviously occur after the Second Reform Act and are thus consistent with no- tions that 1867 was crucial to altering the data-generating process in a large and general way. Second, we would reiterate our comments about cautious interpretation of timing: the claim is not of a binary 'switch' in behavior (and thus an outright repudiation of any other plausible causal agent), but rather a continuous change process that receives a fillip after suffrage expansion.
5.3
Burstiness and Future Cabinet status
One way to verify our presumption—that the outliers from Figure 8 are a 'cabinet-in- waiting'—is to show that, in fact, they went on to fill cabinet roles when their party found itself next in government. To examine this possibility, we considered the 14 times that power switched, in the sense that a new party previously in opposition now formed the government, during the period. For the opposition members in each 'switching' session, we pooled the data and regressed their (binary) status as a cabinet member in the next session on their (binary) status as an outlier in the previous period, along with a time indicator, and bursti- ness as a robustness check. Because we have some repeated observations of MPs we cluster standard errors at the MP level. We make no claims that this causally identifies the effect of shadow cabinet membership on subsequent office-holding: even if outlier status were a per- fect measure of shadow cabinet membership, members of the shadow cabinet undoubtedly differ from other MPs in many ways that would be important to prospects for promotion. However, such an analysis can establish whether or not the evidence is broadly consistent with our claims. The relevant part of our results can be seen in Table 2. In Model 1, we use outlier sta- tus and 'session number' since the Great Reform Act—literally, the number of sessions of parliament that have occurred since 1832 (thus the first session is our data is given the value '1', the second is '2' and so on). We see a positive association for both variables: that is, being an outlier helps an MP be promoted next time his party is in office, and, in fact, as time passes he is unconditionally more likely to be promoted. This latter finding is sensible because ministers were increasingly drawn from the Commons rather than the Lords. In Model 2, we try an alternate measure of 'leadership-ness', the number of speeches given by the MP. As we see from the higher AIC, this model does not fit the data as well as the previous one, implying that using the burstiness outlier metric provides useful extra leverage over more traditional alternatives. In Model 3 we add our key interaction term between time and outlier status. As expected and consistent with our claims, the coefficient on being an outlier remains positive and sig- nificant. The coefficient on session number is similarly positive and significant, and is larger in this specification. Importantly the interaction effect is significant, and smaller than the combined effect of being an outlier and the session number.23 Thus the net effect of being an outlier is that one was more likely to be promoted to office as time passed. 24 Notice that this model has a smaller AIC than the previous effort, suggesting it is a better fit to the data. Moreover, a likelihood ratio test favors the model with the interaction. Finally, in Model 4, we add a variable measuring whether or not (1 or 0) the member had previously served in the Cabinet.  As expected, the coefficient is positive, but note that, crucially, being an outlier still matters (our coefficient is statistically significant).  All told, our burstiness outlier metric is sound and helpful: it genuinely measures some notion of being in the shadow cabinet that is not simply captured by the number of speeches given, or having been previously selected as a cabinet minister.25
outlier increases as time goes on does not per se rule out either theory as a possibility. But nor do we intend it to alone. Rather, we consider it part of a series of evidential pieces that add up to a story in which something other than the immediate aftermath of the Great Re- form Act matters: there is no plateau here, and with every passing session the relationship between burstiness and cabinet office becomes stronger. This serves in part to imply that we have successfully identified a good measure of shadow cabinet membership. 26
5.4
Summary
We have three interrelated results: 1. though the cabinet was always more bursty than the opposition, the latter became relatively more assertive in agenda-control terms around the time of the Second Reform Act (1867). 2. within the opposition, 'outliers'—that is, extremely bursty individuals—became fewer in number over time, with marked shifts downwards at the time of the Second Reform Act, and in the mid-1880s. By the turn of the twentieth century, a group of individuals approximately the size of a 'shadow cabinet' (in terms of shadowing the major offices of state) had emerged. 3. the key informal institution of interest—the purported relationship between being in the shadow cabinet, and being in the cabinet when the party in question took power— was present for the entire period, and became increasingly strong over time (in that an outlier in later sessions was more likely to find himself promoted to cabinet office than an outlier in earlier sessions)
6
Discussion
Informal institutions form the core of practical politics in Westminster systems where statute law is often silent: this includes the role of parties (there is none, constitutionally speaking), the role of whipping (never officially acknowledged) and the role of the Prime Minister itself (which has never been formally defined).  This means that scholars of these polities, and Comparative Politics more generally, have a particularly pressing interest in understanding how inferences may be made about these norms and rules, if they are to plot their emer- gence and evolution over time. In this paper, we considered the role of the shadow cabinet as a 'government-in-waiting', a vital organization that ensures citizens an alternative to the present government at election time—even if this does not ultimately mean that the people's will is implemented as policy (Schumpeter, 1942). First, we showed that informal institutions are helpfully modeled with latent variables. Our solution here was to apply new text-as-data methods to almost a million speeches by mem- bers of parliament between the First and Fourth Reform Acts. Using a 'burstiness' metric, we showed that, after the 1870s, an increasingly small group of opposition ‘leaders’ closed the gap in terms of agenda-setting in their partisan competition with the cabinet. Intriguingly, though the cabinet began its characteristic dominance of procedure in the 1830s (Cox, 1987), it was the broadening of suffrage that accelerated the emergence of the shadow cabinet as an institutional force. Our work joins a large literature on the effects of suffrage expansion on political behavior and policy making at Westminster (e.g. Gash, 1952; Adelman, 1997; Rush, 2001; McLean, 2001; Aidt, Daunton and Dutta, 2010; Berlinski and Dewan, 2011), and by moving the focus to the opposition similarly contributes to the study of comparative parliamentary politics (e.g. Doring, 1995; Holzhacker, 2005). Our work has several broader implications. First, we demonstrated an important case in which an (informal) institution arose 'organically' as a counterpoint to a pre-existing organization—the cabinet—when an external stimulus was presented (in our case, a party orientated electorate).  Our work thus joins a literature that deals with 'institutionalism' (see Hall and Taylor, 1996), and the specific mechanisms by which institutions evolve (see Mahoney and Thelen, 2010). Again, we think our measurement strategy is a way to pro- ceed when faced with the task of charting the development of such organizations over time. Second, we took an explicitly 'agenda-setting' approach—a topic of very general interest to political scientists (e.g. Cobb, Ross and Ross, 1976; Pollack, 1997; Krehbiel, 1998; Cox and McCubbins, 2005). Typically measuring the extent to which bodies or individuals have the power to do so is difficult—especially in parliamentary systems where, in day-to-day oper- ations, oppositions lose and governments win. We have gone part way to resolving that issue. First, we This paper raises several interesting questions that we have left unanswered. have not looked at the screening and selection mechanisms by which MPs joined the shadow cabinet: a specific 'career path' focus for the Victorian period, in line with more modern work (e.g. Benedetto and Hix, 2007; Kam, 2009) is called for. Second, our technique allows for helpful (weighted) word-based summaries of debates. Our focus here was on the relative burstiness of sets of individuals, but it would presumably be beneficial to those interested in ideological changes in Westminster legislatures over time (e.g. Schonhardt-Bailey, 2003; Godbout and Hoyland, 2013) to use a metric like ours to get a sense of exactly how—i.e. on what issues—MPs became divided or unified as their parties evolved.  Related to this endeavor, integrating burstiness with debate sequencing, i.e.  who speaks after whom and what it signals about strengthening but informal relationships across the floor over time, is a task worthy of attention. Third, while it is well known that democratization in Britain included moves away from staffing top party and governance roles with members of the House of Lords, unfortunately, we did not have peers' speech records available for similar analysis to that above. It would certainly be intriguing to document any potential shifting burstiness from one chamber to another over time, especially if such changes affected the way that government and opposition interacted in both places. Finally, with the speech records of other legislatures—such as the US Congress (Jensen et al., 2012)—increasingly available online, it would be intriguing to compare the burstiness of terms in a comparative context, to see how different systems converge or diverge in term use over time. We leave such efforts for future work.

Pseudo-code for burstiness calculation
Supp Info A
Let tdm by the term-document matrix of the speeches, such that each row is a word, and each column a speech. A given ith , jth cell-entry of tdm is a binary indicator {0,1} of whether or not word i appeared in speech j (multiple uses are treated similarly to single occurrences). The steps to calculate our statistics are as follows: for (i in 1:number of rows in tdm){ 1. draw the ith row of tdm, which is a binary vector of occurrences. Thus, supposing there were 10 speeches in the corpus, we might have {0,0,1,1,1,1,0,0,0,0}, with the use of some term appearing in documents 3,4,5,6 and then nowhere else. 2. calculate the burstiness of this term, as described above. That is, for each individual burst, multiply its level (via the estimated i ) by its duration (literally, how long this particular value of i lasts until a new value of i arises). Then sum these terms for all bursts that occur for the term. Denote this sum as b. 3. allocate b to the appropriate location in a document vector (that is, a vector of length equal to the number of documents in the corpus). Notice that this will require simply adding it to whatever the 'running total' for that document currently is (since a given document may have multiple bursty terms). 4. record the time of the start of the maximum or 'peak' burst of the ith term, and the time of the end of that burst.  Using a look-up table, record the MP making the speech that began the peak burst. } The result of this algorithm is (a) a table of bursty terms (i.e.  all terms with non-zero burstiness), each with a starting and ending point of their peak burst, and the identity of the MP who began the peak burst; (b) a table listing every speech and the burstiness of each (which will generally be zero for at least some speeches). Finally, a look-up table is used to aggregate the results of the speech table by MP: that is, each speech is mapped to a unique MP, and his score derived by summing the total burstiness of all the speeches he gave (some of which may be zero scored).
Supp Info B
Correlation between burstiness and ‘speech-
iness’ over time
Figure 9 reports the correlation between all MPs' burstiness and the number of speeches they gave, for the various sessions in the data. Notice that the mean is around 0.6, implying that though the variables are correlated, they do not apparently measure identical concepts. For completeness, we also investigated the correlation between an MP's 'speechiness' rank in a given session, and their burstiness rank during the same period with the idea that the relative share of speeches might provide a better measure of importance within the opposition than the raw number of speeches. In reality, the correlation varies between 0.66 and 0.90, with a mean of 0.83. Although this is higher, there is clearly some room left between the measures.
Robustness: using ‘raw’ burstiness in re-
Supp Info C
gression
Recall that, in our account, being an 'leader' (and thus in the shadow cabinet) is associated with being an outlier in burstiness terms. In Table 3 we consider the robustness of this
definition by fitting models using raw burstiness as an alternative measure of leadership (both additionally and as a substitute for outlier status).² ⁷ As we see in Model I, the coefficient on outlier status is essentially unchanged, though a little extra explanatory power is added (AIC is lower). All in all, this suggests that it is being an outlier that 'matters', rather than simply being bursty from the opposition backbenches. Finally, we consider Model II that does not use our outlier status variable at all, and relies solely upon an MP's burstiness, in addition to the time variable and the interaction. Note that this model does a relatively poorer job (in terms of fit) than the variant using the outlier metric we explained above.