# **4. A new measure of policy volatility**

The results presented above show that LDA can detect changes in political discourse that reflect broader shifts in the political context and objectives of a country. They also illustrated visually that the stability of this discourse—captured by the topic evolutions discovered by LDA—varies by country. To construct our measure of policy volatility, we exploit this variation in topic prevalence over time and across countries. In other words, we transform the visual representation of the topic evolutions presented earlier into one unique indicator of policy stability.

The underlying assumption behind our measure is that the content of presidential speeches reflects the policy objectives and priorities of their respective governments. Consequently, one might expect that in countries where presidents have tended to provide stability to the policies of their predecessors, with relatively few disruptions to the policy agendas of preceding years, the thematic content of presidential speeches will also tend to remain relatively constant. Conversely, in countries where the policy areas prioritized by the government have not remained consistent over time, one might expect that the content of presidential speeches will be characterized by greater volatility. From this set of assumptions, we can construct different measures of policy stability based on how the thematic content of presidential speeches has evolved over time. Specifically, in countries with greater policy stability, the topics covered in presidential speeches are expected to remain relatively constant, whereas in countries with low policy stability, the topics discussed are likely to be more erratic.

Given its ability to extract the topics and topic composition of documents, LDA provides an adequate method to measure how “continuous” the content of presidential speeches has remained over time. Our measures of policy volatility are based on the topic proportions of each speech (θ), which give an estimate of the proportion of the speech that discusses each topic. As a first step in constructing these measures, we perform a process of “depuration” by removing those topics for which the topic proportion did not exceed 10% in any speech. The assumption is that if the prevalence of a topic never exceeds 10% of the speech for any year, that topic does not convey a strong signal of the policy priorities of the speech and thus has limited value from an analytical point of view.

The first of our measures—the baseline policy volatility measure—is the one we use in the main exposition of our results, while the other alternative measures are meant to act as robustness checks. For each country, the baseline policy volatility measure consists of the percent change between 1940 and 2010 of the share of the three topics that were most prevalent in the year 1940. In other words, we identify the three topics with the highest topic proportion (θ) in the speeches of 1940, and compute how prevalent these three topics are in the speeches of 1940 and 2010 by calculating the sum of their topic proportions. We then take the percent change of these sums. The underlying concept is that in countries with low policy volatility, the content of the speeches—which we take as a reflection of policy—should remain relatively constant over time, and so the difference between the topics discussed in 2010 and 1940 should be smaller in countries with lower policy volatility.

16