Exploring the UN General Debates with Dynamic Topic Models | by Luke Lefebure | Towards Data Science

In this article, I introduce the intuition behind Dynamic Topic Models and demonstrate this method’s ability to discover evolving narratives in unstructured text through an application to a corpus of speeches.
By reading, I hope you will pick up an understanding of Dynamic Topic Models, the problem that they try to solve, practical tips for working through a project with text data, and some insights about modern world affairs! If you would like to follow along with the code, you can find that here.
Each fall, world leaders gather in New York to participate in the General Debate of the United Nations General Assembly. This year’s session concluded earlier this month after discussion around issues such as nuclear weapons and the war in Syria.
Speeches given at the General Debate form a historical record of the issues that have commanded the attention of the international community. Hidden in this text data are insights about how the dialogue changes over time, and Dynamic Topic Models can help us quantitatively model that evolution.
Topic modeling is a generic term used to describe the process of finding topics in a corpus of unstructured text. A popular method for this is Latent Dirichlet Allocation (LDA) which is a generative model that learns a predefined number of latent topics, where each topic is represented as a distribution over terms and each document as a distribution over topics. The article below does a good job at explaining the idea, and the Wikipedia page is also a good reference.
medium.com
LDA is a very powerful technique for the qualitative analysis of large corpora because of its highly interpretable topics, and it is also useful for dimensionality reduction as it transforms sparse document-term matrices into fixed low dimensional document-topic matrices. However, LDA ignores the temporal aspect present in many document collections.
Consider a topic about US politics found by LDA in a corpus of news from the past decade. You might expect the most probable terms to be “congress”, “election”, “vote”, etc. However, the model treats a document from 2010 the same as one from 2018 with respect to the term probabilities, so it is unable to learn that the term “trump”, for example, was almost nonexistent in the context of politics until relatively recently.
Dynamic Topic Models (DTMs), as presented by Blei and Lafferty here, address this problem by extending the idea of LDA to allow topic representations to evolve over fixed time intervals such as years. Specifically, the documents within each time slice are modeled with a topic model of the same dimension, and each topic in time slice t evolves from a corresponding topic in time slice t-1.
The generative process of LDA remains more or less the same aside from the key difference that the overall topic distribution and the term distribution for each topic differ depending on the time slice. In particular, the parameters for these distributions “evolve” at each time slice by being drawn from distributions centered around the corresponding values from the previous time slice.
The end result of this is a series of LDA-like topic models that are sequentially tied together. A topic learned by a DTM is thus a sequence of related distributions over terms. With this intuition, I move now to its application.
A corpus consisting of speeches at the General Debate from 1970 through 2015 is hosted on Kaggle (7,507 in total). The dataset was originally released last year (see here for the paper) by researchers in the UK and Ireland who used it to study the position of different countries on various policy dimensions.
Each speech is tagged with the year and session it was given and the ISO Alpha-3 code of the country that the speaker represented.
A key observation in this dataset is that each of these speeches consists of discussion on a multitude of topics. If every speech contains discussion on poverty and terrorism, a topic model trained on entire speeches as documents in a bag of words representation will have no way of understanding that terms like “poverty” and “terrorism” should be representative of different topics.
To counter this problem, I tokenize each speech into paragraphs and treat each paragraph as a separate document for the analysis. A simple rule based approach that looks for sentences separated by a newline character performs reasonably well on the task of paragraph tokenization for this dataset. After this step, the number of documents jumps from 7,507 (full speeches) to 283,593 (paragraphs).
After expanding each speech into multiple documents, I word tokenize, normalize each term by lowercasing and lemmatizing, and trim low frequency terms from the vocabulary. The end result is a vocabulary of 7,054 terms and a bag of words representation for each document that can be used as input to the DTM.
To determine the number of topics to use, I ran LDA on a few different time slices individually with different numbers of topics (10, 15, 20, 30) to get a feel for the problem. Through manual inspection, 15 seemed to produce the most interpretable topics, so that is what I settled on for the DTM. More experimentation and rigorous quantitative evaluation could certainly improve this.
Using gensim's Python wrapper to the original DTM C++ code, inferring the parameters of a DTM is then straightforward, albeit slow. Inference took about 8 hours on an n1-standard-2 (2 vCPUs, 7.5 GB memory) instance on Google Cloud Platform. However, I ran on a single core, so this time can probably be cut down if you can get the parallelized version of the original C++ to compile.
The model discovered very interpretable topics, and I examine a few in depth here. Specifically, I show for a few topics the top terms at a sample of time slices as well as plots of probabilities of notable terms over time. A complete list of the topics discovered by the model and their top terms can be found in the appendix at the end of this article.
The Preamble of the UN Charter states:
We the peoples of the United Nations determined ... to reaffirm faith in fundamental human rights, in the dignity and worth of the human person, in the equal rights of men and women and of nations large and small.
It is no surprise then that human rights is a perennial topic at the General Debate and one that the model was able to discover. Despite the notion of gender equality appearing in the charter quoted above, the model shows that it still took quite some time for the terms “woman” and “gender” to really catch on. Also, note the rising use of “humankind” coupled with the decline of the use of “mankind”.
Ever since the creation of modern Israel in 1948, there has been a constant debate in which the UN has been intimately involved around Arab-Israeli conflict, borders, settlements, and the treatment of the Palestinian people. In fact, there is an entire Wikipedia page devoted to the matter.
The model shows an increasing focus on Gaza after its blockade by Israel and Egypt in 2007 and a constant discussion around Israeli settlements, a practice the UN has deemed illegal under international law. The rise of the term “two” is interesting, likely coming from growing discussion around a “two-state solution”.
The discussion on Africa has evolved from its colonial past to independence, the era of apartheid, and finally the birth of new governments, elections, and democracy.
The Cold War era brought an arms race between the US and the Soviet Union and subsequent discussion on disarmament. More recently, the focus has been on Iran.
The Millennium Development Goals and the Sustainable Development Goals are UN initiatives introduced in 2000 and 2015 respectively to focus on solutions to issues such as poverty, education, and gender equality. This topic seems to have evolved from generic discussion around international development towards these structured programs.
This exercise demonstrated how DTMs represent topics and this method’s incredible ability to discover narratives in unstructured text. Remember, the model did all of the above without any human intelligence!
I could potentially improve the methods used by experimenting with a larger vocabulary size, different numbers of topics and evaluation thereof, phrase extraction (e.g. “human_rights” could be one token instead of having “human” and “rights” separately), and different stemmers or lemmatizers. Other methods that go beyond the bag of words representation and use word embeddings could be interesting to explore as well.
One aspect of DTMs not leveraged here (but one that I would like to explore further) is the learned vector representations for documents. These are not only useful for computing similarity values between documents but also for finding representative documents for each topic. For example, it would be interesting to get all of the speeches from US representatives, pick out the most representative paragraphs for each time slice for, say, the human rights topic, and use those paragraphs to construct a thread on the evolution on US discourse on human rights.
Overall, this turned out to be a fascinating exercise. If you would like to experiment yourself or dive deeper into the implementation details, all of the code for this analysis can again be found in the repo here. Thanks for following along, and if you have additional ideas or questions, please leave a comment!
Below is a complete list of the topics discovered by the DTM. For each topic, the top ten terms from the 1975, 1995, and 2015 time slices are shown.
Topic 0            1975           1995           20150       security        council        council1            non       security       security2          state         member         member3        council          state         reform4         cyprus        general         united5  international         reform         nation6      principle         nation      permanent7         nation         united          state8        aligned  international           must9        charter           must  internationalTopic 1            1975           1995           20150            sea  international          state1            law          state         island2          state         island          small3          latin           drug  international4        america        country        country5       resource          small           sids6       american        america          crime7        country      caribbean      caribbean8     conference     convention     developing9  international          crime        pacificTopic 2          1975           1995           20150      nuclear        nuclear         weapon1  disarmament         weapon        nuclear2       weapon         treaty           iran3          arm    disarmament         treaty4        state  proliferation            arm5       soviet            non      agreement6       treaty            arm  international7        power          state            use8        union           test            non9         race  international       chemicalTopic 3            1975           1995           20150          right          right          right1          human          human          human2         people        respect            law3  international      democracy          woman4        freedom         people         people5      principle  international        respect6        justice            law        freedom7      political        freedom           rule8        respect      principle  international9           must        society          peaceTopic 4            1975           1995           20150       economic       economic       economic1  international    development        country2          world  international          world3        country          world         global4            new            new  international5          order         social         social6    development      political    development7       relation        country      political8      political         global         crisis9             co    cooperation            newTopic 5          1975         1995           20150         east        peace          state1       middle         east          peace2         arab       middle       solution3       israel   resolution    palestinian4        peace       people       conflict5  palestinian  palestinian  international6       people       israel          syria7        right      process         people8    territory       region           east9    palestine    agreement         israelTopic 6         1975          1995           20150    republic        united         united1      people        nation         nation2      united         peace  international3       state     operation          peace4      nation       keeping       conflict5  government      republic           role6       korea      conflict   peacekeeping7    peaceful        bosnia        mission8        viet  organization        support9     country   herzegovina       republicTopic 7        1975         1995           20150  operation  cooperation         region1         co       region    cooperation2      peace     regional  international3    country     european        country4     europe      country       security5   security       europe       regional6   relation        union       european7   european        state      terrorism8     region     security          union9    detente    stability         effortTopic 8           1975        1995        20150        africa      africa     african1         south     african     country2       african     country      people3        people  government  government4        regime       peace    republic5      southern   community      africa6       namibia      people   political7     apartheid  democratic   community8  independence     process  democratic9        racist      effort       peaceTopic 9        1975      1995       20150      world       war    climate1          u     world     change2        one  conflict      world3        war      cold     global4    problem      many     threat5       time         u  challenge6  situation     peace       must7      peace       one    country8      power   problem     crisis9   conflict      must   conflictTopic 10           1975        1995       20150        people      people     people1      struggle     country        war2       country     refugee       year3  independence        year      world4      national       state    country5         force       force        one6    liberation         war  terrorist7         state  government      today8   imperialist         one      state9   imperialism        life       manyTopic 11            1975           1995       20150         nation         nation     nation1         united         united     united2   organization   organization      world3          world          world          u4          peace           must       must5         member  international  challenge6  international              u     future7        charter           role      peace8           role          peace       year9          state            new     globalTopic 12          1975         1995         20150      country      country  development1   developing  development      country2  development     economic         goal3     economic   developing  sustainable4        world        trade          per5    developed     resource       agenda6     resource      economy         cent7        trade    financial      poverty8        price    programme       health9   assistance   assistance    educationTopic 13         1975       1995       20150          mr    general    general1     general    session   assembly2   president   assembly    session3    assembly         mr  president4     session  president         mr5  delegation  secretary       like6        also       also      would7        like       like  secretary8        wish   election       also9    election       wish      sixtyTopic 14         1975         1995         20150     session   conference  development1    assembly  development         20152     general      general           703  conference         year       agenda4  resolution       agenda         year5        year     assembly  sustainable6     special      meeting   conference7  government       summit           pv8      united         held           159     meeting         last         post