Analyzing Sentiment of City Council Meetings
By: Aaron Abrahamson
Tags: NLP, Sentiment Analysis, City Council, Hackathons, Seattle

Analyzing Sentiment of City Council Meetings

Image by Kevin Norris from Pixabay

Last weekend I participated in my first hackathon: St. Hack-Trick’s Day 2020, put on by the great people at DemocracyLab. It was a really enriching experience and I got to work on a really cool project!

The Council Data Project’s purpose is to empower citizens and journalists by making city council activities discoverable and understandable by more than just legislators. It’s currently live for Seattle here. Seattle does not provide text transcripts of their meetings (only video records) and the Council Data Project automatically creates text transcripts, indexes them, and then makes them available online. I’ve been wanting to explore sentiment analysis and thought this was a perfect opportunity to do so!

Sentiment analysis is “contextual mining of text which identifies and extracts subjective information in source material.”¹ Specifically, for our purposes, I wanted to explore the sentiment of each speaker, and see if changes in sentiment, or overall high/low periods of sentiment, indicated if there were interesting parts or periods of debate amongst speakers.

In its current form, the text transcript data is unlabeled, meaning that the data is just text with no classifiers attached to it. This limits what models we can use to unsupervised models. I identified two models to try — NLTK’s VADER and TextBlob. There is also IBM’s Watson, but that has a price tag and would not scale well. In this example, I am just going to use VADER, but I would like to try it out with TextBlog soon. VADER is pre-trained on social media, so it is not the perfect tool for this scenario.

Let’s look at the transcripts! First, we need to import all of the modules we will use and download the data from CDP:

import pandas as pd

import nltk

from nltk.sentiment.vader import SentimentIntensityAnalyzer

from cdptools import CDPInstance, configs

import altair as alt

import numpy as np

import json # connect to Council Data Project database

seattle = CDPInstance(configs.SEATTLE) # Download a specific event, save the JSON

manifest = seattle.get_transcript_manifest() found_event = manifest.loc[manifest.event_id == "bb35a74c-53db-40f7-9af0-c0b296c2696a"].iloc[0] save_path = seattle.file_store.download_file(found_event.filename) # Load transcript JSON file

with open(save_path, "r") as read_in:

transcript = json.load(read_in)

for s in transcript["data"][:2]:

print(s) {'speaker': '', 'data': [{'start_time': 12.679, 'end_time': 20.186, 'text': 'Good morning, and welcome to the February 11, 2020, meeting of the public safety and human services committee.'}, {'start_time': 20.186, 'end_time': 41.808, 'text': "It's 9:36A.M., and I am Lisa Herbold, the chair of this Committee and Councilmember to District 1. I call the meeting to order, welcome to my Cocommittee members, council member Morales and Councilmember Lewis."}, {'start_time': 41.808, 'end_time': 45.178, 'text': 'I want to go over the agenda, quickly.'}, {'start_time': 45.178, 'end_time': 47.914, 'text': 'Today we will start with approval of agenda.'}, {'start_time': 47.914, 'end_time': 86.453, 'text': "Followed by public comment and the items of business include, appoint of two members of the community police Commission, hearing from the office of the inspector general about her 2020 work plan, and also a presentation from the human services Department that will include a focus on the Hsd, the human services director's report, and then a focus exclusively on the Nonhomelessness investments side of the human services side of the portfolio."}, {'start_time': 86.453, 'end_time': 92.225, 'text': 'If there are no objections, I would like to approve the agenda.'}]}

Each time someone speaks it lists each sentence they say, with start and stop times. It currently does not know who is actually speaking (which is why speaker is an empty string). I wrote a function to parse this JSON and store each speaker and their sentences in a list of lists. Now let’s look at that sentiment!

# Download vader lexicon, and instatiate a SentimentIntensityAnalyzer object nltk.download('vader_lexicon')

sid = SentimentIntensityAnalyzer() def get_vader_score(sent: str) -> list:

# takes in a string and then outputs VADER sentiment score

ss = sid.polarity_scores(sent)

out_list = []

for k in sorted(ss):

out_list.append([k, ss[k]])

return out_list # Let's look at the eighth speaker's first sentence

get_vader_score(transcript[7][0]) > [['compound', 0.6486], ['neg', 0.0], ['neu', 0.361], ['pos', 0.639]]

The scores can range from -1 to 1, indicating the polarity for that score. 1 being positive, -1 being negative (just to be clear). So for my purposes, we really just need the compound score, so I rewrote the above function to just output the compound score. Here is an in-depth article on how VADER computes the scores if you are interested in learning more.

I originally computed the sentiment for each sentence, the resulting graph was very noisy.

VADER Compound score for each sentence in a city council meeting

Next, I plotted the average sentiment of each speaker’s sentences. This proved to be much more useful. Please note the visualizations are from separate meetings.

Average VADER Compound score for each speaker in a city council meeting

Seeing that huge downward spike got us so excited — this was it! Now let’s look at the speaker block that provided that result:

“And you can get bullied from landlords. I’ve gotten bullied from my landlord. We’ve gotten bullied from our landlord.”

The result makes sense, though, considering how the model operates. Bullying is bad (and it was mentioned three times) so the model will reflect that, but it is not really useful for our purposes.

Conclusion

Overall the sentiment is always high, in general, when you look at many meetings. This makes sense, as council members are usually cordial and professional and parliamentary procedure enforces this.

VADER proved to not be too useful for our purposes because of this. I’d like to try TextBlob’s sentiment analysis package, but I am not hopeful that it will be any more useful. A model not exposed to the unique language of city council meetings will not be able to properly analyze the language.

Going forward, the next steps I see for exploring this would be to manually label transcripts for points of argument or debate and then use supervised learning techniques. Wish me luck!

Sources: