NLP Techniques with Shakespeare’s Plays
By: Kyle Aguilar
Tags: Classification, Machine Learning, Deep Learning, Data Science, NLP

Photo by Matt Riches on Unsplash

NLP Techniques with Shakespeare’s Plays

Cleaning and Classifying Text with the Bard

Given the vast quantities of text data being generated today, Natural Language Processing is one of the most interesting aspects of data science, while also being perhaps one of the more finicky. Text data can be difficult to wrangle, complicated by language/dialect, punctuation, time period, and more. However, it is an incredibly commonly-used tool — who goes through their workday without encountering autocomplete or spell check? For my project, I wanted to explore how NLP techniques can be used to process text for classification — specifically, author identification.

My project as outlined here explores one of the most interesting questions in literature, which is the authorship of Shakespeare’s plays. A majority of scholars reject any challenge to his authorship, although a small number of voices insist that he did not have the knowledge of the royal court, understanding of continental European geography, or the education to write the plays, and propose that they could have been written by another Elizabethan playwright such as Christopher Marlowe, or even by Sir Francis Bacon. My objective is to create a classification model for distinguishing his plays, and to explore the ways of visualizing these texts.

The Data

My entire dataset consists of 39 Shakespeare plays (this includes the partial attributions of the Two Noble Kinsmen and Edward III) and 50 non-Shakespeare plays. The chart will also give you a better sense of the lengths of the plays in each class by number of words with stopwords removed, and how much each class contributes to the corpus (about the same, although there are several much longer non-Shakespeare plays). All of these were downloaded as .txt files from Project Gutenberg. The non-Shakespeare plays were all written by other English Renaissance playwrights, such as Ben Jonson, Thomas Middleton, John Fletcher and Francis Beaumont who worked in the same time period, the late 1500s through early 1600s. Plays performed on the English Renaissance stage were infrequently published, so the plays that exist by Shakespeare’s peers are few and far between. There are a few limitations with this data. For one, our sample size of Shakespeare’s entire corpus of plays means I only have 39 data points for our target class. Additionally, the non-Shakespeare plays also suffer from their own authorship issues, and a number of them have been published under Shakespeare’s name, and are listed under Project Gutenberg as such. Finally, there is a comparative lack of editing and spelling modifications for the non-Shakespeare plays, and some words may be spelled differently across plays.

Before proceeding with data cleaning I randomly split the data into training/test folders at random, resulting in 71 texts in the training set and 18 texts in the test set (20%). I am not dealing with a significant class imbalance here — our target comprises 43% of our data.

Cleaning the Data

Below is an example of a .txt snippet as it was uploaded. There are in many cases a high volume of notes both preceding and following the text that I wanted to erase — some editor’s notes, essays describing the work and its publishing history, footnotes divulging various spellings and line differences by editions, and legal language inserted by Project Gutenberg describing limitations on distribution. Every play had differently formatted notes of this sort, so I edited the texts to start with the dramatis personae, and end before the **End of Project Gutenberg** line that many(but not all) texts contain. To make absolutely sure that I was removing any intervening or ending notes I passed words for removal (ex. ‘Shakespeare’, ‘Gutenberg’, ‘electronic’) to strip any lingering unwanted lines.

'That I neither feel how she should be loved, nor know how she

',

'should be worthy, is the opinion that fire cannot melt out of me: I will

',

'die in it at the stake.

',

I next removed stopwords and punctuation, including extra Elizabethan-era stopwords supplied by Bryan Bumgardner and available here. I then split sentences to use a Parts-of-Speech Tagger in order to lemmatize each word. Each lemmatized play was then tokenized to create TF-IDF representations of the data for use with the classification models. TF-IDF (term frequency-inverse document frequency) is a commonly-used vectorized representation of all words in a corpus, showing how important/rare a word is to the corpus by describing it’s frequency both within a single text and the number of documents containing it. The resulting shape of this transformation is a 71 x 41,361 matrix. I also reduced the size of these vectors by creating bigram representations of the data (for example, “but, soft”, “soft, what”, “what, light”, “light, from”); I also used a dimensionality-reduced TF-IDF using scikit-learn’s Truncated SVD, which performs linear dimensionality reduction on matrix representations of text, and can work with sparse matrices efficiently. The resulting reduced matrix retains 80% of the explained variance.

Exploratory Data Analysis

Although I had no other features besides the texts themselves, there are a number of ways of exploring the differences in our classes. It is likely that Shakespeare’s most common words were in all likelihood very different from his peers. You can see below that the 20 most common tokens in our two classes for our training data are quite separate, with even less overlap than I had been expecting. Perhaps based on this, one can infer that Shakespeare was writing more historical plays (or at least, plays involving courts) than his peers.

Although I am working with 41,361 “features” in the full TF-IDF, I used using t-distributed Stochastic Neighbor Embedding (t-SNE) to visualize this high-dimensional data in two dimensions and view how our classes are distributed in relation to one another as below. There seems to be some overlap between the areas our classes represent, but this held promise for my ability to classify.

t-SNE for Unigrams of Training Data

Another way to use t-SNE is to create a Word2Vec model, a two-layer neural network that produces a matrix for every word in the corpus, and generates similarities based on how likely it is that words will co-occur. I used a very wide window of 20 (distance from each word) to determine co-occurrence, and created a vector for each word with size 100 for visualization of each class, and a min_count of 200 words (words that occur fewer than 200 times are ignored). We can see the resulting visualization below using all of the texts in our training set, and use these vectors to pull most similar words based on our text. For example, the 5 words most similar to love are ‘heart’, ’mine’, ‘thought’, ‘fair’ and ‘desire’.

While it is interesting to see how T-SNE visualizations express the similarities within our data, it is important to note that as a dimensionality-reduction algorithm that different runs produce different results based on the perplexity and other hyperparameters set in our Word2Vec model, so we can continue to explore how these graphs best express data of this size. For further reading on t-SNE and plotting classes and

Classification Models

I used the three aforementioned TF-IDF representations to build the below models, as well as a word-embedded data set using GloVe weights. GloVe (Global Vectors for Word Representation) is a Stanford-built NLP method containing pre-trained weights in order to generate a data representation of our corpus. I used Grid Search to tune the hyperparameters for both the Random Forest and the XGBoost, and used a Pipeline to pass the GloVe embeddings to both XGBoost and the RNN.

Random Forest

Naive Bayes (both Multinomial and Complement)

Support Vector Machine (with a Linear Kernel)

XGBoost

LSTM Recurrent Neural Network

The most successful models were XGBoost using the full TF-IDF representation of our unigrams, and a Recurrent Neural Network using the pretrained GloVe weights, which both achieved a 100% accuracy on both our testing and training data, followed by Random Forest with a Grid Search using the full TF-IDF, which achieved a 94% accuracy, and an F1-Score of 94% on our target class. Initial concerns based on the XGBoost with word embeddings that the GloVe weights were not suitable for my data were disproven using the RNN. You can see the outcome of these models below, although many other models were either discarded due to underperformance similar to that of the Random Forest with the reduced-dimensionality TF-IDF, which tended to perform poorly in a number of my models, and the RNN, which was tuned repeatedly to ensure it was not overfit to my training/validation data.

Topic Modeling

Last but not least, I used Gensim’s Latent Dirichlet Assocation tools to explore possibilities for performing unsupervised learning — in this case, topic modeling. See below for an example of this outcome using all of the texts in both classes together using Gensim’s pyLDAvis, which provides an interactive dashboard for the LDA model. This topic pulled ‘king’ as the most important token for this topic, not surprising since it also pulls in character names from a number of political/historical plays (Edward III, King Lear, the Henriad). On the right we also able to tune λ/relevance, allowing us to adjust the tradeoff between probability for the term to occur in each topic, and lift, which is the ratio of the term within the topic to its occurrence elsewhere in the corpus.

Notably, with changes to the number of topics between 3 and 30, my coherence score never achieved above .36, so although these topics are interesting in terms of associating certain characters together, these are not high-quality interpretable topics, and there is some overlap even with our 7 topics.

Conclusion

These texts provided an exciting way to explore NLP techniques for text cleaning, visualization, classification modeling and topic modeling. Authorship attribution is just one way of handling text data, and these techniques are applicable to everything from tagging of fake news/spam, identifying writing-style similarities throughout literature, or catching plagiarism. Even with a smaller corpus, this process’ transformations and resulting datasets resulted in 100% accuracy on our data using not one, but two models. Looking ahead, there are a number of ways to expand this project — for example, expanding our non-target class as new texts continue to be added to Project Gutenberg, or perhaps even creating a recommendation system for Shakespeare lovers to find new reading materials.

The full code for this project is available on Github.

Additional Sources:

Syed, Shaheen, and Marco Spruit. “Full-Text or Abstract? Examining Topic Coherence Scores Using Latent Dirichlet Allocation.” 2017 IEEE International Conference on Data Science and Advanced Analytics (DSAA), 2017, doi:10.1109/dsaa.2017.61.

“Topic Modeling in Python with Gensim.” Machine Learning Plus, 16 Apr. 2020, www.machinelearningplus.com/nlp/topic-modeling-gensim-python.