OpenAI GPT-2: Understanding Language Generation through Visualization
By: Jesse Vig
Tags: Machine Learning, NLP, Artificial Intelligence, Deep Learning, Data Visualization

OpenAI GPT-2: Understanding Language Generation through Visualization

How the super-sized language model is able to finish your thoughts.

In the eyes of most NLP researchers, 2018 was a year of great technological advancement, with new pre-trained NLP models shattering records on tasks ranging from sentiment analysis to question answering.

But for others, 2018 was the year that NLP ruined Sesame Street forever.

First came ELMo (Embeddings from Language Models) and then BERT (Bidirectional Encoder Representations from Transformers), and now BigBird sits atop the GLUE leaderboard. My own thinking has been so corrupted by this naming convention that when I hear “I’ve been playing with Bert,” for example, the image that pops into my head is not of the fuzzy unibrowed conehead from my childhood, but instead is something like this:

I can’t unsee that one, Illustrated BERT!

I ask you — if Sesame Street isn’t safe from NLP model branding, what is?

But there was one model that left my childhood memories intact, an algorithm that remained nameless and faceless, referred to by its authors from OpenAI simply as “the language model” or “our method.” Only when authors of another paper needed to compare their model to this nameless creation was it deemed worthy of a moniker. And it wasn’t ERNIE or GroVeR or CookieMonster; the name described exactly what the algorithm was, and no more: GPT, the Generative Pre-trained Transformer.

But in the same breath that it was given its name, GPT was unceremoniously knocked off the GLUE leaderboard by BERT. One reason for GPT’s downfall was that it was pre-trained using traditional language modeling, i.e., predicting the next word in a sentence. In contrast, BERT was pre-trained using masked language modeling, which is more of a fill-in-the-blanks exercise: guessing missing (“masked”) words given the words that came before and after. This bidirectional architecture enabled BERT to learn richer representations and ultimately perform better across NLP benchmarks.

So in late 2018, it seemed that OpenAI’s GPT would be forever known to history as that generically-named, quaintly-unidirectional predecessor to BERT.

But 2019 has told a different story. It turns out that the unidirectional architecture that led to GPT’s downfall in 2018 gave it powers to do something that BERT never could (or at least wasn’t designed for): write stories about talking unicorns:

From https://blog.openai.com/better-language-models/ . Edited for length.

You see, left-to-right language modeling is more than just a pre-training exercise; it also enables a very practical task: language generation. If you can predict the next word in a sentence, you can predict the word after that, and the next one after that, and pretty soon you have… a lot of words. And if your language modeling is good enough, these words will form meaningful sentences, and the sentences will form coherent paragraphs, and these paragraphs will form, well, just about anything you want.

And on February 14, 2019, the OpenAI’s language model did get good enough — good enough to write stories of talking unicorns, generate fake news, and write anti-recycling manifestos. It was even given a new name: GPT-2.

So what was the secret to GPT-2's human-like writing abilities? There were no fundamental algorithmic breakthroughs; this was a feat of scaling up. GPT-2 has a whopping 1.5 billion parameters (10X more than the original GPT) and is trained on the text from 8 million websites.

How does one make sense of a model with 1.5 billion parameters? Let’s see if visualization can help.

Visualizing GPT-2

OpenAI did not release the full GPT-2 model due to concerns of malicious use, but they did release a smaller version equivalent in size to the original GPT (117 M parameters), trained on the new, larger dataset. Although not as powerful as the large model, the smaller version still has some language generation chops. Let’s see if visualization can help us better understand this model.

Note: Code for creating these visualizations can be found on GitHub.

An Illustrative Example

Let’s see how the GPT-2 small model finishes this sentence:

The dog on the ship ran

Here’s what the model generated:

The dog on the ship ran off, and the dog was found by the crew.

Seems pretty reasonable, right? Now let’s tweak the example slightly by changing dog to motor and see what the model generates:

The motor on the ship ran

And now the completed sentence:

The motor on the ship ran at a speed of about 100 miles per hour.

By changing that one word at the beginning of the sentence, we’ve got a completely different outcome. The model seems to understand that the type of running a dog does is completely different from the type that a motor does.

How does GPT-2 know to pay such close attention to dog vs motor, especially since these words occur earlier in the sentence? Well, the GPT-2 is based on the Transformer, which is an attention model — it learns to focus attention on the previous words that are the most relevant to the task at hand: predicting the next word in the sentence.

Let’s see where GPT-2 focuses attention for “The dog on the ship ran”:

The lines, read left-to-right, show where the model pays attention when guessing the next word in the sentence (color intensity represents the attention strength). So, when guessing the next word after ran, the model pays close attention to dog in this case. This makes sense, because knowing who or what is doing the running is crucial to guessing what comes next.

In linguistics terminology, the model is focusing on the head of the noun phrase the dog on the ship. There are many other linguistic properties that GPT-2 captures as well, because the above attention pattern is just one of the 144 attention patterns in the model. GPT-2 has 12 layers, each with 12 independent attention mechanisms, called “heads”; the result is 12 x 12 = 144 distinct attention patterns. Here we visualize all of them, highlighting the one we just looked at: