Text Generation with Pretrained GPT2 Using PyTorch | by Raymond Cheng | Towards Data Science

Text Generation is one of the most exciting applications of Natural Language Processing (NLP) in recent years. Most of us have probably heard of GPT-3, a powerful language model that can possibly generate close to human-level texts. However, models like these are extremely difficult to train because of their heavy size, so pretrained models are usually preferred where applicable.
In this article, we will teach you how to generate text using pretrained GPT-2, the lighter predecessor of GPT-3. We will be using the notable Transformers library developed by Huggingface. If you want to know how to fine-tune GPT-2 on your own custom dataset to generate domain-specific text, then you can refer to my previous post:
towardsdatascience.com
If using pretrained GPT-2 is enough, you’re in the right place! Without further ado, let’s get started with the tutorial!
Step 1: Install Library
Step 2: Import Library
Step 3: Build Text Generation Pipeline
Step 4: Define the Text to Start Generating From
Step 5: Start Generating
BONUS: Generate Text in any Language
To install Huggingface Transformers, we need to make sure PyTorch is installed. If you haven’t installed PyTorch, go to its official website and follow its instructions to install it.
After installing PyTorch, you can install Huggingface Transformers by running:
pip install transformers
After successfully installing Transformers, you can now import its pipeline module:
from transformers import pipeline
The pipeline module is an abstraction layer that takes away the complexity of code and allows an easy way of performing different NLP tasks.
Now, we can start building the pipeline for text generation. We can do so by:
text_generation = pipeline(“text-generation”)
The default model for the text generation pipeline is GPT-2, the most popular decoder-based transformer model for language generation.
Now, we can start defining the prefix text we want to generate from. Let’s give it a more general starting sentence:
The world is
prefix_text = "The world is"
After we define our starting text, now it’s time to do generation! We can simply do so by running:
generated_text= text_generation(prefix_text, max_length=50, do_sample=False)[0]print(generated_text[‘generated_text’])
The above code specifies a max_length of 50 tokens and sampling to be off. The output should be:
The world is a better place if you’re a good person.
I’m not saying that you should be a bad person. I’m saying that you should be a good person.
I’m not saying that you should be a bad
As we can see it’s quite fascinating how a computer can be able to generate text that kind of makes sense although it’s not perfect. One issue with the output is that it is repetitive at the end. This can possibly be solved by using different decoding schemes (e.g. top-k/top-p sampling) and playing around with different values, but it is out of the scope of this article. To learn more about decoding schemes and how you can implement it, check out the Huggingface’s official tutorial and the TextGeneration pipeline documentation.
First, to generate text in another language, we will need a language model previously trained on a corpus of that language; otherwise; we will have to fine-tune by ourselves, which is a tedious task. Fortunately, Huggingface provides a list of models that are released by the warm NLP community (link here), and chances are that a language model is previously fine-tuned on the language of your choice.
Let’s say we want to generate text in Chinese. This GPT2 model by CKIPLab is pretrained on a Chinese corpus, and so we can use their model without needing to fine-tune by ourselves.
Following their documentation, we can start by importing the relevant tokenizer and model modules:
from transformers import BertTokenizerFast, AutoModelWithLMHead
Then, we can build the tokenizer and the model accordingly:
tokenizer = BertTokenizerFast.from_pretrained(‘bert-base-chinese’)model = AutoModelWithLMHead.from_pretrained(‘ckiplab/gpt2-base-chinese’)
Next, we feed our new tokenizer and model as parameters to instantiate the pipeline:
text_generation = pipeline(“text-generation”, model=model, tokenizer=tokenizer)
After that, we define our prefix text again, this time in Chinese:
我 想 要 去
prefix_text = "我 想 要 去"## I want to go 
Using the same code as above, we can now generate from the prefix text:
generated_text= text_generation(prefix_text, max_length=50, do_sample=False)[0]print(generated_text['generated_text'])
Now, you should see the following output:
我 想 要 去 看 看 。 」 他 說 : 「 我 們 不 能 說, 我 們 不 能 說, 我 們 不 能 說, 我 們 不 能 說, 我 們 不 能 說, 我 們 不 能 說, 我 們
## I want to go see around.” He says: “We can’t say, we can’t say, we can’t say, we can’t say, we can’t say, we can’t say, we
Although the generated text is far from perfect, again, this is a topic for another article.
And here you have it! Hope you now know how to implement text generation using the simple API interface provided by Huggingface with pretrained models. For your convenience, I have attached a Jupyter notebook here:
from transformers import pipeline

text_generation = pipeline("text-generation")

prefix_text = "The world is"

generated_text= text_generation(prefix_text, max_length=50, do_sample=False)[0]
print(generated_text['generated_text'])

Setting `pad_token_id` to `eos_token_id`:50256 for open-end generation.

The world is a better place if you're a good person.

I'm not saying that you should be a bad person. I'm saying that you should be a good person.

I'm not saying that you should be a bad

from transformers import pipeline, BertTokenizerFast, AutoModelWithLMHead

tokenizer = BertTokenizerFast.from_pretrained('bert-base-chinese')
model = AutoModelWithLMHead.from_pretrained('ckiplab/gpt2-base-chinese')

text_generation = pipeline("text-generation", model=model, tokenizer=tokenizer)

prefix_text = "我 想 要 去"

generated_text= text_generation(prefix_text, max_length=50, do_sample=False)[0]
print(generated_text['generated_text'])

Setting `pad_token_id` to `eos_token_id`:102 for open-end generation.

我 想 要 去 看 看 。 」 他 說 : 「 我 們 不 能 說, 我 們 不 能 說, 我 們 不 能 說, 我 們 不 能 說, 我 們 不 能 說, 我 們 不 能 說, 我 們

And that’s it! Hope you enjoyed this post. If you have any questions, feel free to comment down below. Also, please subscribe to my email list to receive new articles from me. You can also check out my previous posts if interested :)
towardsdatascience.com
towardsdatascience.com
towardsdatascience.com
towardsdatascience.com
towardsdatascience.com
Brown, Tom B., et al. “Language models are few-shot learners.” arXiv preprint arXiv:2005.14165 (2020).
Radford, Alec, et al. “Language models are unsupervised multitask learners.” OpenAI blog 1.8 (2019): 9.
Transformers Github, Huggingface
Transformers Official Documentation, Huggingface
Pytorch Official Website, Facebook AI Research
Fan, Angela, Mike Lewis, and Yann Dauphin. “Hierarchical neural story generation.” arXiv preprint arXiv:1805.04833 (2018).
Welleck, Sean, et al. “Neural text generation with unlikelihood training.” arXiv preprint arXiv:1908.04319 (2019).
CKIPLab Transformers Github, Chinese Knowlege and Information Processing at the Institute of Information Science and the Institute of Linguistics of Academia Sinica