How to get started using transformers for NLP
By: Josh Zwiebel
Tags: Transformers, Jupyter Notebook, Python

This article works best when you can try out the different methods yourself — run my notebook on deepnote.com to try it!

I love the transformers library. It is by far the easiest way to get started using transformers for NLP, which is currently the bleeding edge.

The first step is grabbing the model and the tokenizer from the transformers library

import tensorflow as tf

import tensorflow_hub as hub

from transformers.modeling_tf_openai import TFOpenAIGPTLMHeadModel #this is the GPT transformer with additional layers added for easy language modeling from transformers.tokenization_openai import OpenAIGPTTokenizer import simpletransformers

Don’t worry if the following cell takes some time! The model just needs a minute to download.

model = TFOpenAIGPTLMHeadModel.from_pretrained("openai-gpt") tokenizer = OpenAIGPTTokenizer.from_pretrained("openai-gpt")

Note: GPT2 is a newer transformer model for text generation, but when I had written this code only GPT was supported for text generation

prompt_text = "<Think of a piece of text to supply as a prompt!>" encoded_prompt = tokenizer.encode(prompt_text,add_special_tokens=False,return_tensors='tf')

Since transformers supplies a tokenizer for the GPT model, this was a much easier solution than using the universal sentence encoder. The tensors return as type pytorch as opposed to tensorflow because I find that I run into fewer bugs when that is the case (this might be a result that the library was originally written for pytorch).

num_sequences = 1 # this is the number of different sequences/ sentences the generator will create given the prompt length = 15 # if you want to strictly follow the prompt for this question set the length to one. If you want to see how creative GPT can be feel free to amend

Let’s get some sequences!

generated_sequences = model.generate(

input_ids=encoded_prompt,

do_sample=True,

max_length=length + len(encoded_prompt[0]),

temperature=1.0,

top_k=5,

top_p=0.9,

repetition_penalty=1.0)

There is a lot to unpack in that cell. Credit where credit is due- the library has fantastic documentation, so let us unpack.

Input ids is simply specifying what the input will be.

Do_sample — this prevents the model from just picking the most likely word at every step greedily.

Max_len — what is the length of sequence we want?

Temperature — I find this one really interesting. This is a measure of how risky our model will be when picking words. Feel free to tweak this!

Top_k and top_p are similar in that they limit the number of words the model considers when decoding before randomly sampling from the word probabilities.

The repetition penalty is meant to avoid sentences that repeat themselves without anything really interesting.

Kudos to both the docs and this excellent medium post for helping me understand the nitty gritty here.

for sequence in generated_sequences:

for sequence in generated_sequences: text = tokenizer.decode(sequence, clean_up_tokenization_spaces=True) print(text)

And tada! We have predicted text. However, that is not the only way to generate text. By far the simplest way is with a pipeline.