High-quality sentence paraphraser using Transformers in NLP | by Ramsri Goutham | Towards Data Science

The input to our program will be any English sentence -
Four private astronauts launched to orbit by Elon Musk’s SpaceX returned to Earth Saturday evening, splashing down into the ocean off the east coast of Florida after a three-day mission.
The output will be paraphrased version of the same sentence. Paraphrasing a sentence means, you create a new sentence that expresses the same meaning using a different choice of words.
After a three-day mission, four private astronauts sent by Elon Musk's SpaceX returned to Earth on Saturday evening, splashing down into the ocean off the east coast of Florida.
From rewriting your old Social Media posts to college essays to augmenting the dataset when you don't have many examples for your text classification model, there are several use-cases for a paraphraser.
As part of building Questgen.ai we created a custom dataset filtered on top of ParaNMT to preserve only diverse high-quality paraphrases.
Diverse here means that pairs of sentences are selected such that there is a significant difference in word order or at least the paraphrased output differs by multiple word changes.
If you prefer an easy-to-use Google Colab Notebook, it can be found at Questgen’s Github Repo.
!pip install transformers==4.10.2!pip install sentencepiece==0.1.96
We will use the pre-trained model uploaded to the HuggingFace Transformers library hub to run the paraphraser. We will use diverse beam search decoding strategy that gives best results for paraphrases output.
More examples can be found in the Google Colab demo mentioned above.
from transformers import AutoTokenizer, AutoModelForSeq2SeqLMmodel = AutoModelForSeq2SeqLM.from_pretrained("ramsrigouthamg/t5-large-paraphraser-diverse-high-quality")tokenizer = AutoTokenizer.from_pretrained("ramsrigouthamg/t5-large-paraphraser-diverse-high-quality")import torchdevice = torch.device("cuda" if torch.cuda.is_available() else "cpu")print ("device ",device)model = model.to(device)# Diverse Beam searchcontext = "Once, a group of frogs was roaming around the forest in search of water."text = "paraphrase: "+context + " 
"encoding = tokenizer.encode_plus(text,max_length =128, padding=True, return_tensors="pt")input_ids,attention_mask  = encoding["input_ids"].to(device), encoding["attention_mask"].to(device)model.eval()diverse_beam_outputs = model.generate(    input_ids=input_ids,attention_mask=attention_mask,    max_length=128,    early_stopping=True,    num_beams=5,    num_beam_groups = 5,    num_return_sequences=5,    diversity_penalty = 0.70)print ("\n\n")print ("Original: ",context)for beam_output in diverse_beam_outputs:    sent = tokenizer.decode(beam_output, skip_special_tokens=True,clean_up_tokenization_spaces=True)    print (sent)
The output from the above code is:
Original: Once, a group of frogs was roaming around the forest in search of water.paraphrasedoutput: A herd of frogs was wandering around the woods in search of water.paraphrasedoutput: A herd of frogs was wandering around the woods in search of water.paraphrasedoutput: A gang of frogs was wandering around the forest in search of water at one time.paraphrasedoutput: A herd of frogs was swaning around the woods in search of water.paraphrasedoutput: A gang of frogs was roaming about the woods in search of water once more.
That’s it! You have a high-quality state-of-the-art sentence paraphraser that you can use in your projects. A word of caution is that the paraphraser output is not always perfect so a human-in-the-loop system might be necessary.
I am running a 4-week cohort-based course on “Practical Introduction to NLP” with Maven, the world’s best platform for cohort-based learning. If you wish to transform yourself from a Python developer to a junior NLP developer with practical project experience in 4 weeks, grab your spot now!
Happy coding!
If you would like to stay connected please feel free to follow me on Twitter and Linkedin