Community

This page regroups resources around 🤗 Transformers developed by the community.

Community resources:

Community notebooks:

Notebook	Description	Author
Fine-tune a pre-trained Transformer to generate lyrics	How to generate lyrics in the style of your favorite artist by fine-tuning a GPT-2 model	Aleksey Korshuk
Train T5 in Tensorflow 2	How to train T5 for any task using Tensorflow 2. This notebook demonstrates a Question & Answer task implemented in Tensorflow 2 using SQUAD	Muhammad Harris
Train T5 on TPU	How to train T5 on SQUAD with Transformers and Nlp	Suraj Patil
Fine-tune T5 for Classification and Multiple Choice	How to fine-tune T5 for classification and multiple choice tasks using a text-to-text format with PyTorch Lightning	Suraj Patil
Fine-tune DialoGPT on New Datasets and Languages	How to fine-tune the DialoGPT model on a new dataset for open-dialog conversational chatbots	Nathan Cooper
Long Sequence Modeling with Reformer	How to train on sequences as long as 500,000 tokens with Reformer	Patrick von Platen
Fine-tune BART for Summarization	How to fine-tune BART for summarization with fastai using blurr	Wayde Gilliam
Fine-tune a pre-trained Transformer on anyone’s tweets	How to generate tweets in the style of your favorite Twitter account by fine-tuning a GPT-2 model	Boris Dayma
Optimize 🤗 Hugging Face models with Weights & Biases	A complete tutorial showcasing W&B integration with Hugging Face	Boris Dayma
Pretrain Longformer	How to build a “long” version of existing pretrained models	Iz Beltagy
Fine-tune Longformer for QA	How to fine-tune longformer model for QA task	Suraj Patil
Evaluate Model with 🤗nlp	How to evaluate longformer on TriviaQA with nlp	Patrick von Platen
Fine-tune T5 for Sentiment Span Extraction	How to fine-tune T5 for sentiment span extraction using a text-to-text format with PyTorch Lightning	Lorenzo Ampil
Fine-tune DistilBert for Multiclass Classification	How to fine-tune DistilBert for multiclass classification with PyTorch	Abhishek Kumar Mishra
Fine-tune BERT for Multi-label Classification	How to fine-tune BERT for multi-label classification using PyTorch	Abhishek Kumar Mishra
Fine-tune T5 for Summarization	How to fine-tune T5 for summarization in PyTorch and track experiments with WandB	Abhishek Kumar Mishra
Speed up Fine-Tuning in Transformers with Dynamic Padding / Bucketing	How to speed up fine-tuning by a factor of 2 using dynamic padding / bucketing	Michael Benesty
Pretrain Reformer for Masked Language Modeling	How to train a Reformer model with bi-directional self-attention layers	Patrick von Platen
Expand and Fine Tune Sci-BERT	How to increase vocabulary of a pretrained SciBERT model from AllenAI on the CORD dataset and pipeline it.	Tanmay Thakur
Fine Tune BlenderBotSmall for Summarization using the Trainer API	How to fine tune BlenderBotSmall for summarization on a custom dataset, using the Trainer API.	Tanmay Thakur
Fine-tune Electra and interpret with Integrated Gradients	How to fine-tune Electra for sentiment analysis and interpret predictions with Captum Integrated Gradients	Eliza Szczechla
fine-tune a non-English GPT-2 Model with Trainer class	How to fine-tune a non-English GPT-2 Model with Trainer class	Philipp Schmid
Fine-tune a DistilBERT Model for Multi Label Classification task	How to fine-tune a DistilBERT Model for Multi Label Classification task	Dhaval Taunk
Fine-tune ALBERT for sentence-pair classification	How to fine-tune an ALBERT model or another BERT-based model for the sentence-pair classification task	Nadir El Manouzi
Fine-tune Roberta for sentiment analysis	How to fine-tune a Roberta model for sentiment analysis	Dhaval Taunk
Evaluating Question Generation Models	How accurate are the answers to questions generated by your seq2seq transformer model?	Pascal Zoleko
Classify text with DistilBERT and Tensorflow	How to fine-tune DistilBERT for text classification in TensorFlow	Peter Bayerle
Leverage BERT for Encoder-Decoder Summarization on CNN/Dailymail	How to warm-start a EncoderDecoderModel with a bert-base-uncased checkpoint for summarization on CNN/Dailymail	Patrick von Platen
Leverage RoBERTa for Encoder-Decoder Summarization on BBC XSum	How to warm-start a shared EncoderDecoderModel with a roberta-base checkpoint for summarization on BBC/XSum	Patrick von Platen
Fine-tune TAPAS on Sequential Question Answering (SQA)	How to fine-tune TapasForQuestionAnswering with a tapas-base checkpoint on the Sequential Question Answering (SQA) dataset	Niels Rogge
Evaluate TAPAS on Table Fact Checking (TabFact)	How to evaluate a fine-tuned TapasForSequenceClassification with a tapas-base-finetuned-tabfact checkpoint using a combination of the 🤗 datasets and 🤗 transformers libraries	Niels Rogge
Fine-tuning mBART for translation	How to fine-tune mBART using Seq2SeqTrainer for Hindi to English translation	Vasudev Gupta
Fine-tune LayoutLM on FUNSD (a form understanding dataset)	How to fine-tune LayoutLMForTokenClassification on the FUNSD dataset for information extraction from scanned documents	Niels Rogge
Fine-Tune DistilGPT2 and Generate Text	How to fine-tune DistilGPT2 and generate text	Aakash Tripathi
Fine-Tune LED on up to 8K tokens	How to fine-tune LED on pubmed for long-range summarization	Patrick von Platen
Evaluate LED on Arxiv	How to effectively evaluate LED on long-range summarization	Patrick von Platen
Fine-tune LayoutLM on RVL-CDIP (a document image classification dataset)	How to fine-tune LayoutLMForSequenceClassification on the RVL-CDIP dataset for scanned document classification	Niels Rogge
Wav2Vec2 CTC decoding with GPT2 adjustment	How to decode CTC sequence with language model adjustment	Eric Lam
Fine-tune BART for summarization in two languages with Trainer class	How to fine-tune BART for summarization in two languages with Trainer class	Eliza Szczechla
Evaluate Big Bird on Trivia QA	How to evaluate BigBird on long document question answering on Trivia QA	Patrick von Platen
Create video captions using Wav2Vec2	How to create YouTube captions from any video by transcribing the audio with Wav2Vec	Niklas Muennighoff
Fine-tune the Vision Transformer on CIFAR-10 using PyTorch Lightning	How to fine-tune the Vision Transformer (ViT) on CIFAR-10 using HuggingFace Transformers, Datasets and PyTorch Lightning	Niels Rogge
Fine-tune the Vision Transformer on CIFAR-10 using the 🤗 Trainer	How to fine-tune the Vision Transformer (ViT) on CIFAR-10 using HuggingFace Transformers, Datasets and the 🤗 Trainer	Niels Rogge
Evaluate LUKE on Open Entity, an entity typing dataset	How to evaluate LukeForEntityClassification on the Open Entity dataset	Ikuya Yamada
Evaluate LUKE on TACRED, a relation extraction dataset	How to evaluate LukeForEntityPairClassification on the TACRED dataset	Ikuya Yamada
Evaluate LUKE on CoNLL-2003, an important NER benchmark	How to evaluate LukeForEntitySpanClassification on the CoNLL-2003 dataset	Ikuya Yamada
Evaluate BigBird-Pegasus on PubMed dataset	How to evaluate BigBirdPegasusForConditionalGeneration on PubMed dataset	Vasudev Gupta
Speech Emotion Classification with Wav2Vec2	How to leverage a pretrained Wav2Vec2 model for Emotion Classification on the MEGA dataset	Mehrdad Farahani
Detect objects in an image with DETR	How to use a trained DetrForObjectDetection model to detect objects in an image and visualize attention	Niels Rogge
Fine-tune DETR on a custom object detection dataset	How to fine-tune DetrForObjectDetection on a custom object detection dataset	Niels Rogge