Transformers documentation

Community

Hugging Face's logo
Join the Hugging Face community

and get access to the augmented documentation experience

to get started

Community

This page regroups resources around 🤗 Transformers developed by the community.

Community resources:

Resource Description Author
Hugging Face Transformers Glossary Flashcards A set of flashcards based on the Transformers Docs Glossary that has been put into a form which can be easily learnt/revised using Anki an open source, cross platform app specifically designed for long term knowledge retention. See this Introductory video on how to use the flashcards. Darigov Research

Community notebooks:

Notebook Description Author
Fine-tune a pre-trained Transformer to generate lyrics How to generate lyrics in the style of your favorite artist by fine-tuning a GPT-2 model Aleksey Korshuk Open In Colab
Train T5 in Tensorflow 2 How to train T5 for any task using Tensorflow 2. This notebook demonstrates a Question & Answer task implemented in Tensorflow 2 using SQUAD Muhammad Harris Open In Colab
Train T5 on TPU How to train T5 on SQUAD with Transformers and Nlp Suraj Patil Open In Colab
Fine-tune T5 for Classification and Multiple Choice How to fine-tune T5 for classification and multiple choice tasks using a text-to-text format with PyTorch Lightning Suraj Patil Open In Colab
Fine-tune DialoGPT on New Datasets and Languages How to fine-tune the DialoGPT model on a new dataset for open-dialog conversational chatbots Nathan Cooper Open In Colab
Long Sequence Modeling with Reformer How to train on sequences as long as 500,000 tokens with Reformer Patrick von Platen Open In Colab
Fine-tune BART for Summarization How to fine-tune BART for summarization with fastai using blurr Wayde Gilliam Open In Colab
Fine-tune a pre-trained Transformer on anyone’s tweets How to generate tweets in the style of your favorite Twitter account by fine-tuning a GPT-2 model Boris Dayma Open In Colab
Optimize 🤗 Hugging Face models with Weights & Biases A complete tutorial showcasing W&B integration with Hugging Face Boris Dayma Open In Colab
Pretrain Longformer How to build a “long” version of existing pretrained models Iz Beltagy Open In Colab
Fine-tune Longformer for QA How to fine-tune longformer model for QA task Suraj Patil Open In Colab
Evaluate Model with 🤗nlp How to evaluate longformer on TriviaQA with nlp Patrick von Platen Open In Colab
Fine-tune T5 for Sentiment Span Extraction How to fine-tune T5 for sentiment span extraction using a text-to-text format with PyTorch Lightning Lorenzo Ampil Open In Colab
Fine-tune DistilBert for Multiclass Classification How to fine-tune DistilBert for multiclass classification with PyTorch Abhishek Kumar Mishra Open In Colab
Fine-tune BERT for Multi-label Classification How to fine-tune BERT for multi-label classification using PyTorch Abhishek Kumar Mishra Open In Colab
Fine-tune T5 for Summarization How to fine-tune T5 for summarization in PyTorch and track experiments with WandB Abhishek Kumar Mishra Open In Colab
Speed up Fine-Tuning in Transformers with Dynamic Padding / Bucketing How to speed up fine-tuning by a factor of 2 using dynamic padding / bucketing Michael Benesty Open In Colab
Pretrain Reformer for Masked Language Modeling How to train a Reformer model with bi-directional self-attention layers Patrick von Platen Open In Colab
Expand and Fine Tune Sci-BERT How to increase vocabulary of a pretrained SciBERT model from AllenAI on the CORD dataset and pipeline it. Tanmay Thakur Open In Colab
Fine Tune BlenderBotSmall for Summarization using the Trainer API How to fine tune BlenderBotSmall for summarization on a custom dataset, using the Trainer API. Tanmay Thakur Open In Colab
Fine-tune Electra and interpret with Integrated Gradients How to fine-tune Electra for sentiment analysis and interpret predictions with Captum Integrated Gradients Eliza Szczechla Open In Colab
fine-tune a non-English GPT-2 Model with Trainer class How to fine-tune a non-English GPT-2 Model with Trainer class Philipp Schmid Open In Colab
Fine-tune a DistilBERT Model for Multi Label Classification task How to fine-tune a DistilBERT Model for Multi Label Classification task Dhaval Taunk Open In Colab
Fine-tune ALBERT for sentence-pair classification How to fine-tune an ALBERT model or another BERT-based model for the sentence-pair classification task Nadir El Manouzi Open In Colab
Fine-tune Roberta for sentiment analysis How to fine-tune a Roberta model for sentiment analysis Dhaval Taunk Open In Colab
Evaluating Question Generation Models How accurate are the answers to questions generated by your seq2seq transformer model? Pascal Zoleko Open In Colab
Classify text with DistilBERT and Tensorflow How to fine-tune DistilBERT for text classification in TensorFlow Peter Bayerle Open In Colab
Leverage BERT for Encoder-Decoder Summarization on CNN/Dailymail How to warm-start a EncoderDecoderModel with a bert-base-uncased checkpoint for summarization on CNN/Dailymail Patrick von Platen Open In Colab
Leverage RoBERTa for Encoder-Decoder Summarization on BBC XSum How to warm-start a shared EncoderDecoderModel with a roberta-base checkpoint for summarization on BBC/XSum Patrick von Platen Open In Colab
Fine-tune TAPAS on Sequential Question Answering (SQA) How to fine-tune TapasForQuestionAnswering with a tapas-base checkpoint on the Sequential Question Answering (SQA) dataset Niels Rogge Open In Colab
Evaluate TAPAS on Table Fact Checking (TabFact) How to evaluate a fine-tuned TapasForSequenceClassification with a tapas-base-finetuned-tabfact checkpoint using a combination of the 🤗 datasets and 🤗 transformers libraries Niels Rogge Open In Colab
Fine-tuning mBART for translation How to fine-tune mBART using Seq2SeqTrainer for Hindi to English translation Vasudev Gupta Open In Colab
Fine-tune LayoutLM on FUNSD (a form understanding dataset) How to fine-tune LayoutLMForTokenClassification on the FUNSD dataset for information extraction from scanned documents Niels Rogge Open In Colab
Fine-Tune DistilGPT2 and Generate Text How to fine-tune DistilGPT2 and generate text Aakash Tripathi Open In Colab
Fine-Tune LED on up to 8K tokens How to fine-tune LED on pubmed for long-range summarization Patrick von Platen Open In Colab
Evaluate LED on Arxiv How to effectively evaluate LED on long-range summarization Patrick von Platen Open In Colab
Fine-tune LayoutLM on RVL-CDIP (a document image classification dataset) How to fine-tune LayoutLMForSequenceClassification on the RVL-CDIP dataset for scanned document classification Niels Rogge Open In Colab
Wav2Vec2 CTC decoding with GPT2 adjustment How to decode CTC sequence with language model adjustment Eric Lam Open In Colab
Fine-tune BART for summarization in two languages with Trainer class How to fine-tune BART for summarization in two languages with Trainer class Eliza Szczechla Open In Colab
Evaluate Big Bird on Trivia QA How to evaluate BigBird on long document question answering on Trivia QA Patrick von Platen Open In Colab
Create video captions using Wav2Vec2 How to create YouTube captions from any video by transcribing the audio with Wav2Vec Niklas Muennighoff Open In Colab
Fine-tune the Vision Transformer on CIFAR-10 using PyTorch Lightning How to fine-tune the Vision Transformer (ViT) on CIFAR-10 using HuggingFace Transformers, Datasets and PyTorch Lightning Niels Rogge Open In Colab
Fine-tune the Vision Transformer on CIFAR-10 using the 🤗 Trainer How to fine-tune the Vision Transformer (ViT) on CIFAR-10 using HuggingFace Transformers, Datasets and the 🤗 Trainer Niels Rogge Open In Colab
Evaluate LUKE on Open Entity, an entity typing dataset How to evaluate LukeForEntityClassification on the Open Entity dataset Ikuya Yamada Open In Colab
Evaluate LUKE on TACRED, a relation extraction dataset How to evaluate LukeForEntityPairClassification on the TACRED dataset Ikuya Yamada Open In Colab
Evaluate LUKE on CoNLL-2003, an important NER benchmark How to evaluate LukeForEntitySpanClassification on the CoNLL-2003 dataset Ikuya Yamada Open In Colab
Evaluate BigBird-Pegasus on PubMed dataset How to evaluate BigBirdPegasusForConditionalGeneration on PubMed dataset Vasudev Gupta Open In Colab
Speech Emotion Classification with Wav2Vec2 How to leverage a pretrained Wav2Vec2 model for Emotion Classification on the MEGA dataset Mehrdad Farahani Open In Colab
Detect objects in an image with DETR How to use a trained DetrForObjectDetection model to detect objects in an image and visualize attention Niels Rogge Open In Colab
Fine-tune DETR on a custom object detection dataset How to fine-tune DetrForObjectDetection on a custom object detection dataset Niels Rogge Open In Colab