Course documentation

Basic usage completed!

Hugging Face's logo
Join the Hugging Face community

and get access to the augmented documentation experience

to get started

Basic usage completed!

Great job following the course up to here! To recap, in this chapter you:

  • Learned the basic building blocks of a Transformer model.
  • Learned what makes up a tokenization pipeline.
  • Saw how to use a Transformer model in practice.
  • Learned how to leverage a tokenizer to convert text to tensors that are understandable by the model.
  • Set up a tokenizer and a model together to get from text to predictions.
  • Learned the limitations of input IDs, and learned about attention masks.
  • Played around with versatile and configurable tokenizer methods.

From now on, you should be able to freely navigate the 🤗 Transformers docs: the vocabulary will sound familiar, and you’ve already seen the methods that you’ll use the majority of the time.