Hugging Face Hub documentation
We are helping the community work together towards the goal of advancing Artificial Intelligence 🔥.
Not one company, even the Tech Titans, will be able to “solve AI” by themselves – the only way we'll achieve this is by sharing knowledge and resources. On the Hugging Face Hub we are building the largest collection of models, datasets and metrics in order to democratize and advance AI for everyone 🚀. The Hugging Face Hub works as a central place where anyone can share and explore models and datasets.
The Hugging Face Hub hosts Git-based repositories which are storage spaces that can contain all your files 💾.
These repositories have multiple advantages over other hosting solutions:
- commit history and diffs
On top of that, Hugging Face Hub repositories have many other advantages:
- Repos provide useful metadata about their tasks, languages, metrics, etc.
- Anyone can play with the model directly in the browser!
- Training metrics charts are displayed if the repository contains TensorBoard traces.
- An API is provided to use the models in production settings.
- Over 10 frameworks such as 🤗 Transformers, Asteroid and ESPnet support using models from the Hugging Face Hub.
Many repos have a widget that allows anyone to do inference directly on the browser!
Here are some examples:
- Named Entity Recognition using spaCy.
- Image Classification using 🤗 Transformers
- Text to Speech using ESPnet.
- Sentence Similarity using Sentence Transformers.
You can try out all the widgets here.
The Inference API allows you to send HTTP requests to models in the Hugging Face Hub. The Inference API is 2x to 10x faster than the widgets! ⚡⚡
To determine which pipeline and widget to display (
translation, etc.), we analyze information in the repo such as the metadata provided in the model card and configuration files. This information is mapped to a single
pipeline_tag. We choose to expose only one widget per model for simplicity.
For most use cases, the model type is determined from the tags. For example, if there is
tag: text-classification in the metadata, the inferred
pipeline_tag will be
🤗 Transformers however, the model type is determined automatically from
config.json. The architecture can be used to determine the type: for example,
AutoModelForTokenClassification corresponds to
token-classification. If you're really interested in this, you can see pseudo-code in this gist.
You can always manually override your pipeline type with pipeline_tag: xxx in your model card metadata.
You can find all the supported tasks here.
Here are some with links to examples:
text-classification, for instance
token-classification, for instance
question-answering, for instance
translation, for instance
summarization, for instance
conversational, for instance
text-generation, for instance
fill-mask, for instance
zero-shot-classification(implemented on top of a nli
text-classificationmodel), for instance
table-question-answering, for instance
sentence-similarity, for instance
You have access to the repos as with any other Git-based repository! You can even upload very large files.
Here is a video to learn more about it from our course!