Many model repos have a widget that allows anyone to run inferences directly in the browser!
Here are some examples:
- Named Entity Recognition using spaCy.
- Image Classification using 🤗 Transformers
- Text to Speech using ESPnet.
- Sentence Similarity using Sentence Transformers.
You can try out all the widgets here.
A widget is automatically created for your model when you upload it to the Hub. To determine which pipeline and widget to display (
translation, etc.), we analyze information in the repo, such as the metadata provided in the model card and configuration files. This information is mapped to a single
pipeline_tag. We choose to expose only one widget per model for simplicity.
For most use cases, we determine the model type from the tags. For example, if there is
tag: text-classification in the model card metadata, the inferred
pipeline_tag will be
For some libraries, such as 🤗
Transformers, the model type might be inferred automatically based from configuration files (
config.json). The architecture can determine the type: for example,
AutoModelForTokenClassification corresponds to
token-classification. If you’re interested in this, you can see pseudo-code in this gist.
You can always manually override your pipeline type with
pipeline_tag: xxx in your model card metadata.
You can specify the widget input in the model card metadata section:
widget: - text: "Jens Peter Hansen kommer fra Danmark"
You can provide more than one example input. In the examples dropdown menu of the widget, they will appear as
Example 2, etc. Optionally, you can supply
example_title as well.
widget: - text: "Is this review positive or negative? Review: Best cast iron skillet you will ever buy." example_title: "Sentiment analysis" - text: "Barack Obama nominated Hilary Clinton as his secretary of state on Monday. He chose her because she had ..." example_title: "Coreference resolution" - text: "On a shelf, there are five books: a gray book, a red book, a purple book, a blue book, and a black book ..." example_title: "Logic puzzles" - text: "The two men running to become New York City's next mayor will face off in their first debate Wednesday night ..." example_title: "Reading comprehension"
Moreover, you can specify non-text example inputs in the model card metadata. Refer here for a complete list of sample input formats for all widget types. For vision & audio widget types, provide example inputs with
src rather than
For example, allow users to choose from two sample audio files for automatic speech recognition tasks by:
widget: - src: https://example.org/somewhere/speech_samples/sample1.flac example_title: Speech sample 1 - src: https://example.org/somewhere/speech_samples/sample2.flac example_title: Speech sample 2
Note that you can also include example files in your model repository and use them as:
widget: - src: https://huggingface.co/username/model_repo/resolve/main/sample1.flac example_title: Custom Speech Sample 1
We provide example inputs for some languages and most widget types in the DefaultWidget.ts file. If some examples are missing, we welcome PRs from the community to add them!
You can find all the supported tasks here.
Here are some links to examples:
text-classification, for instance
token-classification, for instance
question-answering, for instance
translation, for instance
summarization, for instance
conversational, for instance
text-generation, for instance
fill-mask, for instance
zero-shot-classification(implemented on top of a nli
text-classificationmodel), for instance
table-question-answering, for instance
sentence-similarity, for instance
Generally, the Inference API for a model uses the default pipeline settings associated with each task. But if you’d like to change the pipeline’s default settings and specify additional inference parameters, you can configure the parameters directly through the model card metadata. Refer here for some of the most commonly used parameters associated with each task.
For example, if you want to specify an aggregation strategy for a NER task in the widget:
inference: parameters: aggregation_strategy: "none"
Or if you’d like to change the temperature for a summarization task in the widget:
inference: parameters: temperature: 0.7
The Inference API allows you to send HTTP requests to models in the Hugging Face Hub, and it’s 2x to 10x faster than the widgets! ⚡⚡ Learn more about it by reading the Inference API documentation.