Pipelines

The pipelines are a great and easy way to use models for inference. These pipelines are objects that abstract most of the complex code from the library, offering a simple API dedicated to several tasks, including Named Entity Recognition, Masked Language Modeling, Sentiment Analysis, Feature Extraction and Question Answering. See the task summary for examples of use.

There are two categories of pipeline abstractions to be aware about:

The pipeline abstraction

The pipeline abstraction is a wrapper around all the other available pipelines. It is instantiated as any other pipeline but requires an additional argument which is the task.

Simple call on one item:

>>> pipe = pipeline("text-classification")
>>> pipe("This restaurant is awesome")
[{'label': 'POSITIVE', 'score': 0.9998743534088135}]

To call a pipeline on many items, you can either call with a list.

>>> pipe = pipeline("text-classification")
>>> pipe(["This restaurant is awesome", "This restaurant is aweful"])
[{'label': 'POSITIVE', 'score': 0.9998743534088135},
 {'label': 'NEGATIVE', 'score': 0.9996669292449951}]

To iterate of full datasets it is recommended to use a dataset directly. This means you don’t need to allocate the whole dataset at once, nor do you need to do batching yourself. This should work just as fast as custom loops on GPU. If it doesn’t don’t hesitate to create an issue.

pipe = pipeline("automatic-speech-recognition", model="facebook/wav2vec2-base-960h", device=0)
dataset = datasets.load_dataset("superb", name="asr", split="test")

# KeyDataset (only `pt`) will simply return the item in the dict returned by the dataset item
# as we're not interested in the `target` part of the dataset.
for out in tqdm.tqdm(pipe(KeyDataset(dataset, "file"))):
    print(out)
    # {"text": "NUMBER TEN FRESH NELLY IS WAITING ON YOU GOOD NIGHT HUSBAND"}
    # {"text": ....}
    # ....
transformers.pipeline(task: str, model: Optional = None, config: Optional[Union[str, transformers.configuration_utils.PretrainedConfig]] = None, tokenizer: Optional[Union[str, transformers.tokenization_utils.PreTrainedTokenizer]] = None, feature_extractor: Optional[Union[str, SequenceFeatureExtractor]] = None, framework: Optional[str] = None, revision: Optional[str] = None, use_fast: bool = True, use_auth_token: Optional[Union[bool, str]] = None, model_kwargs: Dict[str, Any] = {}, **kwargs) → transformers.pipelines.base.Pipeline[source]

Utility factory method to build a Pipeline.

Pipelines are made of:

  • A tokenizer in charge of mapping raw textual input to token.

  • A model to make predictions from the inputs.

  • Some (optional) post processing for enhancing model’s output.

Parameters

  • task (str) –

    The task defining which pipeline will be returned. Currently accepted tasks are:

  • model (str or PreTrainedModel or TFPreTrainedModel, optional) –

    The model that will be used by the pipeline to make predictions. This can be a model identifier or an actual instance of a pretrained model inheriting from PreTrainedModel (for PyTorch) or TFPreTrainedModel (for TensorFlow).

    If not provided, the default for the task will be loaded.

  • config (str or PretrainedConfig, optional) –

    The configuration that will be used by the pipeline to instantiate the model. This can be a model identifier or an actual pretrained model configuration inheriting from PretrainedConfig.

    If not provided, the default configuration file for the requested model will be used. That means that if model is given, its default configuration will be used. However, if model is not supplied, this task’s default model’s config is used instead.

  • tokenizer (str or PreTrainedTokenizer, optional) –

    The tokenizer that will be used by the pipeline to encode data for the model. This can be a model identifier or an actual pretrained tokenizer inheriting from PreTrainedTokenizer.

    If not provided, the default tokenizer for the given model will be loaded (if it is a string). If model is not specified or not a string, then the default tokenizer for config is loaded (if it is a string). However, if config is also not given or not a string, then the default tokenizer for the given task will be loaded.

  • feature_extractor (str or PreTrainedFeatureExtractor, optional) –

    The feature extractor that will be used by the pipeline to encode data for the model. This can be a model identifier or an actual pretrained feature extractor inheriting from PreTrainedFeatureExtractor.

    Feature extractors are used for non-NLP models, such as Speech or Vision models as well as multi-modal models. Multi-modal models will also require a tokenizer to be passed.

    If not provided, the default feature extractor for the given model will be loaded (if it is a string). If model is not specified or not a string, then the default feature extractor for config is loaded (if it is a string). However, if config is also not given or not a string, then the default feature extractor for the given task will be loaded.

  • framework (str, optional) –

    The framework to use, either "pt" for PyTorch or "tf" for TensorFlow. The specified framework must be installed.

    If no framework is specified, will default to the one currently installed. If no framework is specified and both frameworks are installed, will default to the framework of the model, or to PyTorch if no model is provided.

  • revision (str, optional, defaults to "main") – When passing a task name or a string model identifier: The specific model version to use. It can be a branch name, a tag name, or a commit id, since we use a git-based system for storing models and other artifacts on huggingface.co, so revision can be any identifier allowed by git.

  • use_fast (bool, optional, defaults to True) – Whether or not to use a Fast tokenizer if possible (a PreTrainedTokenizerFast).

  • use_auth_token (str or bool, optional) – The token to use as HTTP bearer authorization for remote files. If True, will use the token generated when running transformers-cli login (stored in huggingface). revision(str, optional, defaults to "main"):

  • model_kwargs – Additional dictionary of keyword arguments passed along to the model’s from_pretrained(..., **model_kwargs) function.

  • kwargs – Additional keyword arguments passed along to the specific pipeline init (see the documentation for the corresponding pipeline class for possible values).

Returns

A suitable pipeline for the task.

Return type

Pipeline

Examples:

>>> from transformers import pipeline, AutoModelForTokenClassification, AutoTokenizer

>>> # Sentiment analysis pipeline
>>> pipeline('sentiment-analysis')

>>> # Question answering pipeline, specifying the checkpoint identifier
>>> pipeline('question-answering', model='distilbert-base-cased-distilled-squad', tokenizer='bert-base-cased')

>>> # Named entity recognition pipeline, passing in a specific model and tokenizer
>>> model = AutoModelForTokenClassification.from_pretrained("dbmdz/bert-large-cased-finetuned-conll03-english")
>>> tokenizer = AutoTokenizer.from_pretrained("bert-base-cased")
>>> pipeline('ner', model=model, tokenizer=tokenizer)

Implementing a pipeline

Implementing a new pipeline

The task specific pipelines

AudioClassificationPipeline

class transformers.AudioClassificationPipeline(*args, **kwargs)[source]

Audio classification pipeline using any AutoModelForAudioClassification. This pipeline predicts the class of a raw waveform or an audio file. In case of an audio file, ffmpeg should be installed to support multiple audio formats.

This pipeline can currently be loaded from pipeline() using the following task identifier: "audio-classification".

See the list of available models on huggingface.co/models.

Parameters

  • model (PreTrainedModel or TFPreTrainedModel) – The model that will be used by the pipeline to make predictions. This needs to be a model inheriting from PreTrainedModel for PyTorch and TFPreTrainedModel for TensorFlow.

  • tokenizer (PreTrainedTokenizer) – The tokenizer that will be used by the pipeline to encode data for the model. This object inherits from PreTrainedTokenizer.

  • modelcard (str or ModelCard, optional) – Model card attributed to the model for this pipeline.

  • framework (str, optional) –

    The framework to use, either "pt" for PyTorch or "tf" for TensorFlow. The specified framework must be installed.

    If no framework is specified, will default to the one currently installed. If no framework is specified and both frameworks are installed, will default to the framework of the model, or to PyTorch if no model is provided.

  • task (str, defaults to "") – A task-identifier for the pipeline.

  • args_parser (ArgumentHandler, optional) – Reference to the object in charge of parsing supplied pipeline parameters.

  • device (int, optional, defaults to -1) – Device ordinal for CPU/GPU supports. Setting this to -1 will leverage CPU, a positive will run the model on the associated CUDA device id.

  • binary_output (bool, optional, defaults to False) – Flag indicating if the output the pipeline should happen in a binary format (i.e., pickle) or as raw text.

__call__(inputs: Union[numpy.ndarray, bytes, str], **kwargs)[source]

Classify the sequence(s) given as inputs. See the AutomaticSpeechRecognitionPipeline documentation for more information.

Parameters

  • inputs (np.ndarray or bytes or str) – The inputs is either a raw waveform (np.ndarray of shape (n, ) of type np.float32 or np.float64) at the correct sampling rate (no further check will be done) or a str that is the filename of the audio file, the file will be read at the correct sampling rate to get the waveform using ffmpeg. This requires ffmpeg to be installed on the system. If inputs is bytes it is supposed to be the content of an audio file and is interpreted by ffmpeg in the same way.

  • top_k (int, optional, defaults to None) – The number of top labels that will be returned by the pipeline. If the provided number is None or higher than the number of labels available in the model configuration, it will default to the number of labels.

Returns

  • label (str) – The label predicted.

  • score (float) – The corresponding probability.

Return type

A list of dict with the following keys

postprocess(model_outputs, top_k=5)[source]

Postprocess will receive the raw outputs of the _forward method, generally tensors, and reformat them into something more friendly. Generally it will output a list or a dict or results (containing just strings and numbers).

preprocess(inputs)[source]

Preprocess will take the input_ of a specific pipeline and return a dictionnary of everything necessary for _forward to run properly. It should contain at least one tensor, but might have arbitrary other items.

AutomaticSpeechRecognitionPipeline

class transformers.AutomaticSpeechRecognitionPipeline(feature_extractor: Union[SequenceFeatureExtractor, str], *args, **kwargs)[source]

Pipeline that aims at extracting spoken text contained within some audio.

The input can be either a raw waveform or a audio file. In case of the audio file, ffmpeg should be installed for to support multiple audio formats

__call__(inputs: Union[numpy.ndarray, bytes, str], **kwargs)[source]

Classify the sequence(s) given as inputs. See the AutomaticSpeechRecognitionPipeline documentation for more information.

Parameters

inputs (np.ndarray or bytes or str) – The inputs is either a raw waveform (np.ndarray of shape (n, ) of type np.float32 or np.float64) at the correct sampling rate (no further check will be done) or a str that is the filename of the audio file, the file will be read at the correct sampling rate to get the waveform using ffmpeg. This requires ffmpeg to be installed on the system. If inputs is bytes it is supposed to be the content of an audio file and is interpreted by ffmpeg in the same way.

Returns

  • text (str) – The recognized text.

Return type

A dict with the following keys

postprocess(model_outputs)[source]

Postprocess will receive the raw outputs of the _forward method, generally tensors, and reformat them into something more friendly. Generally it will output a list or a dict or results (containing just strings and numbers).

preprocess(inputs)[source]

Preprocess will take the input_ of a specific pipeline and return a dictionnary of everything necessary for _forward to run properly. It should contain at least one tensor, but might have arbitrary other items.

ConversationalPipeline

class transformers.Conversation(text: str = None, conversation_id: uuid.UUID = None, past_user_inputs=None, generated_responses=None)[source]

Utility class containing a conversation and its history. This class is meant to be used as an input to the ConversationalPipeline. The conversation contains a number of utility function to manage the addition of new user input and generated model responses. A conversation needs to contain an unprocessed user input before being passed to the ConversationalPipeline. This user input is either created when the class is instantiated, or by calling conversational_pipeline.append_response("input") after a conversation turn.

Parameters

  • text (str, optional) – The initial user input to start the conversation. If not provided, a user input needs to be provided manually using the add_user_input() method before the conversation can begin.

  • conversation_id (uuid.UUID, optional) – Unique identifier for the conversation. If not provided, a random UUID4 id will be assigned to the conversation.

  • past_user_inputs (List[str], optional) – Eventual past history of the conversation of the user. You don’t need to pass it manually if you use the pipeline interactively but if you want to recreate history you need to set both past_user_inputs and generated_responses with equal length lists of strings

  • generated_responses (List[str], optional) – Eventual past history of the conversation of the model. You don’t need to pass it manually if you use the pipeline interactively but if you want to recreate history you need to set both past_user_inputs and generated_responses with equal length lists of strings

Usage:

conversation = Conversation("Going to the movies tonight - any suggestions?")

# Steps usually performed by the model when generating a response:
# 1. Mark the user input as processed (moved to the history)
conversation.mark_processed()
# 2. Append a mode response
conversation.append_response("The Big lebowski.")

conversation.add_user_input("Is it good?")
class transformers.ConversationalPipeline(*args, **kwargs)[source]

Multi-turn conversational pipeline.

This conversational pipeline can currently be loaded from pipeline() using the following task identifier: "conversational".

The models that this pipeline can use are models that have been fine-tuned on a multi-turn conversational task, currently: ‘microsoft/DialoGPT-small’, ‘microsoft/DialoGPT-medium’, ‘microsoft/DialoGPT-large’. See the up-to-date list of available models on huggingface.co/models.

Usage:

conversational_pipeline = pipeline("conversational")

conversation_1 = Conversation("Going to the movies tonight - any suggestions?")
conversation_2 = Conversation("What's the last book you have read?")

conversational_pipeline([conversation_1, conversation_2])

conversation_1.add_user_input("Is it an action movie?")
conversation_2.add_user_input("What is the genre of this book?")

conversational_pipeline([conversation_1, conversation_2])
Parameters

  • model (PreTrainedModel or TFPreTrainedModel) – The model that will be used by the pipeline to make predictions. This needs to be a model inheriting from PreTrainedModel for PyTorch and TFPreTrainedModel for TensorFlow.

  • tokenizer (PreTrainedTokenizer) – The tokenizer that will be used by the pipeline to encode data for the model. This object inherits from PreTrainedTokenizer.

  • modelcard (str or ModelCard, optional) – Model card attributed to the model for this pipeline.

  • framework (str, optional) –

    The framework to use, either "pt" for PyTorch or "tf" for TensorFlow. The specified framework must be installed.

    If no framework is specified, will default to the one currently installed. If no framework is specified and both frameworks are installed, will default to the framework of the model, or to PyTorch if no model is provided.

  • task (str, defaults to "") – A task-identifier for the pipeline.

  • args_parser (ArgumentHandler, optional) – Reference to the object in charge of parsing supplied pipeline parameters.

  • device (int, optional, defaults to -1) – Device ordinal for CPU/GPU supports. Setting this to -1 will leverage CPU, a positive will run the model on the associated CUDA device id.

  • binary_output (bool, optional, defaults to False) – Flag indicating if the output the pipeline should happen in a binary format (i.e., pickle) or as raw text.

  • min_length_for_response (int, optional, defaults to 32) – The minimum length (in number of tokens) for a response.

  • minimum_tokens (int, optional, defaults to 10) – The minimum length of tokens to leave for a response.

__call__(conversations: Union[transformers.pipelines.conversational.Conversation, List[transformers.pipelines.conversational.Conversation]], num_workers=0, **kwargs)[source]

Generate responses for the conversation(s) given as inputs.

Parameters

  • conversations (a Conversation or a list of Conversation) – Conversations to generate responses for.

  • clean_up_tokenization_spaces (bool, optional, defaults to False) – Whether or not to clean up the potential extra spaces in the text output.

  • generate_kwargs – Additional keyword arguments to pass along to the generate method of the model (see the generate method corresponding to your framework here).

Returns

Conversation(s) with updated generated responses for those containing a new user input.

Return type

Conversation or a list of Conversation

postprocess(model_outputs, clean_up_tokenization_spaces=True)[source]

Postprocess will receive the raw outputs of the _forward method, generally tensors, and reformat them into something more friendly. Generally it will output a list or a dict or results (containing just strings and numbers).

preprocess(conversation: transformers.pipelines.conversational.Conversation) → Dict[str, Any][source]

Preprocess will take the input_ of a specific pipeline and return a dictionnary of everything necessary for _forward to run properly. It should contain at least one tensor, but might have arbitrary other items.

FeatureExtractionPipeline

class transformers.FeatureExtractionPipeline(model: Union[PreTrainedModel, TFPreTrainedModel], tokenizer: Optional[transformers.tokenization_utils.PreTrainedTokenizer] = None, feature_extractor: Optional[SequenceFeatureExtractor] = None, modelcard: Optional[transformers.modelcard.ModelCard] = None, framework: Optional[str] = None, task: str = '', args_parser: transformers.pipelines.base.ArgumentHandler = None, device: int = - 1, binary_output: bool = False, **kwargs)[source]

Feature extraction pipeline using no model head. This pipeline extracts the hidden states from the base transformer, which can be used as features in downstream tasks.

This feature extraction pipeline can currently be loaded from pipeline() using the task identifier: "feature-extraction".

All models may be used for this pipeline. See a list of all models, including community-contributed models on huggingface.co/models.

Parameters

  • model (PreTrainedModel or TFPreTrainedModel) – The model that will be used by the pipeline to make predictions. This needs to be a model inheriting from PreTrainedModel for PyTorch and TFPreTrainedModel for TensorFlow.

  • tokenizer (PreTrainedTokenizer) – The tokenizer that will be used by the pipeline to encode data for the model. This object inherits from PreTrainedTokenizer.

  • modelcard (str or ModelCard, optional) – Model card attributed to the model for this pipeline.

  • framework (str, optional) –

    The framework to use, either "pt" for PyTorch or "tf" for TensorFlow. The specified framework must be installed.

    If no framework is specified, will default to the one currently installed. If no framework is specified and both frameworks are installed, will default to the framework of the model, or to PyTorch if no model is provided.

  • task (str, defaults to "") – A task-identifier for the pipeline.

  • args_parser (ArgumentHandler, optional) – Reference to the object in charge of parsing supplied pipeline parameters.

  • device (int, optional, defaults to -1) – Device ordinal for CPU/GPU supports. Setting this to -1 will leverage CPU, a positive will run the model on the associated CUDA device id.

__call__(*args, **kwargs)[source]

Extract the features of the input(s).

Parameters

args (str or List[str]) – One or several texts (or one list of texts) to get the features of.

Returns

The features computed by the model.

Return type

A nested list of float

postprocess(model_outputs)[source]

Postprocess will receive the raw outputs of the _forward method, generally tensors, and reformat them into something more friendly. Generally it will output a list or a dict or results (containing just strings and numbers).

preprocess(inputs) → Dict[str, Union[List[GenericTensor], torch.Tensor, tf.Tensor]][source]

Preprocess will take the input_ of a specific pipeline and return a dictionnary of everything necessary for _forward to run properly. It should contain at least one tensor, but might have arbitrary other items.

FillMaskPipeline

class transformers.FillMaskPipeline(model: Union[PreTrainedModel, TFPreTrainedModel], tokenizer: Optional[transformers.tokenization_utils.PreTrainedTokenizer] = None, feature_extractor: Optional[SequenceFeatureExtractor] = None, modelcard: Optional[transformers.modelcard.ModelCard] = None, framework: Optional[str] = None, task: str = '', args_parser: transformers.pipelines.base.ArgumentHandler = None, device: int = - 1, binary_output: bool = False, **kwargs)[source]

Masked language modeling prediction pipeline using any ModelWithLMHead. See the masked language modeling examples for more information.

This mask filling pipeline can currently be loaded from pipeline() using the following task identifier: "fill-mask".

The models that this pipeline can use are models that have been trained with a masked language modeling objective, which includes the bi-directional models in the library. See the up-to-date list of available models on huggingface.co/models.

Note

This pipeline only works for inputs with exactly one token masked.

Parameters

  • model (PreTrainedModel or TFPreTrainedModel) – The model that will be used by the pipeline to make predictions. This needs to be a model inheriting from PreTrainedModel for PyTorch and TFPreTrainedModel for TensorFlow.

  • tokenizer (PreTrainedTokenizer) – The tokenizer that will be used by the pipeline to encode data for the model. This object inherits from PreTrainedTokenizer.

  • modelcard (str or ModelCard, optional) – Model card attributed to the model for this pipeline.

  • framework (str, optional) –

    The framework to use, either "pt" for PyTorch or "tf" for TensorFlow. The specified framework must be installed.

    If no framework is specified, will default to the one currently installed. If no framework is specified and both frameworks are installed, will default to the framework of the model, or to PyTorch if no model is provided.

  • task (str, defaults to "") – A task-identifier for the pipeline.

  • args_parser (ArgumentHandler, optional) – Reference to the object in charge of parsing supplied pipeline parameters.

  • device (int, optional, defaults to -1) – Device ordinal for CPU/GPU supports. Setting this to -1 will leverage CPU, a positive will run the model on the associated CUDA device id.

  • binary_output (bool, optional, defaults to False) – Flag indicating if the output the pipeline should happen in a binary format (i.e., pickle) or as raw text.

  • top_k (int, defaults to 5) – The number of predictions to return.

  • targets (str or List[str], optional) – When passed, the model will limit the scores to the passed targets instead of looking up in the whole vocab. If the provided targets are not in the model vocab, they will be tokenized and the first resulting token will be used (with a warning, and that might be slower).

__call__(inputs, *args, **kwargs)[source]

Fill the masked token in the text(s) given as inputs.

Parameters

  • args (str or List[str]) – One or several texts (or one list of prompts) with masked tokens.

  • targets (str or List[str], optional) – When passed, the model will limit the scores to the passed targets instead of looking up in the whole vocab. If the provided targets are not in the model vocab, they will be tokenized and the first resulting token will be used (with a warning, and that might be slower).

  • top_k (int, optional) – When passed, overrides the number of predictions to return.

Returns

Each result comes as list of dictionaries with the following keys:

  • sequence (str) – The corresponding input with the mask token prediction.

  • score (float) – The corresponding probability.

  • token (int) – The predicted token id (to replace the masked one).

  • token (str) – The predicted token (to replace the masked one).

Return type

A list or a list of list of dict

postprocess(model_outputs, top_k=5, target_ids=None)[source]

Postprocess will receive the raw outputs of the _forward method, generally tensors, and reformat them into something more friendly. Generally it will output a list or a dict or results (containing just strings and numbers).

preprocess(inputs, return_tensors=None, **preprocess_parameters) → Dict[str, Union[List[Union[List[GenericTensor], torch.Tensor, tf.Tensor]], torch.Tensor, tensorflow.python.framework.ops.Tensor]][source]

Preprocess will take the input_ of a specific pipeline and return a dictionnary of everything necessary for _forward to run properly. It should contain at least one tensor, but might have arbitrary other items.

ImageClassificationPipeline

class transformers.ImageClassificationPipeline(*args, **kwargs)[source]

Image classification pipeline using any AutoModelForImageClassification. This pipeline predicts the class of an image.

This image classification pipeline can currently be loaded from pipeline() using the following task identifier: "image-classification".

See the list of available models on huggingface.co/models.

Parameters

  • model (PreTrainedModel or TFPreTrainedModel) – The model that will be used by the pipeline to make predictions. This needs to be a model inheriting from PreTrainedModel for PyTorch and TFPreTrainedModel for TensorFlow.

  • tokenizer (PreTrainedTokenizer) – The tokenizer that will be used by the pipeline to encode data for the model. This object inherits from PreTrainedTokenizer.

  • modelcard (str or ModelCard, optional) – Model card attributed to the model for this pipeline.

  • framework (str, optional) –

    The framework to use, either "pt" for PyTorch or "tf" for TensorFlow. The specified framework must be installed.

    If no framework is specified, will default to the one currently installed. If no framework is specified and both frameworks are installed, will default to the framework of the model, or to PyTorch if no model is provided.

  • task (str, defaults to "") – A task-identifier for the pipeline.

  • args_parser (ArgumentHandler, optional) – Reference to the object in charge of parsing supplied pipeline parameters.

  • device (int, optional, defaults to -1) – Device ordinal for CPU/GPU supports. Setting this to -1 will leverage CPU, a positive will run the model on the associated CUDA device id.

  • binary_output (bool, optional, defaults to False) – Flag indicating if the output the pipeline should happen in a binary format (i.e., pickle) or as raw text.

__call__(images: Union[str, List[str], Image, List[Image]], **kwargs)[source]

Assign labels to the image(s) passed as inputs.

Parameters

  • images (str, List[str], PIL.Image or List[PIL.Image]) –

    The pipeline handles three types of images:

    • A string containing a http link pointing to an image

    • A string containing a local path to an image

    • An image loaded in PIL directly

    The pipeline accepts either a single image or a batch of images, which must then be passed as a string. Images in a batch must all be in the same format: all as http links, all as local paths, or all as PIL images.

  • top_k (int, optional, defaults to 5) – The number of top labels that will be returned by the pipeline. If the provided number is higher than the number of labels available in the model configuration, it will default to the number of labels.

Returns

A dictionary or a list of dictionaries containing result. If the input is a single image, will return a dictionary, if the input is a list of several images, will return a list of dictionaries corresponding to the images.

The dictionaries contain the following keys:

  • label (str) – The label identified by the model.

  • score (int) – The score attributed by the model for that label.

postprocess(model_outputs, top_k=5)[source]

Postprocess will receive the raw outputs of the _forward method, generally tensors, and reformat them into something more friendly. Generally it will output a list or a dict or results (containing just strings and numbers).

preprocess(image)[source]

Preprocess will take the input_ of a specific pipeline and return a dictionnary of everything necessary for _forward to run properly. It should contain at least one tensor, but might have arbitrary other items.

ImageSegmentationPipeline

class transformers.ImageSegmentationPipeline(*args, **kwargs)[source]

Image segmentation pipeline using any AutoModelForImageSegmentation. This pipeline predicts masks of objects and their classes.

This image segmntation pipeline can currently be loaded from pipeline() using the following task identifier: "image-segmentation".

See the list of available models on huggingface.co/models.

Parameters

  • model (PreTrainedModel or TFPreTrainedModel) – The model that will be used by the pipeline to make predictions. This needs to be a model inheriting from PreTrainedModel for PyTorch and TFPreTrainedModel for TensorFlow.

  • tokenizer (PreTrainedTokenizer) – The tokenizer that will be used by the pipeline to encode data for the model. This object inherits from PreTrainedTokenizer.

  • modelcard (str or ModelCard, optional) – Model card attributed to the model for this pipeline.

  • framework (str, optional) –

    The framework to use, either "pt" for PyTorch or "tf" for TensorFlow. The specified framework must be installed.

    If no framework is specified, will default to the one currently installed. If no framework is specified and both frameworks are installed, will default to the framework of the model, or to PyTorch if no model is provided.

  • task (str, defaults to "") – A task-identifier for the pipeline.

  • args_parser (ArgumentHandler, optional) – Reference to the object in charge of parsing supplied pipeline parameters.

  • device (int, optional, defaults to -1) – Device ordinal for CPU/GPU supports. Setting this to -1 will leverage CPU, a positive will run the model on the associated CUDA device id.

  • binary_output (bool, optional, defaults to False) – Flag indicating if the output the pipeline should happen in a binary format (i.e., pickle) or as raw text.

__call__(*args, **kwargs) → List[Dict[str, Any]][source]

Perform segmentation (detect masks & classes) in the image(s) passed as inputs.

Parameters

  • images (str, List[str], PIL.Image or List[PIL.Image]) –

    The pipeline handles three types of images:

    • A string containing an HTTP(S) link pointing to an image

    • A string containing a local path to an image

    • An image loaded in PIL directly

    The pipeline accepts either a single image or a batch of images. Images in a batch must all be in the same format: all as HTTP(S) links, all as local paths, or all as PIL images.

  • threshold (float, optional, defaults to 0.9) – The probability necessary to make a prediction.

  • mask_threshold (float, optional, defaults to 0.5) – Threshold to use when turning the predicted masks into binary values.

Returns

A dictionary or a list of dictionaries containing the result. If the input is a single image, will return a dictionary, if the input is a list of several images, will return a list of dictionaries corresponding to each image.

The dictionaries contain the following keys:

  • label (str) – The class label identified by the model.

  • score (float) – The score attributed by the model for that label.

  • mask (str) – base64 string of a grayscale (single-channel) PNG image that contain masks information. The PNG image has size (heigth, width) of the original image. Pixel values in the image are either 0 or 255 (i.e. mask is absent VS mask is present).

postprocess(model_outputs, threshold=0.9, mask_threshold=0.5)[source]

Postprocess will receive the raw outputs of the _forward method, generally tensors, and reformat them into something more friendly. Generally it will output a list or a dict or results (containing just strings and numbers).

preprocess(image)[source]

Preprocess will take the input_ of a specific pipeline and return a dictionnary of everything necessary for _forward to run properly. It should contain at least one tensor, but might have arbitrary other items.

NerPipeline

transformers.NerPipeline

alias of transformers.pipelines.token_classification.TokenClassificationPipeline

See TokenClassificationPipeline for all details.

ObjectDetectionPipeline

class transformers.ObjectDetectionPipeline(*args, **kwargs)[source]

Object detection pipeline using any AutoModelForObjectDetection. This pipeline predicts bounding boxes of objects and their classes.

This object detection pipeline can currently be loaded from pipeline() using the following task identifier: "object-detection".

See the list of available models on huggingface.co/models.

Parameters

  • model (PreTrainedModel or TFPreTrainedModel) – The model that will be used by the pipeline to make predictions. This needs to be a model inheriting from PreTrainedModel for PyTorch and TFPreTrainedModel for TensorFlow.

  • tokenizer (PreTrainedTokenizer) – The tokenizer that will be used by the pipeline to encode data for the model. This object inherits from PreTrainedTokenizer.

  • modelcard (str or ModelCard, optional) – Model card attributed to the model for this pipeline.

  • framework (str, optional) –

    The framework to use, either "pt" for PyTorch or "tf" for TensorFlow. The specified framework must be installed.

    If no framework is specified, will default to the one currently installed. If no framework is specified and both frameworks are installed, will default to the framework of the model, or to PyTorch if no model is provided.

  • task (str, defaults to "") – A task-identifier for the pipeline.

  • args_parser (ArgumentHandler, optional) – Reference to the object in charge of parsing supplied pipeline parameters.

  • device (int, optional, defaults to -1) – Device ordinal for CPU/GPU supports. Setting this to -1 will leverage CPU, a positive will run the model on the associated CUDA device id.

  • binary_output (bool, optional, defaults to False) – Flag indicating if the output the pipeline should happen in a binary format (i.e., pickle) or as raw text.

__call__(*args, **kwargs) → List[Dict[str, Any]][source]

Detect objects (bounding boxes & classes) in the image(s) passed as inputs.

Parameters

  • images (str, List[str], PIL.Image or List[PIL.Image]) –

    The pipeline handles three types of images:

    • A string containing an HTTP(S) link pointing to an image

    • A string containing a local path to an image

    • An image loaded in PIL directly

    The pipeline accepts either a single image or a batch of images. Images in a batch must all be in the same format: all as HTTP(S) links, all as local paths, or all as PIL images.

  • threshold (float, optional, defaults to 0.9) – The probability necessary to make a prediction.

Returns

A list of dictionaries or a list of list of dictionaries containing the result. If the input is a single image, will return a list of dictionaries, if the input is a list of several images, will return a list of list of dictionaries corresponding to each image.

The dictionaries contain the following keys:

  • label (str) – The class label identified by the model.

  • score (float) – The score attributed by the model for that label.

  • box (List[Dict[str, int]]) – The bounding box of detected object in image’s original size.

postprocess(model_outputs, threshold=0.9)[source]

Postprocess will receive the raw outputs of the _forward method, generally tensors, and reformat them into something more friendly. Generally it will output a list or a dict or results (containing just strings and numbers).

preprocess(image)[source]

Preprocess will take the input_ of a specific pipeline and return a dictionnary of everything necessary for _forward to run properly. It should contain at least one tensor, but might have arbitrary other items.

QuestionAnsweringPipeline

class transformers.QuestionAnsweringPipeline(model: Union[PreTrainedModel, TFPreTrainedModel], tokenizer: transformers.tokenization_utils.PreTrainedTokenizer, modelcard: Optional[transformers.modelcard.ModelCard] = None, framework: Optional[str] = None, device: int = - 1, task: str = '', **kwargs)[source]

Question Answering pipeline using any ModelForQuestionAnswering. See the question answering examples for more information.

This question answering pipeline can currently be loaded from pipeline() using the following task identifier: "question-answering".

The models that this pipeline can use are models that have been fine-tuned on a question answering task. See the up-to-date list of available models on huggingface.co/models.

Parameters

  • model (PreTrainedModel or TFPreTrainedModel) – The model that will be used by the pipeline to make predictions. This needs to be a model inheriting from PreTrainedModel for PyTorch and TFPreTrainedModel for TensorFlow.

  • tokenizer (PreTrainedTokenizer) – The tokenizer that will be used by the pipeline to encode data for the model. This object inherits from PreTrainedTokenizer.

  • modelcard (str or ModelCard, optional) – Model card attributed to the model for this pipeline.

  • framework (str, optional) –

    The framework to use, either "pt" for PyTorch or "tf" for TensorFlow. The specified framework must be installed.

    If no framework is specified, will default to the one currently installed. If no framework is specified and both frameworks are installed, will default to the framework of the model, or to PyTorch if no model is provided.

  • task (str, defaults to "") – A task-identifier for the pipeline.

  • args_parser (ArgumentHandler, optional) – Reference to the object in charge of parsing supplied pipeline parameters.

  • device (int, optional, defaults to -1) – Device ordinal for CPU/GPU supports. Setting this to -1 will leverage CPU, a positive will run the model on the associated CUDA device id.

  • binary_output (bool, optional, defaults to False) – Flag indicating if the output the pipeline should happen in a binary format (i.e., pickle) or as raw text.

__call__(*args, **kwargs)[source]

Answer the question(s) given as inputs by using the context(s).

Parameters

  • args (SquadExample or a list of SquadExample) – One or several SquadExample containing the question and context.

  • X (SquadExample or a list of SquadExample, optional) – One or several SquadExample containing the question and context (will be treated the same way as if passed as the first positional argument).

  • data (SquadExample or a list of SquadExample, optional) – One or several SquadExample containing the question and context (will be treated the same way as if passed as the first positional argument).

  • question (str or List[str]) – One or several question(s) (must be used in conjunction with the context argument).

  • context (str or List[str]) – One or several context(s) associated with the question(s) (must be used in conjunction with the question argument).

  • topk (int, optional, defaults to 1) – The number of answers to return (will be chosen by order of likelihood). Note that we return less than topk answers if there are not enough options available within the context.

  • doc_stride (int, optional, defaults to 128) – If the context is too long to fit with the question for the model, it will be split in several chunks with some overlap. This argument controls the size of that overlap.

  • max_answer_len (int, optional, defaults to 15) – The maximum length of predicted answers (e.g., only answers with a shorter length are considered).

  • max_seq_len (int, optional, defaults to 384) – The maximum length of the total sentence (context + question) after tokenization. The context will be split in several chunks (using doc_stride) if needed.

  • max_question_len (int, optional, defaults to 64) – The maximum length of the question after tokenization. It will be truncated if needed.

  • handle_impossible_answer (bool, optional, defaults to False) – Whether or not we accept impossible as an answer.

Returns

Each result comes as a dictionary with the following keys:

  • score (float) – The probability associated to the answer.

  • start (int) – The character start index of the answer (in the tokenized version of the input).

  • end (int) – The character end index of the answer (in the tokenized version of the input).

  • answer (str) – The answer to the question.

Return type

A dict or a list of dict

static create_sample(question: Union[str, List[str]], context: Union[str, List[str]]) → Union[transformers.data.processors.squad.SquadExample, List[transformers.data.processors.squad.SquadExample]][source]

QuestionAnsweringPipeline leverages the SquadExample internally. This helper method encapsulate all the logic for converting question(s) and context(s) to SquadExample.

We currently support extractive question answering.

Parameters

  • question (str or List[str]) – The question(s) asked.

  • context (str or List[str]) – The context(s) in which we will look for the answer.

Returns

The corresponding SquadExample grouping question and context.

Return type

One or a list of SquadExample

decode(start: numpy.ndarray, end: numpy.ndarray, topk: int, max_answer_len: int, undesired_tokens: numpy.ndarray) → Tuple[source]

Take the output of any ModelForQuestionAnswering and will generate probabilities for each span to be the actual answer.

In addition, it filters out some unwanted/impossible cases like answer len being greater than max_answer_len or answer end position being before the starting position. The method supports output the k-best answer through the topk argument.

Parameters

  • start (np.ndarray) – Individual start probabilities for each token.

  • end (np.ndarray) – Individual end probabilities for each token.

  • topk (int) – Indicates how many possible answer span(s) to extract from the model output.

  • max_answer_len (int) – Maximum size of the answer to extract from the model’s output.

  • undesired_tokens (np.ndarray) – Mask determining tokens that can be part of the answer

postprocess(model_outputs, top_k=1, handle_impossible_answer=False, max_answer_len=15)[source]

Postprocess will receive the raw outputs of the _forward method, generally tensors, and reformat them into something more friendly. Generally it will output a list or a dict or results (containing just strings and numbers).

preprocess(example, padding='do_not_pad', doc_stride=None, max_question_len=64, max_seq_len=None)[source]

Preprocess will take the input_ of a specific pipeline and return a dictionnary of everything necessary for _forward to run properly. It should contain at least one tensor, but might have arbitrary other items.

span_to_answer(text: str, start: int, end: int) → Dict[str, Union[str, int]][source]

When decoding from token probabilities, this method maps token indexes to actual word in the initial context.

Parameters

  • text (str) – The actual context to extract the answer from.

  • start (int) – The answer starting token index.

  • end (int) – The answer end token index.

Returns

Dictionary like {'answer': str, 'start': int, 'end': int}

SummarizationPipeline

class transformers.SummarizationPipeline(*args, **kwargs)[source]

Summarize news articles and other documents.

This summarizing pipeline can currently be loaded from pipeline() using the following task identifier: "summarization".

The models that this pipeline can use are models that have been fine-tuned on a summarization task, which is currently, ‘bart-large-cnn’, ‘t5-small’, ‘t5-base’, ‘t5-large’, ‘t5-3b’, ‘t5-11b’. See the up-to-date list of available models on huggingface.co/models.

Usage:

# use bart in pytorch
summarizer = pipeline("summarization")
summarizer("An apple a day, keeps the doctor away", min_length=5, max_length=20)

# use t5 in tf
summarizer = pipeline("summarization", model="t5-base", tokenizer="t5-base", framework="tf")
summarizer("An apple a day, keeps the doctor away", min_length=5, max_length=20)
Parameters

  • model (PreTrainedModel or TFPreTrainedModel) – The model that will be used by the pipeline to make predictions. This needs to be a model inheriting from PreTrainedModel for PyTorch and TFPreTrainedModel for TensorFlow.

  • tokenizer (PreTrainedTokenizer) – The tokenizer that will be used by the pipeline to encode data for the model. This object inherits from PreTrainedTokenizer.

  • modelcard (str or ModelCard, optional) – Model card attributed to the model for this pipeline.

  • framework (str, optional) –

    The framework to use, either "pt" for PyTorch or "tf" for TensorFlow. The specified framework must be installed.

    If no framework is specified, will default to the one currently installed. If no framework is specified and both frameworks are installed, will default to the framework of the model, or to PyTorch if no model is provided.

  • task (str, defaults to "") – A task-identifier for the pipeline.

  • args_parser (ArgumentHandler, optional) – Reference to the object in charge of parsing supplied pipeline parameters.

  • device (int, optional, defaults to -1) – Device ordinal for CPU/GPU supports. Setting this to -1 will leverage CPU, a positive will run the model on the associated CUDA device id.

  • binary_output (bool, optional, defaults to False) – Flag indicating if the output the pipeline should happen in a binary format (i.e., pickle) or as raw text.

__call__(*args, **kwargs)[source]

Summarize the text(s) given as inputs.

Parameters

  • documents (str or List[str]) – One or several articles (or one list of articles) to summarize.

  • return_text (bool, optional, defaults to True) – Whether or not to include the decoded texts in the outputs

  • return_tensors (bool, optional, defaults to False) – Whether or not to include the tensors of predictions (as token indices) in the outputs.

  • clean_up_tokenization_spaces (bool, optional, defaults to False) – Whether or not to clean up the potential extra spaces in the text output.

  • generate_kwargs – Additional keyword arguments to pass along to the generate method of the model (see the generate method corresponding to your framework here).

Returns

Each result comes as a dictionary with the following keys:

  • summary_text (str, present when return_text=True) – The summary of the corresponding input.

  • summary_token_ids (torch.Tensor or tf.Tensor, present when return_tensors=True) – The token ids of the summary.

Return type

A list or a list of list of dict

check_inputs(input_length: int, min_length: int, max_length: int) → bool[source]

Checks whether there might be something wrong with given input with regard to the model.

TableQuestionAnsweringPipeline

class transformers.TableQuestionAnsweringPipeline(args_parser=<transformers.pipelines.table_question_answering.TableQuestionAnsweringArgumentHandler object>, *args, **kwargs)[source]

Table Question Answering pipeline using a ModelForTableQuestionAnswering. This pipeline is only available in PyTorch.

This tabular question answering pipeline can currently be loaded from pipeline() using the following task identifier: "table-question-answering".

The models that this pipeline can use are models that have been fine-tuned on a tabular question answering task. See the up-to-date list of available models on huggingface.co/models.

Parameters

  • model (PreTrainedModel or TFPreTrainedModel) – The model that will be used by the pipeline to make predictions. This needs to be a model inheriting from PreTrainedModel for PyTorch and TFPreTrainedModel for TensorFlow.

  • tokenizer (PreTrainedTokenizer) – The tokenizer that will be used by the pipeline to encode data for the model. This object inherits from PreTrainedTokenizer.

  • modelcard (str or ModelCard, optional) – Model card attributed to the model for this pipeline.

  • framework (str, optional) –

    The framework to use, either "pt" for PyTorch or "tf" for TensorFlow. The specified framework must be installed.

    If no framework is specified, will default to the one currently installed. If no framework is specified and both frameworks are installed, will default to the framework of the model, or to PyTorch if no model is provided.

  • task (str, defaults to "") – A task-identifier for the pipeline.

  • args_parser (ArgumentHandler, optional) – Reference to the object in charge of parsing supplied pipeline parameters.

  • device (int, optional, defaults to -1) – Device ordinal for CPU/GPU supports. Setting this to -1 will leverage CPU, a positive will run the model on the associated CUDA device id.

  • binary_output (bool, optional, defaults to False) – Flag indicating if the output the pipeline should happen in a binary format (i.e., pickle) or as raw text.

__call__(*args, **kwargs)[source]

Answers queries according to a table. The pipeline accepts several types of inputs which are detailed below:

  • pipeline(table, query)

  • pipeline(table, [query])

  • pipeline(table=table, query=query)

  • pipeline(table=table, query=[query])

  • pipeline({"table": table, "query": query})

  • pipeline({"table": table, "query": [query]})

  • pipeline([{"table": table, "query": query}, {"table": table, "query": query}])

The table argument should be a dict or a DataFrame built from that dict, containing the whole table:

Example:

data = {
    "actors": ["brad pitt", "leonardo di caprio", "george clooney"],
    "age": ["56", "45", "59"],
    "number of movies": ["87", "53", "69"],
    "date of birth": ["7 february 1967", "10 june 1996", "28 november 1967"],
}

This dictionary can be passed in as such, or can be converted to a pandas DataFrame:

Example:

import pandas as pd
table = pd.DataFrame.from_dict(data)
Parameters

  • table (pd.DataFrame or Dict) – Pandas DataFrame or dictionary that will be converted to a DataFrame containing all the table values. See above for an example of dictionary.

  • query (str or List[str]) – Query or list of queries that will be sent to the model alongside the table.

  • sequential (bool, optional, defaults to False) – Whether to do inference sequentially or as a batch. Batching is faster, but models like SQA require the inference to be done sequentially to extract relations within sequences, given their conversational nature.

  • padding (bool, str or PaddingStrategy, optional, defaults to False) –

    Activates and controls padding. Accepts the following values:

    • True or 'longest': Pad to the longest sequence in the batch (or no padding if only a single sequence if provided).

    • 'max_length': Pad to a maximum length specified with the argument max_length or to the maximum acceptable input length for the model if that argument is not provided.

    • False or 'do_not_pad' (default): No padding (i.e., can output a batch with sequences of different lengths).

  • truncation (bool, str or TapasTruncationStrategy, optional, defaults to False) –

    Activates and controls truncation. Accepts the following values:

    • True or 'drop_rows_to_fit': Truncate to a maximum length specified with the argument max_length or to the maximum acceptable input length for the model if that argument is not provided. This will truncate row by row, removing rows from the table.

    • False or 'do_not_truncate' (default): No truncation (i.e., can output batch with sequence lengths greater than the model maximum admissible input size).

Returns

Each result is a dictionary with the following keys:

  • answer (str) – The answer of the query given the table. If there is an aggregator, the answer will be preceded by AGGREGATOR >.

  • coordinates (List[Tuple[int, int]]) – Coordinates of the cells of the answers.

  • cells (List[str]) – List of strings made up of the answer cell values.

  • aggregator (str) – If the model has an aggregator, this returns the aggregator.

Return type

A dictionary or a list of dictionaries containing results

TextClassificationPipeline

class transformers.TextClassificationPipeline(**kwargs)[source]

Text classification pipeline using any ModelForSequenceClassification. See the sequence classification examples for more information.

This text classification pipeline can currently be loaded from pipeline() using the following task identifier: "sentiment-analysis" (for classifying sequences according to positive or negative sentiments).

If multiple classification labels are available (model.config.num_labels >= 2), the pipeline will run a softmax over the results. If there is a single label, the pipeline will run a sigmoid over the result.

The models that this pipeline can use are models that have been fine-tuned on a sequence classification task. See the up-to-date list of available models on huggingface.co/models.

Parameters

  • model (PreTrainedModel or TFPreTrainedModel) – The model that will be used by the pipeline to make predictions. This needs to be a model inheriting from PreTrainedModel for PyTorch and TFPreTrainedModel for TensorFlow.

  • tokenizer (PreTrainedTokenizer) – The tokenizer that will be used by the pipeline to encode data for the model. This object inherits from PreTrainedTokenizer.

  • modelcard (str or ModelCard, optional) – Model card attributed to the model for this pipeline.

  • framework (str, optional) –

    The framework to use, either "pt" for PyTorch or "tf" for TensorFlow. The specified framework must be installed.

    If no framework is specified, will default to the one currently installed. If no framework is specified and both frameworks are installed, will default to the framework of the model, or to PyTorch if no model is provided.

  • task (str, defaults to "") – A task-identifier for the pipeline.

  • args_parser (ArgumentHandler, optional) – Reference to the object in charge of parsing supplied pipeline parameters.

  • device (int, optional, defaults to -1) – Device ordinal for CPU/GPU supports. Setting this to -1 will leverage CPU, a positive will run the model on the associated CUDA device id.

  • binary_output (bool, optional, defaults to False) – Flag indicating if the output the pipeline should happen in a binary format (i.e., pickle) or as raw text.

  • return_all_scores (bool, optional, defaults to False) – Whether to return all prediction scores or just the one of the predicted class.

  • function_to_apply (str, optional, defaults to "default") –

    The function to apply to the model outputs in order to retrieve the scores. Accepts four different values:

    • "default": if the model has a single label, will apply the sigmoid function on the output. If the model has several labels, will apply the softmax function on the output.

    • "sigmoid": Applies the sigmoid function on the output.

    • "softmax": Applies the softmax function on the output.

    • "none": Does not apply any function on the output.

__call__(*args, **kwargs)[source]

Classify the text(s) given as inputs.

Parameters

  • args (str or List[str]) – One or several texts (or one list of prompts) to classify.

  • return_all_scores (bool, optional, defaults to False) – Whether to return scores for all labels.

  • function_to_apply (str, optional, defaults to "default") –

    The function to apply to the model outputs in order to retrieve the scores. Accepts four different values:

    If this argument is not specified, then it will apply the following functions according to the number of labels:

    • If the model has a single label, will apply the sigmoid function on the output.

    • If the model has several labels, will apply the softmax function on the output.

    Possible values are:

    • "sigmoid": Applies the sigmoid function on the output.

    • "softmax": Applies the softmax function on the output.

    • "none": Does not apply any function on the output.

Returns

Each result comes as list of dictionaries with the following keys:

  • label (str) – The label predicted.

  • score (float) – The corresponding probability.

If self.return_all_scores=True, one such dictionary is returned per label.

Return type

A list or a list of list of dict

postprocess(model_outputs, function_to_apply=None, return_all_scores=False)[source]

Postprocess will receive the raw outputs of the _forward method, generally tensors, and reformat them into something more friendly. Generally it will output a list or a dict or results (containing just strings and numbers).

preprocess(inputs, **tokenizer_kwargs) → Dict[str, Union[List[Union[List[GenericTensor], torch.Tensor, tf.Tensor]], torch.Tensor, tensorflow.python.framework.ops.Tensor]][source]

Preprocess will take the input_ of a specific pipeline and return a dictionnary of everything necessary for _forward to run properly. It should contain at least one tensor, but might have arbitrary other items.

run_single(inputs, preprocess_params, forward_params, postprocess_params)[source]

This pipeline is odd, and return a list when single item is run

TextGenerationPipeline

class transformers.TextGenerationPipeline(*args, **kwargs)[source]

Language generation pipeline using any ModelWithLMHead. This pipeline predicts the words that will follow a specified text prompt.

This language generation pipeline can currently be loaded from pipeline() using the following task identifier: "text-generation".

The models that this pipeline can use are models that have been trained with an autoregressive language modeling objective, which includes the uni-directional models in the library (e.g. gpt2). See the list of available models on huggingface.co/models.

Parameters

  • model (PreTrainedModel or TFPreTrainedModel) – The model that will be used by the pipeline to make predictions. This needs to be a model inheriting from PreTrainedModel for PyTorch and TFPreTrainedModel for TensorFlow.

  • tokenizer (PreTrainedTokenizer) – The tokenizer that will be used by the pipeline to encode data for the model. This object inherits from PreTrainedTokenizer.

  • modelcard (str or ModelCard, optional) – Model card attributed to the model for this pipeline.

  • framework (str, optional) –

    The framework to use, either "pt" for PyTorch or "tf" for TensorFlow. The specified framework must be installed.

    If no framework is specified, will default to the one currently installed. If no framework is specified and both frameworks are installed, will default to the framework of the model, or to PyTorch if no model is provided.

  • task (str, defaults to "") – A task-identifier for the pipeline.

  • args_parser (ArgumentHandler, optional) – Reference to the object in charge of parsing supplied pipeline parameters.

  • device (int, optional, defaults to -1) – Device ordinal for CPU/GPU supports. Setting this to -1 will leverage CPU, a positive will run the model on the associated CUDA device id.

  • binary_output (bool, optional, defaults to False) – Flag indicating if the output the pipeline should happen in a binary format (i.e., pickle) or as raw text.

__call__(text_inputs, **kwargs)[source]

Complete the prompt(s) given as inputs.

Parameters

  • args (str or List[str]) – One or several prompts (or one list of prompts) to complete.

  • return_tensors (bool, optional, defaults to False) – Whether or not to include the tensors of predictions (as token indices) in the outputs.

  • return_text (bool, optional, defaults to True) – Whether or not to include the decoded texts in the outputs.

  • return_full_text (bool, optional, defaults to True) – If set to False only added text is returned, otherwise the full text is returned Only meaningful if return_text is set to True.

  • clean_up_tokenization_spaces (bool, optional, defaults to False) – Whether or not to clean up the potential extra spaces in the text output.

  • prefix (str, optional) – Prefix added to prompt.

  • generate_kwargs – Additional keyword arguments to pass along to the generate method of the model (see the generate method corresponding to your framework here).

Returns

Each result comes as a dictionary with the following keys:

  • generated_text (str, present when return_text=True) – The generated text.

  • generated_token_ids (torch.Tensor or tf.Tensor, present when return_tensors=True) – The token ids of the generated text.

Return type

A list or a list of list of dict

postprocess(model_outputs, return_type=<ReturnType.FULL_TEXT: 2>, clean_up_tokenization_spaces=True)[source]

Postprocess will receive the raw outputs of the _forward method, generally tensors, and reformat them into something more friendly. Generally it will output a list or a dict or results (containing just strings and numbers).

preprocess(prompt_text, prefix='')[source]

Preprocess will take the input_ of a specific pipeline and return a dictionnary of everything necessary for _forward to run properly. It should contain at least one tensor, but might have arbitrary other items.

Text2TextGenerationPipeline

class transformers.Text2TextGenerationPipeline(*args, **kwargs)[source]

Pipeline for text to text generation using seq2seq models.

This Text2TextGenerationPipeline pipeline can currently be loaded from pipeline() using the following task identifier: "text2text-generation".

The models that this pipeline can use are models that have been fine-tuned on a translation task. See the up-to-date list of available models on huggingface.co/models.

Usage:

text2text_generator = pipeline("text2text-generation")
text2text_generator("question: What is 42 ? context: 42 is the answer to life, the universe and everything")
Parameters

  • model (PreTrainedModel or TFPreTrainedModel) – The model that will be used by the pipeline to make predictions. This needs to be a model inheriting from PreTrainedModel for PyTorch and TFPreTrainedModel for TensorFlow.

  • tokenizer (PreTrainedTokenizer) – The tokenizer that will be used by the pipeline to encode data for the model. This object inherits from PreTrainedTokenizer.

  • modelcard (str or ModelCard, optional) – Model card attributed to the model for this pipeline.

  • framework (str, optional) –

    The framework to use, either "pt" for PyTorch or "tf" for TensorFlow. The specified framework must be installed.

    If no framework is specified, will default to the one currently installed. If no framework is specified and both frameworks are installed, will default to the framework of the model, or to PyTorch if no model is provided.

  • task (str, defaults to "") – A task-identifier for the pipeline.

  • args_parser (ArgumentHandler, optional) – Reference to the object in charge of parsing supplied pipeline parameters.

  • device (int, optional, defaults to -1) – Device ordinal for CPU/GPU supports. Setting this to -1 will leverage CPU, a positive will run the model on the associated CUDA device id.

  • binary_output (bool, optional, defaults to False) – Flag indicating if the output the pipeline should happen in a binary format (i.e., pickle) or as raw text.

__call__(*args, **kwargs)[source]

Generate the output text(s) using text(s) given as inputs.

Parameters

  • args (str or List[str]) – Input text for the encoder.

  • return_tensors (bool, optional, defaults to False) – Whether or not to include the tensors of predictions (as token indices) in the outputs.

  • return_text (bool, optional, defaults to True) – Whether or not to include the decoded texts in the outputs.

  • clean_up_tokenization_spaces (bool, optional, defaults to False) – Whether or not to clean up the potential extra spaces in the text output.

  • truncation (TruncationStrategy, optional, defaults to TruncationStrategy.DO_NOT_TRUNCATE) – The truncation strategy for the tokenization within the pipeline. TruncationStrategy.DO_NOT_TRUNCATE (default) will never truncate, but it is sometimes desirable to truncate the input to fit the model’s max_length instead of throwing an error down the line.

  • generate_kwargs – Additional keyword arguments to pass along to the generate method of the model (see the generate method corresponding to your framework here).

Returns

Each result comes as a dictionary with the following keys:

  • generated_text (str, present when return_text=True) – The generated text.

  • generated_token_ids (torch.Tensor or tf.Tensor, present when return_tensors=True) – The token ids of the generated text.

Return type

A list or a list of list of dict

check_inputs(input_length: int, min_length: int, max_length: int)[source]

Checks whether there might be something wrong with given input with regard to the model.

postprocess(model_outputs, return_type=<ReturnType.TEXT: 1>, clean_up_tokenization_spaces=False)[source]

Postprocess will receive the raw outputs of the _forward method, generally tensors, and reformat them into something more friendly. Generally it will output a list or a dict or results (containing just strings and numbers).

preprocess(inputs, truncation=<TruncationStrategy.DO_NOT_TRUNCATE: 'do_not_truncate'>, **kwargs)[source]

Preprocess will take the input_ of a specific pipeline and return a dictionnary of everything necessary for _forward to run properly. It should contain at least one tensor, but might have arbitrary other items.

TokenClassificationPipeline

class transformers.TokenClassificationPipeline(args_parser=<transformers.pipelines.token_classification.TokenClassificationArgumentHandler object>, *args, **kwargs)[source]

Named Entity Recognition pipeline using any ModelForTokenClassification. See the named entity recognition examples for more information.

This token recognition pipeline can currently be loaded from pipeline() using the following task identifier: "ner" (for predicting the classes of tokens in a sequence: person, organisation, location or miscellaneous).

The models that this pipeline can use are models that have been fine-tuned on a token classification task. See the up-to-date list of available models on huggingface.co/models.

Parameters

  • model (PreTrainedModel or TFPreTrainedModel) – The model that will be used by the pipeline to make predictions. This needs to be a model inheriting from PreTrainedModel for PyTorch and TFPreTrainedModel for TensorFlow.

  • tokenizer (PreTrainedTokenizer) – The tokenizer that will be used by the pipeline to encode data for the model. This object inherits from PreTrainedTokenizer.

  • modelcard (str or ModelCard, optional) – Model card attributed to the model for this pipeline.

  • framework (str, optional) –

    The framework to use, either "pt" for PyTorch or "tf" for TensorFlow. The specified framework must be installed.

    If no framework is specified, will default to the one currently installed. If no framework is specified and both frameworks are installed, will default to the framework of the model, or to PyTorch if no model is provided.

  • task (str, defaults to "") – A task-identifier for the pipeline.

  • args_parser (ArgumentHandler, optional) – Reference to the object in charge of parsing supplied pipeline parameters.

  • device (int, optional, defaults to -1) – Device ordinal for CPU/GPU supports. Setting this to -1 will leverage CPU, a positive will run the model on the associated CUDA device id.

  • binary_output (bool, optional, defaults to False) – Flag indicating if the output the pipeline should happen in a binary format (i.e., pickle) or as raw text.

  • ignore_labels (List[str], defaults to ["O"]) – A list of labels to ignore.

  • grouped_entities (bool, optional, defaults to False) – DEPRECATED, use aggregation_strategy instead. Whether or not to group the tokens corresponding to the same entity together in the predictions or not.

  • aggregation_strategy (str, optional, defaults to "none") –

    The strategy to fuse (or not) tokens based on the model prediction.

    • ”none” : Will simply not do any aggregation and simply return raw results from the model

    • ”simple” : Will attempt to group entities following the default schema. (A, B-TAG), (B, I-TAG), (C, I-TAG), (D, B-TAG2) (E, B-TAG2) will end up being [{“word”: ABC, “entity”: “TAG”}, {“word”: “D”, “entity”: “TAG2”}, {“word”: “E”, “entity”: “TAG2”}] Notice that two consecutive B tags will end up as different entities. On word based languages, we might end up splitting words undesirably : Imagine Microsoft being tagged as [{“word”: “Micro”, “entity”: “ENTERPRISE”}, {“word”: “soft”, “entity”: “NAME”}]. Look for FIRST, MAX, AVERAGE for ways to mitigate that and disambiguate words (on languages that support that meaning, which is basically tokens separated by a space). These mitigations will only work on real words, “New york” might still be tagged with two different entities.

    • ”first” : (works only on word based models) Will use the SIMPLE strategy except that words, cannot end up with different tags. Words will simply use the tag of the first token of the word when there is ambiguity.

    • ”average” : (works only on word based models) Will use the SIMPLE strategy except that words, cannot end up with different tags. scores will be averaged first across tokens, and then the maximum label is applied.

    • ”max” : (works only on word based models) Will use the SIMPLE strategy except that words, cannot end up with different tags. Word entity will simply be the token with the maximum score.

__call__(inputs: Union[str, List[str]], **kwargs)[source]

Classify each token of the text(s) given as inputs.

Parameters

inputs (str or List[str]) – One or several texts (or one list of texts) for token classification.

Returns

Each result comes as a list of dictionaries (one for each token in the corresponding input, or each entity if this pipeline was instantiated with an aggregation_strategy) with the following keys:

  • word (str) – The token/word classified.

  • score (float) – The corresponding probability for entity.

  • entity (str) – The entity predicted for that token/word (it is named entity_group when aggregation_strategy is not "none".

  • index (int, only present when aggregation_strategy="none") – The index of the corresponding token in the sentence.

  • start (int, optional) – The index of the start of the corresponding entity in the sentence. Only exists if the offsets are available within the tokenizer

  • end (int, optional) – The index of the end of the corresponding entity in the sentence. Only exists if the offsets are available within the tokenizer

Return type

A list or a list of list of dict

aggregate_words(entities: List[dict], aggregation_strategy: transformers.pipelines.token_classification.AggregationStrategy) → List[dict][source]

Override tokens from a given word that disagree to force agreement on word boundaries.

Example: micro|soft| com|pany| B-ENT I-NAME I-ENT I-ENT will be rewritten with first strategy as microsoft| company| B-ENT I-ENT

gather_pre_entities(sentence: str, input_ids: numpy.ndarray, scores: numpy.ndarray, offset_mapping: Optional[List[Tuple[int, int]]], special_tokens_mask: numpy.ndarray, aggregation_strategy: transformers.pipelines.token_classification.AggregationStrategy) → List[dict][source]

Fuse various numpy arrays into dicts with all the information needed for aggregation

group_entities(entities: List[dict]) → List[dict][source]

Find and group together the adjacent tokens with the same entity predicted.

Parameters

entities (dict) – The entities predicted by the pipeline.

group_sub_entities(entities: List[dict]) → dict[source]

Group together the adjacent tokens with the same entity predicted.

Parameters

entities (dict) – The entities predicted by the pipeline.

postprocess(model_outputs, aggregation_strategy=<AggregationStrategy.NONE: 'none'>)[source]

Postprocess will receive the raw outputs of the _forward method, generally tensors, and reformat them into something more friendly. Generally it will output a list or a dict or results (containing just strings and numbers).

preprocess(sentence)[source]

Preprocess will take the input_ of a specific pipeline and return a dictionnary of everything necessary for _forward to run properly. It should contain at least one tensor, but might have arbitrary other items.

TranslationPipeline

class transformers.TranslationPipeline(*args, **kwargs)[source]

Translates from one language to another.

This translation pipeline can currently be loaded from pipeline() using the following task identifier: "translation_xx_to_yy".

The models that this pipeline can use are models that have been fine-tuned on a translation task. See the up-to-date list of available models on huggingface.co/models.

Usage::

en_fr_translator = pipeline(“translation_en_to_fr”) en_fr_translator(“How old are you?”)

Parameters

  • model (PreTrainedModel or TFPreTrainedModel) – The model that will be used by the pipeline to make predictions. This needs to be a model inheriting from PreTrainedModel for PyTorch and TFPreTrainedModel for TensorFlow.

  • tokenizer (PreTrainedTokenizer) – The tokenizer that will be used by the pipeline to encode data for the model. This object inherits from PreTrainedTokenizer.

  • modelcard (str or ModelCard, optional) – Model card attributed to the model for this pipeline.

  • framework (str, optional) –

    The framework to use, either "pt" for PyTorch or "tf" for TensorFlow. The specified framework must be installed.

    If no framework is specified, will default to the one currently installed. If no framework is specified and both frameworks are installed, will default to the framework of the model, or to PyTorch if no model is provided.

  • task (str, defaults to "") – A task-identifier for the pipeline.

  • args_parser (ArgumentHandler, optional) – Reference to the object in charge of parsing supplied pipeline parameters.

  • device (int, optional, defaults to -1) – Device ordinal for CPU/GPU supports. Setting this to -1 will leverage CPU, a positive will run the model on the associated CUDA device id.

  • binary_output (bool, optional, defaults to False) – Flag indicating if the output the pipeline should happen in a binary format (i.e., pickle) or as raw text.

__call__(*args, **kwargs)[source]

Translate the text(s) given as inputs.

Parameters

  • args (str or List[str]) – Texts to be translated.

  • return_tensors (bool, optional, defaults to False) – Whether or not to include the tensors of predictions (as token indices) in the outputs.

  • return_text (bool, optional, defaults to True) – Whether or not to include the decoded texts in the outputs.

  • clean_up_tokenization_spaces (bool, optional, defaults to False) – Whether or not to clean up the potential extra spaces in the text output.

  • src_lang (str, optional) – The language of the input. Might be required for multilingual models. Will not have any effect for single pair translation models

  • tgt_lang (str, optional) – The language of the desired output. Might be required for multilingual models. Will not have any effect for single pair translation models

  • generate_kwargs – Additional keyword arguments to pass along to the generate method of the model (see the generate method corresponding to your framework here).

Returns

Each result comes as a dictionary with the following keys:

  • translation_text (str, present when return_text=True) – The translation.

  • translation_token_ids (torch.Tensor or tf.Tensor, present when return_tensors=True) – The token ids of the translation.

Return type

A list or a list of list of dict

check_inputs(input_length: int, min_length: int, max_length: int)[source]

Checks whether there might be something wrong with given input with regard to the model.

preprocess(*args, truncation=<TruncationStrategy.DO_NOT_TRUNCATE: 'do_not_truncate'>, src_lang=None, tgt_lang=None)[source]

Preprocess will take the input_ of a specific pipeline and return a dictionnary of everything necessary for _forward to run properly. It should contain at least one tensor, but might have arbitrary other items.

ZeroShotClassificationPipeline

class transformers.ZeroShotClassificationPipeline(args_parser=<transformers.pipelines.zero_shot_classification.ZeroShotClassificationArgumentHandler object>, *args, **kwargs)[source]

NLI-based zero-shot classification pipeline using a ModelForSequenceClassification trained on NLI (natural language inference) tasks.

Any combination of sequences and labels can be passed and each combination will be posed as a premise/hypothesis pair and passed to the pretrained model. Then, the logit for entailment is taken as the logit for the candidate label being valid. Any NLI model can be used, but the id of the entailment label must be included in the model config’s label2id.

This NLI pipeline can currently be loaded from pipeline() using the following task identifier: "zero-shot-classification".

The models that this pipeline can use are models that have been fine-tuned on an NLI task. See the up-to-date list of available models on huggingface.co/models.

Parameters

  • model (PreTrainedModel or TFPreTrainedModel) – The model that will be used by the pipeline to make predictions. This needs to be a model inheriting from PreTrainedModel for PyTorch and TFPreTrainedModel for TensorFlow.

  • tokenizer (PreTrainedTokenizer) – The tokenizer that will be used by the pipeline to encode data for the model. This object inherits from PreTrainedTokenizer.

  • modelcard (str or ModelCard, optional) – Model card attributed to the model for this pipeline.

  • framework (str, optional) –

    The framework to use, either "pt" for PyTorch or "tf" for TensorFlow. The specified framework must be installed.

    If no framework is specified, will default to the one currently installed. If no framework is specified and both frameworks are installed, will default to the framework of the model, or to PyTorch if no model is provided.

  • task (str, defaults to "") – A task-identifier for the pipeline.

  • args_parser (ArgumentHandler, optional) – Reference to the object in charge of parsing supplied pipeline parameters.

  • device (int, optional, defaults to -1) – Device ordinal for CPU/GPU supports. Setting this to -1 will leverage CPU, a positive will run the model on the associated CUDA device id.

  • binary_output (bool, optional, defaults to False) – Flag indicating if the output the pipeline should happen in a binary format (i.e., pickle) or as raw text.

__call__(sequences: Union[str, List[str]], *args, **kwargs)[source]

Classify the sequence(s) given as inputs. See the ZeroShotClassificationPipeline documentation for more information.

Parameters

  • sequences (str or List[str]) – The sequence(s) to classify, will be truncated if the model input is too large.

  • candidate_labels (str or List[str]) – The set of possible class labels to classify each sequence into. Can be a single label, a string of comma-separated labels, or a list of labels.

  • hypothesis_template (str, optional, defaults to "This example is {}.") – The template used to turn each label into an NLI-style hypothesis. This template must include a {} or similar syntax for the candidate label to be inserted into the template. For example, the default template is "This example is {}." With the candidate label "sports", this would be fed into the model like "<cls> sequence to classify <sep> This example is sports . <sep>". The default template works well in many cases, but it may be worthwhile to experiment with different templates depending on the task setting.

  • multi_label (bool, optional, defaults to False) – Whether or not multiple candidate labels can be true. If False, the scores are normalized such that the sum of the label likelihoods for each sequence is 1. If True, the labels are considered independent and probabilities are normalized for each candidate by doing a softmax of the entailment score vs. the contradiction score.

Returns

Each result comes as a dictionary with the following keys:

  • sequence (str) – The sequence for which this is the output.

  • labels (List[str]) – The labels sorted by order of likelihood.

  • scores (List[float]) – The probabilities for each of the labels.

Return type

A dict or a list of dict

postprocess(model_outputs, multi_label=False)[source]

Postprocess will receive the raw outputs of the _forward method, generally tensors, and reformat them into something more friendly. Generally it will output a list or a dict or results (containing just strings and numbers).

preprocess(inputs, candidate_labels=None, hypothesis_template='This example is {}.')[source]

Preprocess will take the input_ of a specific pipeline and return a dictionnary of everything necessary for _forward to run properly. It should contain at least one tensor, but might have arbitrary other items.

Parent class: Pipeline

class transformers.Pipeline(model: Union[PreTrainedModel, TFPreTrainedModel], tokenizer: Optional[transformers.tokenization_utils.PreTrainedTokenizer] = None, feature_extractor: Optional[SequenceFeatureExtractor] = None, modelcard: Optional[transformers.modelcard.ModelCard] = None, framework: Optional[str] = None, task: str = '', args_parser: transformers.pipelines.base.ArgumentHandler = None, device: int = - 1, binary_output: bool = False, **kwargs)[source]

The Pipeline class is the class from which all pipelines inherit. Refer to this class for methods shared across different pipelines.

Base class implementing pipelined operations. Pipeline workflow is defined as a sequence of the following operations:

Input -> Tokenization -> Model Inference -> Post-Processing (task dependent) -> Output

Pipeline supports running on CPU or GPU through the device argument (see below).

Some pipeline, like for instance FeatureExtractionPipeline ('feature-extraction' ) output large tensor object as nested-lists. In order to avoid dumping such large structure as textual data we provide the binary_output constructor argument. If set to True, the output will be stored in the pickle format.

Parameters

  • model (PreTrainedModel or TFPreTrainedModel) – The model that will be used by the pipeline to make predictions. This needs to be a model inheriting from PreTrainedModel for PyTorch and TFPreTrainedModel for TensorFlow.

  • tokenizer (PreTrainedTokenizer) – The tokenizer that will be used by the pipeline to encode data for the model. This object inherits from PreTrainedTokenizer.

  • modelcard (str or ModelCard, optional) – Model card attributed to the model for this pipeline.

  • framework (str, optional) –

    The framework to use, either "pt" for PyTorch or "tf" for TensorFlow. The specified framework must be installed.

    If no framework is specified, will default to the one currently installed. If no framework is specified and both frameworks are installed, will default to the framework of the model, or to PyTorch if no model is provided.

  • task (str, defaults to "") – A task-identifier for the pipeline.

  • args_parser (ArgumentHandler, optional) – Reference to the object in charge of parsing supplied pipeline parameters.

  • device (int, optional, defaults to -1) – Device ordinal for CPU/GPU supports. Setting this to -1 will leverage CPU, a positive will run the model on the associated CUDA device id.

  • binary_output (bool, optional, defaults to False) – Flag indicating if the output the pipeline should happen in a binary format (i.e., pickle) or as raw text.

check_model_type(supported_models: Union[List[str], dict])[source]

Check if the model class is in supported by the pipeline.

Parameters

supported_models (List[str] or dict) – The list of models supported by the pipeline, or a dictionary with model class values.

device_placement()[source]

Context Manager allowing tensor allocation on the user-specified device in framework agnostic way.

Returns

Context manager

Examples:

# Explicitly ask for tensor allocation on CUDA device :0
pipe = pipeline(..., device=0)
with pipe.device_placement():
    # Every framework specific tensor allocation will be done on the request device
    output = pipe(...)
ensure_tensor_on_device(**inputs)[source]

Ensure PyTorch tensors are on the specified device.

Parameters

  • inputs (keyword arguments that should be torch.Tensor, the rest is ignored) – The tensors to place on self.device.

  • on lists only. (Recursive) –

Returns

The same as inputs but on the proper device.

Return type

Dict[str, torch.Tensor]

abstract postprocess(model_outputs: transformers.file_utils.ModelOutput, **postprocess_parameters: Dict) → Any[source]

Postprocess will receive the raw outputs of the _forward method, generally tensors, and reformat them into something more friendly. Generally it will output a list or a dict or results (containing just strings and numbers).

predict(X)[source]

Scikit / Keras interface to transformers’ pipelines. This method will forward to __call__().

abstract preprocess(input_: Any, **preprocess_parameters: Dict) → Dict[str, Union[List[Union[List[GenericTensor], torch.Tensor, tf.Tensor]], torch.Tensor, tensorflow.python.framework.ops.Tensor]][source]

Preprocess will take the input_ of a specific pipeline and return a dictionnary of everything necessary for _forward to run properly. It should contain at least one tensor, but might have arbitrary other items.

save_pretrained(save_directory: str)[source]

Save the pipeline’s model and tokenizer.

Parameters

save_directory (str) – A path to the directory where to saved. It will be created if it doesn’t exist.

transform(X)[source]

Scikit / Keras interface to transformers’ pipelines. This method will forward to __call__().