Hugging Face's logo

Transformers documentation

pipelines的工具

You are viewing v4.38.0 version. A newer version v4.47.1 is available.
Hugging Face's logo
Join the Hugging Face community

and get access to the augmented documentation experience

to get started

pipelines的工具

此页面列出了库为pipelines提供的所有实用程序功能。

其中大多数只有在您研究库中模型的代码时才有用。

参数处理

class transformers.pipelines.ArgumentHandler

< >

( )

Base interface for handling arguments for each Pipeline.

class transformers.pipelines.ZeroShotClassificationArgumentHandler

< >

( )

Handles arguments for zero-shot for text classification by turning each possible label into an NLI premise/hypothesis pair.

class transformers.pipelines.QuestionAnsweringArgumentHandler

< >

( )

QuestionAnsweringPipeline requires the user to provide multiple arguments (i.e. question & context) to be mapped to internal SquadExample.

QuestionAnsweringArgumentHandler manages all the possible to create a SquadExample from the command-line supplied arguments.

数据格式

class transformers.PipelineDataFormat

< >

( output_path: Optional input_path: Optional column: Optional overwrite: bool = False )

Parameters

  • output_path (str) — Where to save the outgoing data.
  • input_path (str) — Where to look for the input data.
  • column (str) — The column to read.
  • overwrite (bool, optional, defaults to False) — Whether or not to overwrite the output_path.

Base class for all the pipeline supported data format both for reading and writing. Supported data formats currently includes:

  • JSON
  • CSV
  • stdin/stdout (pipe)

PipelineDataFormat also includes some utilities to work with multi-columns like mapping from datasets columns to pipelines keyword arguments through the dataset_kwarg_1=dataset_column_1 format.

from_str

< >

( format: str output_path: Optional input_path: Optional column: Optional overwrite = False ) → PipelineDataFormat

Parameters

  • format (str) — The format of the desired pipeline. Acceptable values are "json", "csv" or "pipe".
  • output_path (str, optional) — Where to save the outgoing data.
  • input_path (str, optional) — Where to look for the input data.
  • column (str, optional) — The column to read.
  • overwrite (bool, optional, defaults to False) — Whether or not to overwrite the output_path.

Returns

PipelineDataFormat

The proper data format.

Creates an instance of the right subclass of PipelineDataFormat depending on format.

save

< >

( data: Union )

Parameters

  • data (dict or list of dict) — The data to store.

Save the provided data object with the representation for the current PipelineDataFormat.

save_binary

< >

( data: Union ) → str

Parameters

  • data (dict or list of dict) — The data to store.

Returns

str

Path where the data has been saved.

Save the provided data object as a pickle-formatted binary data on the disk.

class transformers.CsvPipelineDataFormat

< >

( output_path: Optional input_path: Optional column: Optional overwrite = False )

Parameters

  • output_path (str) — Where to save the outgoing data.
  • input_path (str) — Where to look for the input data.
  • column (str) — The column to read.
  • overwrite (bool, optional, defaults to False) — Whether or not to overwrite the output_path.

Support for pipelines using CSV data format.

save

< >

( data: List )

Parameters

  • data (List[dict]) — The data to store.

Save the provided data object with the representation for the current PipelineDataFormat.

class transformers.JsonPipelineDataFormat

< >

( output_path: Optional input_path: Optional column: Optional overwrite = False )

Parameters

  • output_path (str) — Where to save the outgoing data.
  • input_path (str) — Where to look for the input data.
  • column (str) — The column to read.
  • overwrite (bool, optional, defaults to False) — Whether or not to overwrite the output_path.

Support for pipelines using JSON file format.

save

< >

( data: dict )

Parameters

  • data (dict) — The data to store.

Save the provided data object in a json file.

class transformers.PipedPipelineDataFormat

< >

( output_path: Optional input_path: Optional column: Optional overwrite: bool = False )

Parameters

  • output_path (str) — Where to save the outgoing data.
  • input_path (str) — Where to look for the input data.
  • column (str) — The column to read.
  • overwrite (bool, optional, defaults to False) — Whether or not to overwrite the output_path.

Read data from piped input to the python process. For multi columns data, columns should separated by

If columns are provided, then the output will be a dictionary with {column_x: value_x}

save

< >

( data: dict )

Parameters

  • data (dict) — The data to store.

Print the data.

实用函数

class transformers.pipelines.PipelineException

< >

( task: str model: str reason: str )

Parameters

  • task (str) — The task of the pipeline.
  • model (str) — The model used by the pipeline.
  • reason (str) — The error message to display.

Raised by a Pipeline when handling call.

←自定义层和工具 Tokenizers工具→