Hugging Face's logo

Transformers documentation

파이프라인을 위한 유틸리티

You are viewing main version, which requires installation from source. If you'd like regular pip install, checkout the latest stable version (v4.46.3).
Hugging Face's logo
Join the Hugging Face community

and get access to the augmented documentation experience

to get started

파이프라인을 위한 유틸리티

이 페이지는 라이브러리에서 파이프라인을 위해 제공하는 모든 유틸리티 함수들을 나열합니다.

이 함수들 대부분은 라이브러리 내 모델의 코드를 연구할 때만 유용합니다.

인자 처리

class transformers.pipelines.ArgumentHandler

< >

( )

Base interface for handling arguments for each Pipeline.

class transformers.pipelines.ZeroShotClassificationArgumentHandler

< >

( )

Handles arguments for zero-shot for text classification by turning each possible label into an NLI premise/hypothesis pair.

class transformers.pipelines.QuestionAnsweringArgumentHandler

< >

( )

QuestionAnsweringPipeline requires the user to provide multiple arguments (i.e. question & context) to be mapped to internal SquadExample.

QuestionAnsweringArgumentHandler manages all the possible to create a SquadExample from the command-line supplied arguments.

데이터 형식

class transformers.PipelineDataFormat

< >

( output_path: typing.Optional[str] input_path: typing.Optional[str] column: typing.Optional[str] overwrite: bool = False )

Parameters

  • output_path (str) — Where to save the outgoing data.
  • input_path (str) — Where to look for the input data.
  • column (str) — The column to read.
  • overwrite (bool, optional, defaults to False) — Whether or not to overwrite the output_path.

Base class for all the pipeline supported data format both for reading and writing. Supported data formats currently includes:

  • JSON
  • CSV
  • stdin/stdout (pipe)

PipelineDataFormat also includes some utilities to work with multi-columns like mapping from datasets columns to pipelines keyword arguments through the dataset_kwarg_1=dataset_column_1 format.

from_str

< >

( format: str output_path: typing.Optional[str] input_path: typing.Optional[str] column: typing.Optional[str] overwrite = False ) → PipelineDataFormat

Parameters

  • format (str) — The format of the desired pipeline. Acceptable values are "json", "csv" or "pipe".
  • output_path (str, optional) — Where to save the outgoing data.
  • input_path (str, optional) — Where to look for the input data.
  • column (str, optional) — The column to read.
  • overwrite (bool, optional, defaults to False) — Whether or not to overwrite the output_path.

Returns

PipelineDataFormat

The proper data format.

Creates an instance of the right subclass of PipelineDataFormat depending on format.

save

< >

( data: typing.Union[dict, typing.List[dict]] )

Parameters

  • data (dict or list of dict) — The data to store.

Save the provided data object with the representation for the current PipelineDataFormat.

save_binary

< >

( data: typing.Union[dict, typing.List[dict]] ) → str

Parameters

  • data (dict or list of dict) — The data to store.

Returns

str

Path where the data has been saved.

Save the provided data object as a pickle-formatted binary data on the disk.

class transformers.CsvPipelineDataFormat

< >

( output_path: typing.Optional[str] input_path: typing.Optional[str] column: typing.Optional[str] overwrite = False )

Parameters

  • output_path (str) — Where to save the outgoing data.
  • input_path (str) — Where to look for the input data.
  • column (str) — The column to read.
  • overwrite (bool, optional, defaults to False) — Whether or not to overwrite the output_path.

Support for pipelines using CSV data format.

save

< >

( data: typing.List[dict] )

Parameters

  • data (List[dict]) — The data to store.

Save the provided data object with the representation for the current PipelineDataFormat.

class transformers.JsonPipelineDataFormat

< >

( output_path: typing.Optional[str] input_path: typing.Optional[str] column: typing.Optional[str] overwrite = False )

Parameters

  • output_path (str) — Where to save the outgoing data.
  • input_path (str) — Where to look for the input data.
  • column (str) — The column to read.
  • overwrite (bool, optional, defaults to False) — Whether or not to overwrite the output_path.

Support for pipelines using JSON file format.

save

< >

( data: dict )

Parameters

  • data (dict) — The data to store.

Save the provided data object in a json file.

class transformers.PipedPipelineDataFormat

< >

( output_path: typing.Optional[str] input_path: typing.Optional[str] column: typing.Optional[str] overwrite: bool = False )

Parameters

  • output_path (str) — Where to save the outgoing data.
  • input_path (str) — Where to look for the input data.
  • column (str) — The column to read.
  • overwrite (bool, optional, defaults to False) — Whether or not to overwrite the output_path.

Read data from piped input to the python process. For multi columns data, columns should separated by

If columns are provided, then the output will be a dictionary with {column_x: value_x}

save

< >

( data: dict )

Parameters

  • data (dict) — The data to store.

Print the data.

유틸리티

class transformers.pipelines.PipelineException

< >

( task: str model: str reason: str )

Parameters

  • task (str) — The task of the pipeline.
  • model (str) — The model used by the pipeline.
  • reason (str) — The error message to display.

Raised by a Pipeline when handling call.

< > Update on GitHub

←사용자 정의 레이어 및 유틸리티 토크나이저를 위한 유틸리티→