Hugging Face's logo

Transformers documentation

파이프라인을 위한 유틸리티

Hugging Face's logo
Join the Hugging Face community

and get access to the augmented documentation experience

to get started

파이프라인을 위한 유틸리티

이 페이지는 라이브러리에서 파이프라인을 위해 제공하는 모든 유틸리티 함수들을 나열합니다.

이 함수들 대부분은 라이브러리 내 모델의 코드를 연구할 때만 유용합니다.

인자 처리

class transformers.pipelines.ArgumentHandler

< >

( )

Base interface for handling arguments for each Pipeline.

class transformers.pipelines.ZeroShotClassificationArgumentHandler

< >

( )

Handles arguments for zero-shot for text classification by turning each possible label into an NLI premise/hypothesis pair.

class transformers.pipelines.QuestionAnsweringArgumentHandler

< >

( )

QuestionAnsweringPipeline requires the user to provide multiple arguments (i.e. question & context) to be mapped to internal SquadExample.

QuestionAnsweringArgumentHandler manages all the possible to create a SquadExample from the command-line supplied arguments.

데이터 형식

class transformers.PipelineDataFormat

< >

( output_path: typing.Optional[str] input_path: typing.Optional[str] column: typing.Optional[str] overwrite: bool = False )

Parameters

  • output_path (str) — Where to save the outgoing data.
  • input_path (str) — Where to look for the input data.
  • column (str) — The column to read.
  • overwrite (bool, optional, defaults to False) — Whether or not to overwrite the output_path.

Base class for all the pipeline supported data format both for reading and writing. Supported data formats currently includes:

  • JSON
  • CSV
  • stdin/stdout (pipe)

PipelineDataFormat also includes some utilities to work with multi-columns like mapping from datasets columns to pipelines keyword arguments through the dataset_kwarg_1=dataset_column_1 format.

from_str

< >

( format: str output_path: typing.Optional[str] input_path: typing.Optional[str] column: typing.Optional[str] overwrite = False ) → PipelineDataFormat

Parameters

  • format (str) — The format of the desired pipeline. Acceptable values are "json", "csv" or "pipe".
  • output_path (str, optional) — Where to save the outgoing data.
  • input_path (str, optional) — Where to look for the input data.
  • column (str, optional) — The column to read.
  • overwrite (bool, optional, defaults to False) — Whether or not to overwrite the output_path.

Returns

PipelineDataFormat

The proper data format.

Creates an instance of the right subclass of PipelineDataFormat depending on format.

save

< >

( data: typing.Union[dict, typing.List[dict]] )

Parameters

  • data (dict or list of dict) — The data to store.

Save the provided data object with the representation for the current PipelineDataFormat.

save_binary

< >

( data: typing.Union[dict, typing.List[dict]] ) → str

Parameters

  • data (dict or list of dict) — The data to store.

Returns

str

Path where the data has been saved.

Save the provided data object as a pickle-formatted binary data on the disk.

class transformers.CsvPipelineDataFormat

< >

( output_path: typing.Optional[str] input_path: typing.Optional[str] column: typing.Optional[str] overwrite = False )

Parameters

  • output_path (str) — Where to save the outgoing data.
  • input_path (str) — Where to look for the input data.
  • column (str) — The column to read.
  • overwrite (bool, optional, defaults to False) — Whether or not to overwrite the output_path.

Support for pipelines using CSV data format.

save

< >

( data: typing.List[dict] )

Parameters

  • data (List[dict]) — The data to store.

Save the provided data object with the representation for the current PipelineDataFormat.

class transformers.JsonPipelineDataFormat

< >

( output_path: typing.Optional[str] input_path: typing.Optional[str] column: typing.Optional[str] overwrite = False )

Parameters

  • output_path (str) — Where to save the outgoing data.
  • input_path (str) — Where to look for the input data.
  • column (str) — The column to read.
  • overwrite (bool, optional, defaults to False) — Whether or not to overwrite the output_path.

Support for pipelines using JSON file format.

save

< >

( data: dict )

Parameters

  • data (dict) — The data to store.

Save the provided data object in a json file.

class transformers.PipedPipelineDataFormat

< >

( output_path: typing.Optional[str] input_path: typing.Optional[str] column: typing.Optional[str] overwrite: bool = False )

Parameters

  • output_path (str) — Where to save the outgoing data.
  • input_path (str) — Where to look for the input data.
  • column (str) — The column to read.
  • overwrite (bool, optional, defaults to False) — Whether or not to overwrite the output_path.

Read data from piped input to the python process. For multi columns data, columns should separated by

If columns are provided, then the output will be a dictionary with {column_x: value_x}

save

< >

( data: dict )

Parameters

  • data (dict) — The data to store.

Print the data.

유틸리티

class transformers.pipelines.PipelineException

< >

( task: str model: str reason: str )

Parameters

  • task (str) — The task of the pipeline.
  • model (str) — The model used by the pipeline.
  • reason (str) — The error message to display.

Raised by a Pipeline when handling call.

< > Update on GitHub

←사용자 정의 레이어 및 유틸리티 토크나이저를 위한 유틸리티→