Hugging Face's logo

Optimum documentation

Dummy Input Generators

You are viewing main version, which requires installation from source. If you'd like regular pip install, checkout the latest stable version (v1.23.3).
Hugging Face's logo
Join the Hugging Face community

and get access to the augmented documentation experience

to get started

Dummy Input Generators

It is very common to have to generate dummy inputs to perform a task (tracing, exporting a model to some backend, testing model outputs, etc). The goal of DummyInputGenerator classes is to make this generation easy and re-usable.

Base class

class optimum.utils.DummyInputGenerator

< >

( )

Generates dummy inputs for the supported input names, in the requested framework.

concat_inputs

< >

( inputs dim: int )

Parameters

  • inputs — The list of tensors in a given framework to concatenate.
  • dim (int) — The dimension along which to concatenate.

Concatenates inputs together.

constant_tensor

< >

( shape: typing.List[int] value: typing.Union[int, float] = 1 dtype: typing.Optional[typing.Any] = None framework: str = 'pt' )

Parameters

  • shape (List[int]) — The shape of the constant tensor.
  • value (Union[int, float], defaults to 1) — The value to fill the constant tensor with.
  • dtype (Optional[Any], defaults to None) — The dtype of the constant tensor.
  • framework (str, defaults to "pt") — The requested framework.

Generates a constant tensor.

generate

< >

( input_name: str framework: str = 'pt' int_dtype: str = 'int64' float_dtype: str = 'fp32' )

Parameters

  • input_name (str) — The name of the input to generate.
  • framework (str, defaults to "pt") — The requested framework.
  • int_dtype (str, defaults to "int64") — The dtypes of generated integer tensors.
  • float_dtype (str, defaults to "fp32") — The dtypes of generated float tensors.

Generates the dummy input matching input_name for the requested framework.

pad_input_on_dim

< >

( input_ dim: int desired_length: typing.Optional[int] = None padding_length: typing.Optional[int] = None value: typing.Union[int, float] = 1 dtype: typing.Optional[typing.Any] = None )

Parameters

  • input_ — The tensor to pad.
  • dim (int) — The dimension along which to pad.
  • desired_length (Optional[int], defaults to None) — The desired length along the dimension after padding.
  • padding_length (Optional[int], defaults to None) — The length to pad along the dimension.
  • value (Union[int, float], defaults to 1) — The value to use for padding.
  • dtype (Optional[Any], defaults to None) — The dtype of the padding.

Pads an input either to the desired length, or by a padding length.

random_float_tensor

< >

( shape: typing.List[int] min_value: float = 0 max_value: float = 1 framework: str = 'pt' dtype: str = 'fp32' )

Parameters

  • shape (List[int]) — The shape of the random tensor.
  • min_value (float, defaults to 0) — The minimum value allowed.
  • max_value (float, defaults to 1) — The maximum value allowed.
  • framework (str, defaults to "pt") — The requested framework.
  • dtype (str, defaults to "fp32") — The dtype of the generated float tensor. Could be “fp32”, “fp16”, “bf16”.

Generates a tensor of random floats in the [min_value, max_value) range.

random_int_tensor

< >

( shape: typing.List[int] max_value: int min_value: int = 0 framework: str = 'pt' dtype: str = 'int64' )

Parameters

  • shape (List[int]) — The shape of the random tensor.
  • max_value (int) — The maximum value allowed.
  • min_value (int, defaults to 0) — The minimum value allowed.
  • framework (str, defaults to "pt") — The requested framework.
  • dtype (str, defaults to "int64") — The dtype of the generated integer tensor. Could be “int64”, “int32”, “int8”.

Generates a tensor of random integers in the [min_value, max_value) range.

random_mask_tensor

< >

( shape: typing.List[int] padding_side: str = 'right' framework: str = 'pt' dtype: str = 'int64' )

Parameters

  • shape (List[int]) — The shape of the random tensor.
  • padding_side (str, defaults to “right”) — The side on which the padding is applied.
  • framework (str, defaults to "pt") — The requested framework.
  • dtype (str, defaults to "int64") — The dtype of the generated integer tensor. Could be “int64”, “int32”, “int8”.

Generates a mask tensor either right or left padded.

supports_input

< >

( input_name: str ) → bool

Parameters

  • input_name (str) — The name of the input to generate.

Returns

bool

A boolean specifying whether the input is supported.

Checks whether the DummyInputGenerator supports the generation of the requested input.

Existing dummy input generators

class optimum.utils.DummyTextInputGenerator

< >

( task: str normalized_config: NormalizedTextConfig batch_size: int = 2 sequence_length: int = 16 num_choices: int = 4 random_batch_size_range: typing.Optional[typing.Tuple[int, int]] = None random_sequence_length_range: typing.Optional[typing.Tuple[int, int]] = None random_num_choices_range: typing.Optional[typing.Tuple[int, int]] = None padding_side: str = 'right' **kwargs )

Generates dummy encoder text inputs.

class optimum.utils.DummyDecoderTextInputGenerator

< >

( task: str normalized_config: NormalizedTextConfig batch_size: int = 2 sequence_length: int = 16 num_choices: int = 4 random_batch_size_range: typing.Optional[typing.Tuple[int, int]] = None random_sequence_length_range: typing.Optional[typing.Tuple[int, int]] = None random_num_choices_range: typing.Optional[typing.Tuple[int, int]] = None padding_side: str = 'right' **kwargs )

Generates dummy decoder text inputs.

class optimum.utils.DummyPastKeyValuesGenerator

< >

( task: str normalized_config: NormalizedTextConfig batch_size: int = 2 sequence_length: int = 16 random_batch_size_range: typing.Optional[typing.Tuple[int, int]] = None random_sequence_length_range: typing.Optional[typing.Tuple[int, int]] = None **kwargs )

Generates dummy past_key_values inputs.

class optimum.utils.DummySeq2SeqPastKeyValuesGenerator

< >

( task: str normalized_config: typing.Union[optimum.utils.normalized_config.NormalizedSeq2SeqConfig, optimum.utils.normalized_config.NormalizedEncoderDecoderConfig] batch_size: int = 2 sequence_length: int = 16 encoder_sequence_length: typing.Optional[int] = None random_batch_size_range: typing.Optional[typing.Tuple[int, int]] = None random_sequence_length_range: typing.Optional[typing.Tuple[int, int]] = None **kwargs )

Generates dummy past_key_values inputs for seq2seq architectures.

class optimum.utils.DummyBboxInputGenerator

< >

( task: str normalized_config: NormalizedConfig batch_size: int = 2 sequence_length: int = 16 random_batch_size_range: typing.Optional[typing.Tuple[int, int]] = None random_sequence_length_range: typing.Optional[typing.Tuple[int, int]] = None **kwargs )

Generates dummy bbox inputs.

class optimum.utils.DummyVisionInputGenerator

< >

( task: str normalized_config: NormalizedVisionConfig batch_size: int = 2 num_channels: int = 3 width: int = 64 height: int = 64 **kwargs )

Generates dummy vision inputs.

class optimum.utils.DummyAudioInputGenerator

< >

( task: str normalized_config: NormalizedConfig batch_size: int = 2 feature_size: int = 80 nb_max_frames: int = 3000 audio_sequence_length: int = 16000 **kwargs )

< > Update on GitHub

←GPTQ quantization Normalized configurations→