Hugging Face's logo

Optimum documentation

Configuration

You are viewing main version, which requires installation from source. If you'd like regular pip install, checkout the latest stable version (v1.23.3).
Hugging Face's logo
Join the Hugging Face community

and get access to the augmented documentation experience

to get started

Configuration

The configuration classes are the way to specify how a task should be done. Here is how a quantization can be configered:

  1. Quantization: Performed by the ~furiosa.FuriosaQuantizer, quantization can be set using a ~furiosa.configuration.QuantizationConfig. A calibration step is required in some cases (post training static quantization), which can be specified using a ~furiosa.configuration.CalibrationConfig.

QuantizationConfig

class optimum.furiosa.QuantizationConfig

< >

( with_quantize: bool = True normalized_pixel_outputs: typing.Sequence[int] = None )

Parameters

  • with_quantize (bool, defaults to True) — WWhether to put a Quantize operator at the beginning of the resulting model.
  • normalized_pixel_outputs ( Sequence[int, defaults to None) —: A sequence of indices of output tensors in the ONNX model that produce pixel values in a normalized format ranging from 0.0 to 1.0. If specified, the corresponding output tensors in the resulting quantized model will generate pixel values in an unnormalized format from 0 to 255, represented as unsigned 8-bit integers (uint8).

QuantizationConfig is the configuration class handling all the FuriosaAI quantization parameters.

CalibrationConfig

class optimum.furiosa.CalibrationConfig

< >

( dataset_name: str dataset_config_name: str dataset_split: str dataset_num_samples: int method: CalibrationMethod percentage: typing.Optional[float] = None )

Parameters

  • dataset_name (str) — The name of the calibration dataset.
  • dataset_config_name (str) — The name of the calibration dataset configuration.
  • dataset_split (str) — Which split of the dataset is used to perform the calibration step.
  • dataset_num_samples (int) — The number of samples composing the calibration dataset.
  • method (CalibrationMethod) — The method chosen to calculate the activations quantization parameters using the calibration dataset.
  • percentage (Optional[float], defaults to None) — The percentage to use when computing the activations quantization ranges when performing the calibration step using the Percentile method.

CalibrationConfig is the configuration class handling all the FurioaAI parameters related to the calibration step of static quantization.

FuriosaConfig

class optimum.furiosa.FuriosaAIConfig

< >

( opset: typing.Optional[int] = None quantization: typing.Optional[optimum.furiosa.configuration.QuantizationConfig] = None calibration: typing.Optional[optimum.furiosa.configuration.CalibrationConfig] = None **kwargs )

←Models Quantization→