neulab/docprompting-conala huggingface dataset:
class torch.quantization.ObserverBase(dtype) [source]
 
Base observer Module. Any observer implementation should derive from this class. Concrete observers should follow the same API. In forward, they will update the statistics of the observed Tensor. And they should provide a calculate_qparams function that computes the quantization parameters given the collected statistics.  Parameters 
dtype – Quantized data type    
classmethod with_args(**kwargs)  
Wrapper that allows creation of class factories. This can be useful when there is a need to create classes with the same constructor arguments, but different instances. Example: >>> Foo.with_args = classmethod(_with_args)
>>> foo_builder = Foo.with_args(a=3, b=4).with_args(answer=42)
>>> foo_instance1 = foo_builder()
>>> foo_instance2 = foo_builder()
>>> id(foo_instance1) == id(foo_instance2)
False
classmethod with_args(**kwargs)  
Wrapper that allows creation of class factories. This can be useful when there is a need to create classes with the same constructor arguments, but different instances. Example: >>> Foo.with_args = classmethod(_with_args)
>>> foo_builder = Foo.with_args(a=3, b=4).with_args(answer=42)
>>> foo_instance1 = foo_builder()
>>> foo_instance2 = foo_builder()
>>> id(foo_instance1) == id(foo_instance2)
False
class torch.quantization.PerChannelMinMaxObserver(ch_axis=0, dtype=torch.quint8, qscheme=torch.per_channel_affine, reduce_range=False, quant_min=None, quant_max=None) [source]
 
Observer module for computing the quantization parameters based on the running per channel min and max values. This observer uses the tensor min/max statistics to compute the per channel quantization parameters. The module records the running minimum and maximum of incoming tensors, and uses this statistic to compute the quantization parameters.  Parameters 
 
ch_axis – Channel axis 
dtype – Quantized data type 
qscheme – Quantization scheme to be used 
reduce_range – Reduces the range of the quantized data type by 1 bit 
quant_min – Minimum quantization value. If unspecified, it will follow the 8-bit setup. 
quant_max – Maximum quantization value. If unspecified, it will follow the 8-bit setup.    The quantization parameters are computed the same way as in MinMaxObserver, with the difference that the running min/max values are stored per channel. Scales and zero points are thus computed per channel as well.  Note If the running minimum equals to the running maximum, the scales and zero_points are set to 1.0 and 0.
torch.quantization.prepare(model, inplace=False, allow_list=None, observer_non_leaf_module_list=None, prepare_custom_config_dict=None) [source]
 
Prepares a copy of the model for quantization calibration or quantization-aware training. Quantization configuration should be assigned preemptively to individual submodules in .qconfig attribute. The model will be attached with observer or fake quant modules, and qconfig will be propagated.  Parameters 
 
model – input model to be modified in-place 
inplace – carry out model transformations in-place, the original module is mutated 
allow_list – list of quantizable modules 
observer_non_leaf_module_list – list of non-leaf modules we want to add observer 
prepare_custom_config_dict – customization configuration dictionary for prepare function    # Example of prepare_custom_config_dict:
prepare_custom_config_dict = {
    # user will manually define the corresponding observed
    # module class which has a from_float class method that converts
    # float custom module to observed custom module
    "float_to_observed_custom_module_class": {
        CustomModule: ObservedCustomModule
    }
 }
torch.quantization.prepare_qat(model, mapping=None, inplace=False) [source]
 
Prepares a copy of the model for quantization calibration or quantization-aware training and converts it to quantized version. Quantization configuration should be assigned preemptively to individual submodules in .qconfig attribute.  Parameters 
 
model – input model to be modified in-place 
mapping – dictionary that maps float modules to quantized modules to be replaced. 
inplace – carry out model transformations in-place, the original module is mutated
torch.quantization.propagate_qconfig_(module, qconfig_dict=None, allow_list=None) [source]
 
Propagate qconfig through the module hierarchy and assign qconfig attribute on each leaf module  Parameters 
 
module – input module 
qconfig_dict – dictionary that maps from name or type of submodule to quantization configuration, qconfig applies to all submodules of a given module unless qconfig for the submodules are specified (when the submodule already has qconfig attribute)   Returns 
None, module is modified inplace with qconfig attached
class torch.quantization.QConfig [source]
 
Describes how to quantize a layer or a part of the network by providing settings (observer classes) for activations and weights respectively. Note that QConfig needs to contain observer classes (like MinMaxObserver) or a callable that returns instances on invocation, not the concrete observer instances themselves. Quantization preparation function will instantiate observers multiple times for each of the layers. Observer classes have usually reasonable default arguments, but they can be overwritten with with_args method (that behaves like functools.partial): my_qconfig = QConfig(activation=MinMaxObserver.with_args(dtype=torch.qint8), weight=default_observer.with_args(dtype=torch.qint8))
class torch.quantization.QConfigDynamic [source]
 
Describes how to dynamically quantize a layer or a part of the network by providing settings (observer classes) for weights. It’s like QConfig, but for dynamic quantization. Note that QConfigDynamic needs to contain observer classes (like MinMaxObserver) or a callable that returns instances on invocation, not the concrete observer instances themselves. Quantization function will instantiate observers multiple times for each of the layers. Observer classes have usually reasonable default arguments, but they can be overwritten with with_args method (that behaves like functools.partial): my_qconfig = QConfigDynamic(weight=default_observer.with_args(dtype=torch.qint8))
torch.quantization.quantize(model, run_fn, run_args, mapping=None, inplace=False) [source]
 
Quantize the input float model with post training static quantization. First it will prepare the model for calibration, then it calls run_fn which will run the calibration step, after that we will convert the model to a quantized model.  Parameters 
 
model – input float model 
run_fn – a calibration function for calibrating the prepared model 
run_args – positional arguments for run_fn
 
inplace – carry out model transformations in-place, the original module is mutated 
mapping – correspondence between original module types and quantized counterparts   Returns 
Quantized model.
torch.quantization.quantize_dynamic(model, qconfig_spec=None, dtype=torch.qint8, mapping=None, inplace=False) [source]
 
Converts a float model to dynamic (i.e. weights-only) quantized model. Replaces specified modules with dynamic weight-only quantized versions and output the quantized model. For simplest usage provide dtype argument that can be float16 or qint8. Weight-only quantization by default is performed for layers with large weights size - i.e. Linear and RNN variants. Fine grained control is possible with qconfig and mapping that act similarly to quantize(). If qconfig is provided, the dtype argument is ignored.  Parameters 
 
model – input model 
qconfig_spec – 
Either:  A dictionary that maps from name or type of submodule to quantization configuration, qconfig applies to all submodules of a given module unless qconfig for the submodules are specified (when the submodule already has qconfig attribute). Entries in the dictionary need to be QConfigDynamic instances. A set of types and/or submodule names to apply dynamic quantization to, in which case the dtype argument is used to specify the bit-width   
inplace – carry out model transformations in-place, the original module is mutated 
mapping – maps type of a submodule to a type of corresponding dynamically quantized version with which the submodule needs to be replaced
torch.quantization.quantize_qat(model, run_fn, run_args, inplace=False) [source]
 
Do quantization aware training and output a quantized model  Parameters 
 
model – input model 
run_fn – a function for evaluating the prepared model, can be a function that simply runs the prepared model or a training loop 
run_args – positional arguments for run_fn
   Returns 
Quantized model.
class torch.quantization.QuantStub(qconfig=None) [source]
 
Quantize stub module, before calibration, this is same as an observer, it will be swapped as nnq.Quantize in convert.  Parameters 
qconfig – quantization configuration for the tensor, if qconfig is not provided, we will get qconfig from parent modules
class torch.quantization.QuantWrapper(module) [source]
 
A wrapper class that wraps the input module, adds QuantStub and DeQuantStub and surround the call to module with call to quant and dequant modules. This is used by the quantization utility functions to add the quant and dequant modules, before convert function QuantStub will just be observer, it observes the input tensor, after convert, QuantStub will be swapped to nnq.Quantize which does actual quantization. Similarly for DeQuantStub.
class torch.quantization.RecordingObserver(**kwargs) [source]
 
The module is mainly for debug and records the tensor values during runtime.  Parameters 
 
dtype – Quantized data type 
qscheme – Quantization scheme to be used 
reduce_range – Reduces the range of the quantized data type by 1 bit
torch.quantization.swap_module(mod, mapping, custom_module_class_mapping) [source]
 
Swaps the module if it has a quantized counterpart and it has an observer attached.  Parameters 
 
mod – input module 
mapping – a dictionary that maps from nn module to nnq module   Returns 
The corresponding quantized module of mod
torch.quantize_per_channel(input, scales, zero_points, axis, dtype) → Tensor  
Converts a float tensor to a per-channel quantized tensor with given scales and zero points.  Parameters 
 
input (Tensor) – float tensor to quantize 
scales (Tensor) – float 1D tensor of scales to use, size should match input.size(axis)
 
zero_points (int) – integer 1D tensor of offset to use, size should match input.size(axis)
 
axis (int) – dimension on which apply per-channel quantization 
dtype (torch.dtype) – the desired data type of returned tensor. Has to be one of the quantized dtypes: torch.quint8, torch.qint8, torch.qint32
   Returns 
A newly quantized tensor  Return type 
Tensor   Example: >>> x = torch.tensor([[-1.0, 0.0], [1.0, 2.0]])
>>> torch.quantize_per_channel(x, torch.tensor([0.1, 0.01]), torch.tensor([10, 0]), 0, torch.quint8)
tensor([[-1.,  0.],
        [ 1.,  2.]], size=(2, 2), dtype=torch.quint8,
       quantization_scheme=torch.per_channel_affine,
       scale=tensor([0.1000, 0.0100], dtype=torch.float64),
       zero_point=tensor([10,  0]), axis=0)
>>> torch.quantize_per_channel(x, torch.tensor([0.1, 0.01]), torch.tensor([10, 0]), 0, torch.quint8).int_repr()
tensor([[  0,  10],
        [100, 200]], dtype=torch.uint8)
torch.quantize_per_tensor(input, scale, zero_point, dtype) → Tensor  
Converts a float tensor to a quantized tensor with given scale and zero point.  Parameters 
 
input (Tensor) – float tensor to quantize 
scale (float) – scale to apply in quantization formula 
zero_point (int) – offset in integer value that maps to float zero 
dtype (torch.dtype) – the desired data type of returned tensor. Has to be one of the quantized dtypes: torch.quint8, torch.qint8, torch.qint32
   Returns 
A newly quantized tensor  Return type 
Tensor   Example: >>> torch.quantize_per_tensor(torch.tensor([-1.0, 0.0, 1.0, 2.0]), 0.1, 10, torch.quint8)
tensor([-1.,  0.,  1.,  2.], size=(4,), dtype=torch.quint8,
       quantization_scheme=torch.per_tensor_affine, scale=0.1, zero_point=10)
>>> torch.quantize_per_tensor(torch.tensor([-1.0, 0.0, 1.0, 2.0]), 0.1, 10, torch.quint8).int_repr()
tensor([ 0, 10, 20, 30], dtype=torch.uint8)
class torch.quasirandom.SobolEngine(dimension, scramble=False, seed=None) [source]
 
The torch.quasirandom.SobolEngine is an engine for generating (scrambled) Sobol sequences. Sobol sequences are an example of low discrepancy quasi-random sequences. This implementation of an engine for Sobol sequences is capable of sampling sequences up to a maximum dimension of 21201. It uses direction numbers from https://web.maths.unsw.edu.au/~fkuo/sobol/ obtained using the search criterion D(6) up to the dimension 21201. This is the recommended choice by the authors. References  Art B. Owen. Scrambling Sobol and Niederreiter-Xing points. Journal of Complexity, 14(4):466-489, December 1998. I. M. Sobol. The distribution of points in a cube and the accurate evaluation of integrals. Zh. Vychisl. Mat. i Mat. Phys., 7:784-802, 1967.   Parameters 
 
dimension (Int) – The dimensionality of the sequence to be drawn 
scramble (bool, optional) – Setting this to True will produce scrambled Sobol sequences. Scrambling is capable of producing better Sobol sequences. Default: False. 
seed (Int, optional) – This is the seed for the scrambling. The seed of the random number generator is set to this, if specified. Otherwise, it uses a random seed. Default: None
    Examples: >>> soboleng = torch.quasirandom.SobolEngine(dimension=5)
>>> soboleng.draw(3)
tensor([[0.5000, 0.5000, 0.5000, 0.5000, 0.5000],
        [0.7500, 0.2500, 0.7500, 0.2500, 0.7500],
        [0.2500, 0.7500, 0.2500, 0.7500, 0.2500]])
  
draw(n=1, out=None, dtype=torch.float32) [source]
 
Function to draw a sequence of n points from a Sobol sequence. Note that the samples are dependent on the previous samples. The size of the result is (n,dimension)(n, dimension) .  Parameters 
 
n (Int, optional) – The length of sequence of points to draw. Default: 1 
out (Tensor, optional) – The output tensor 
dtype (torch.dtype, optional) – the desired data type of the returned tensor. Default: torch.float32
    
  
draw_base2(m, out=None, dtype=torch.float32) [source]
 
Function to draw a sequence of 2**m points from a Sobol sequence. Note that the samples are dependent on the previous samples. The size of the result is (2∗∗m,dimension)(2**m, dimension) .  Parameters 
 
m (Int) – The (base2) exponent of the number of points to draw. 
out (Tensor, optional) – The output tensor 
dtype (torch.dtype, optional) – the desired data type of the returned tensor. Default: torch.float32
    
  
fast_forward(n) [source]
 
Function to fast-forward the state of the SobolEngine by n steps. This is equivalent to drawing n samples without using the samples.  Parameters 
n (Int) – The number of steps to fast-forward by.   
  
reset() [source]
 
Function to reset the SobolEngine to base state.
draw(n=1, out=None, dtype=torch.float32) [source]
 
Function to draw a sequence of n points from a Sobol sequence. Note that the samples are dependent on the previous samples. The size of the result is (n,dimension)(n, dimension) .  Parameters 
 
n (Int, optional) – The length of sequence of points to draw. Default: 1 
out (Tensor, optional) – The output tensor 
dtype (torch.dtype, optional) – the desired data type of the returned tensor. Default: torch.float32
draw_base2(m, out=None, dtype=torch.float32) [source]
 
Function to draw a sequence of 2**m points from a Sobol sequence. Note that the samples are dependent on the previous samples. The size of the result is (2∗∗m,dimension)(2**m, dimension) .  Parameters 
 
m (Int) – The (base2) exponent of the number of points to draw. 
out (Tensor, optional) – The output tensor 
dtype (torch.dtype, optional) – the desired data type of the returned tensor. Default: torch.float32
fast_forward(n) [source]
 
Function to fast-forward the state of the SobolEngine by n steps. This is equivalent to drawing n samples without using the samples.  Parameters 
n (Int) – The number of steps to fast-forward by.
torch.rad2deg(input, *, out=None) → Tensor  
Returns a new tensor with each of the elements of input converted from angles in radians to degrees.  Parameters 
input (Tensor) – the input tensor.  Keyword Arguments 
out (Tensor, optional) – the output tensor.   Example: >>> a = torch.tensor([[3.142, -3.142], [6.283, -6.283], [1.570, -1.570]])
>>> torch.rad2deg(a)
tensor([[ 180.0233, -180.0233],
        [ 359.9894, -359.9894],
        [  89.9544,  -89.9544]])
torch.rand(*size, *, out=None, dtype=None, layout=torch.strided, device=None, requires_grad=False) → Tensor  
Returns a tensor filled with random numbers from a uniform distribution on the interval [0,1)[0, 1)  The shape of the tensor is defined by the variable argument size.  Parameters 
size (int...) – a sequence of integers defining the shape of the output tensor. Can be a variable number of arguments or a collection like a list or tuple.  Keyword Arguments 
 
out (Tensor, optional) – the output tensor. 
dtype (torch.dtype, optional) – the desired data type of returned tensor. Default: if None, uses a global default (see torch.set_default_tensor_type()). 
layout (torch.layout, optional) – the desired layout of returned Tensor. Default: torch.strided. 
device (torch.device, optional) – the desired device of returned tensor. Default: if None, uses the current device for the default tensor type (see torch.set_default_tensor_type()). device will be the CPU for CPU tensor types and the current CUDA device for CUDA tensor types. 
requires_grad (bool, optional) – If autograd should record operations on the returned tensor. Default: False.    Example: >>> torch.rand(4)
tensor([ 0.5204,  0.2503,  0.3525,  0.5673])
>>> torch.rand(2, 3)
tensor([[ 0.8237,  0.5781,  0.6879],
        [ 0.3816,  0.7249,  0.0998]])
torch.randint(low=0, high, size, *, generator=None, out=None, dtype=None, layout=torch.strided, device=None, requires_grad=False) → Tensor  
Returns a tensor filled with random integers generated uniformly between low (inclusive) and high (exclusive). The shape of the tensor is defined by the variable argument size.  Note With the global dtype default (torch.float32), this function returns a tensor with dtype torch.int64.   Parameters 
 
low (int, optional) – Lowest integer to be drawn from the distribution. Default: 0. 
high (int) – One above the highest integer to be drawn from the distribution. 
size (tuple) – a tuple defining the shape of the output tensor.   Keyword Arguments 
 
generator (torch.Generator, optional) – a pseudorandom number generator for sampling 
out (Tensor, optional) – the output tensor. 
dtype (torch.dtype, optional) – the desired data type of returned tensor. Default: if None, uses a global default (see torch.set_default_tensor_type()). 
layout (torch.layout, optional) – the desired layout of returned Tensor. Default: torch.strided. 
device (torch.device, optional) – the desired device of returned tensor. Default: if None, uses the current device for the default tensor type (see torch.set_default_tensor_type()). device will be the CPU for CPU tensor types and the current CUDA device for CUDA tensor types. 
requires_grad (bool, optional) – If autograd should record operations on the returned tensor. Default: False.    Example: >>> torch.randint(3, 5, (3,))
tensor([4, 3, 4])


>>> torch.randint(10, (2, 2))
tensor([[0, 2],
        [5, 5]])


>>> torch.randint(3, 10, (2, 2))
tensor([[4, 5],
        [6, 7]])
torch.randint_like(input, low=0, high, *, dtype=None, layout=torch.strided, device=None, requires_grad=False, memory_format=torch.preserve_format) → Tensor  
Returns a tensor with the same shape as Tensor input filled with random integers generated uniformly between low (inclusive) and high (exclusive).  Parameters 
 
input (Tensor) – the size of input will determine size of the output tensor. 
low (int, optional) – Lowest integer to be drawn from the distribution. Default: 0. 
high (int) – One above the highest integer to be drawn from the distribution.   Keyword Arguments 
 
dtype (torch.dtype, optional) – the desired data type of returned Tensor. Default: if None, defaults to the dtype of input. 
layout (torch.layout, optional) – the desired layout of returned tensor. Default: if None, defaults to the layout of input. 
device (torch.device, optional) – the desired device of returned tensor. Default: if None, defaults to the device of input. 
requires_grad (bool, optional) – If autograd should record operations on the returned tensor. Default: False. 
memory_format (torch.memory_format, optional) – the desired memory format of returned Tensor. Default: torch.preserve_format.
torch.randn(*size, *, out=None, dtype=None, layout=torch.strided, device=None, requires_grad=False) → Tensor  
Returns a tensor filled with random numbers from a normal distribution with mean 0 and variance 1 (also called the standard normal distribution).  outi∼N(0,1)\text{out}_{i} \sim \mathcal{N}(0, 1)  
The shape of the tensor is defined by the variable argument size.  Parameters 
size (int...) – a sequence of integers defining the shape of the output tensor. Can be a variable number of arguments or a collection like a list or tuple.  Keyword Arguments 
 
out (Tensor, optional) – the output tensor. 
dtype (torch.dtype, optional) – the desired data type of returned tensor. Default: if None, uses a global default (see torch.set_default_tensor_type()). 
layout (torch.layout, optional) – the desired layout of returned Tensor. Default: torch.strided. 
device (torch.device, optional) – the desired device of returned tensor. Default: if None, uses the current device for the default tensor type (see torch.set_default_tensor_type()). device will be the CPU for CPU tensor types and the current CUDA device for CUDA tensor types. 
requires_grad (bool, optional) – If autograd should record operations on the returned tensor. Default: False.    Example: >>> torch.randn(4)
tensor([-2.1436,  0.9966,  2.3426, -0.6366])
>>> torch.randn(2, 3)
tensor([[ 1.5954,  2.8929, -1.0923],
        [ 1.1719, -0.4709, -0.1996]])
torch.randn_like(input, *, dtype=None, layout=None, device=None, requires_grad=False, memory_format=torch.preserve_format) → Tensor  
Returns a tensor with the same size as input that is filled with random numbers from a normal distribution with mean 0 and variance 1. torch.randn_like(input) is equivalent to torch.randn(input.size(), dtype=input.dtype, layout=input.layout, device=input.device).  Parameters 
input (Tensor) – the size of input will determine size of the output tensor.  Keyword Arguments 
 
dtype (torch.dtype, optional) – the desired data type of returned Tensor. Default: if None, defaults to the dtype of input. 
layout (torch.layout, optional) – the desired layout of returned tensor. Default: if None, defaults to the layout of input. 
device (torch.device, optional) – the desired device of returned tensor. Default: if None, defaults to the device of input. 
requires_grad (bool, optional) – If autograd should record operations on the returned tensor. Default: False. 
memory_format (torch.memory_format, optional) – the desired memory format of returned Tensor. Default: torch.preserve_format.