torch.polygamma(n, input, *, out=None) → Tensor
Computes the nthn^{th}  derivative of the digamma function on input. n≥0n \geq 0  is called the order of the polygamma function.  ψ(n)(x)=d(n)dx(n)ψ(x)\psi^{(n)}(x) = \frac{d^{(n)}}{dx^{(n)}} \psi(x)
 Note This function is implemented only for nonnegative integers n≥0n \geq 0 .   Parameters

n (int) – the order of the polygamma function
input (Tensor) – the input tensor.   Keyword Arguments
out (Tensor, optional) – the output tensor.    Example::

>>> a = torch.tensor([1, 0.5])
>>> torch.polygamma(1, a)
tensor([1.64493, 4.9348])
>>> torch.polygamma(2, a)
tensor([ -2.4041, -16.8288])
>>> torch.polygamma(3, a)
tensor([ 6.4939, 97.4091])
>>> torch.polygamma(4, a)
tensor([ -24.8863, -771.4742])


torch.pow(input, exponent, *, out=None) → Tensor
Takes the power of each element in input with exponent and returns a tensor with the result. exponent can be either a single float number or a Tensor with the same number of elements as input. When exponent is a scalar value, the operation applied is:  outi=xiexponent\text{out}_i = x_i ^ \text{exponent}
When exponent is a tensor, the operation applied is:  outi=xiexponenti\text{out}_i = x_i ^ {\text{exponent}_i}
When exponent is a tensor, the shapes of input and exponent must be broadcastable.  Parameters

input (Tensor) – the input tensor.
exponent (float or tensor) – the exponent value   Keyword Arguments
out (Tensor, optional) – the output tensor.   Example: >>> a = torch.randn(4)
>>> a
tensor([ 0.4331,  1.2475,  0.6834, -0.2791])
>>> torch.pow(a, 2)
tensor([ 0.1875,  1.5561,  0.4670,  0.0779])
>>> exp = torch.arange(1., 5.)

>>> a = torch.arange(1., 5.)
>>> a
tensor([ 1.,  2.,  3.,  4.])
>>> exp
tensor([ 1.,  2.,  3.,  4.])
>>> torch.pow(a, exp)
tensor([   1.,    4.,   27.,  256.])

torch.pow(self, exponent, *, out=None) → Tensor
 self is a scalar float value, and exponent is a tensor. The returned tensor out is of the same shape as exponent The operation applied is:  outi=selfexponenti\text{out}_i = \text{self} ^ {\text{exponent}_i}
 Parameters

self (float) – the scalar base value for the power operation
exponent (Tensor) – the exponent tensor   Keyword Arguments
out (Tensor, optional) – the output tensor.   Example: >>> exp = torch.arange(1., 5.)
>>> base = 2
>>> torch.pow(base, exp)
tensor([  2.,   4.,   8.,  16.])


torch.prod(input, *, dtype=None) → Tensor
Returns the product of all elements in the input tensor.  Parameters
input (Tensor) – the input tensor.  Keyword Arguments
dtype (torch.dtype, optional) – the desired data type of returned tensor. If specified, the input tensor is casted to dtype before the operation is performed. This is useful for preventing data type overflows. Default: None.   Example: >>> a = torch.randn(1, 3)
>>> a
tensor([[-0.8020,  0.5428, -1.5854]])
>>> torch.prod(a)
tensor(0.6902)

torch.prod(input, dim, keepdim=False, *, dtype=None) → Tensor
 Returns the product of each row of the input tensor in the given dimension dim. If keepdim is True, the output tensor is of the same size as input except in the dimension dim where it is of size 1. Otherwise, dim is squeezed (see torch.squeeze()), resulting in the output tensor having 1 fewer dimension than input.  Parameters

input (Tensor) – the input tensor.
dim (int) – the dimension to reduce.
keepdim (bool) – whether the output tensor has dim retained or not.   Keyword Arguments
dtype (torch.dtype, optional) – the desired data type of returned tensor. If specified, the input tensor is casted to dtype before the operation is performed. This is useful for preventing data type overflows. Default: None.   Example: >>> a = torch.randn(4, 2)
>>> a
tensor([[ 0.5261, -0.3837],
        [ 1.1857, -0.2498],
        [-1.1646,  0.0705],
        [ 1.1131, -1.0629]])
>>> torch.prod(a, 1)
tensor([-0.2018, -0.2962, -0.0821, -1.1831])


torch.promote_types(type1, type2) → dtype
Returns the torch.dtype with the smallest size and scalar kind that is not smaller nor of lower kind than either type1 or type2. See type promotion documentation for more information on the type promotion logic.  Parameters

type1 (torch.dtype) –
type2 (torch.dtype) –     Example: >>> torch.promote_types(torch.int32, torch.float32)
torch.float32
>>> torch.promote_types(torch.uint8, torch.long)
torch.long


torch.qr(input, some=True, *, out=None) -> (Tensor, Tensor)
Computes the QR decomposition of a matrix or a batch of matrices input, and returns a namedtuple (Q, R) of tensors such that input=QR\text{input} = Q R  with QQ  being an orthogonal matrix or batch of orthogonal matrices and RR  being an upper triangular matrix or batch of upper triangular matrices. If some is True, then this function returns the thin (reduced) QR factorization. Otherwise, if some is False, this function returns the complete QR factorization.  Warning torch.qr is deprecated. Please use torch.linalg.qr() instead. Differences with torch.linalg.qr:
torch.linalg.qr takes a string parameter mode instead of some:
some=True is equivalent of mode='reduced': both are the default
some=False is equivalent of mode='complete'.      Warning If you plan to backpropagate through QR, note that the current backward implementation is only well-defined when the first min(input.size(−1),input.size(−2))\min(input.size(-1), input.size(-2))  columns of input are linearly independent. This behavior will propably change once QR supports pivoting.   Note This function uses LAPACK for CPU inputs and MAGMA for CUDA inputs, and may produce different (valid) decompositions on different device types or different platforms.   Parameters

input (Tensor) – the input tensor of size (∗,m,n)(*, m, n)  where * is zero or more batch dimensions consisting of matrices of dimension m×nm \times n .
some (bool, optional) –
Set to True for reduced QR decomposition and False for complete QR decomposition. If k = min(m, n) then:
some=True : returns (Q, R) with dimensions (m, k), (k, n) (default)
'some=False': returns (Q, R) with dimensions (m, m), (m, n)     Keyword Arguments
out (tuple, optional) – tuple of Q and R tensors. The dimensions of Q and R are detailed in the description of some above.   Example: >>> a = torch.tensor([[12., -51, 4], [6, 167, -68], [-4, 24, -41]])
>>> q, r = torch.qr(a)
>>> q
tensor([[-0.8571,  0.3943,  0.3314],
        [-0.4286, -0.9029, -0.0343],
        [ 0.2857, -0.1714,  0.9429]])
>>> r
tensor([[ -14.0000,  -21.0000,   14.0000],
        [   0.0000, -175.0000,   70.0000],
        [   0.0000,    0.0000,  -35.0000]])
>>> torch.mm(q, r).round()
tensor([[  12.,  -51.,    4.],
        [   6.,  167.,  -68.],
        [  -4.,   24.,  -41.]])
>>> torch.mm(q.t(), q).round()
tensor([[ 1.,  0.,  0.],
        [ 0.,  1., -0.],
        [ 0., -0.,  1.]])
>>> a = torch.randn(3, 4, 5)
>>> q, r = torch.qr(a, some=False)
>>> torch.allclose(torch.matmul(q, r), a)
True
>>> torch.allclose(torch.matmul(q.transpose(-2, -1), q), torch.eye(5))
True


torch.quantile(input, q) → Tensor
Returns the q-th quantiles of all elements in the input tensor, doing a linear interpolation when the q-th quantile lies between two data points.  Parameters

input (Tensor) – the input tensor.
q (float or Tensor) – a scalar or 1D tensor of quantile values in the range [0, 1]    Example: >>> a = torch.randn(1, 3)
>>> a
tensor([[ 0.0700, -0.5446,  0.9214]])
>>> q = torch.tensor([0, 0.5, 1])
>>> torch.quantile(a, q)
tensor([-0.5446,  0.0700,  0.9214])

torch.quantile(input, q, dim=None, keepdim=False, *, out=None) → Tensor
 Returns the q-th quantiles of each row of the input tensor along the dimension dim, doing a linear interpolation when the q-th quantile lies between two data points. By default, dim is None resulting in the input tensor being flattened before computation. If keepdim is True, the output dimensions are of the same size as input except in the dimensions being reduced (dim or all if dim is None) where they have size 1. Otherwise, the dimensions being reduced are squeezed (see torch.squeeze()). If q is a 1D tensor, an extra dimension is prepended to the output tensor with the same size as q which represents the quantiles.  Parameters

input (Tensor) – the input tensor.
q (float or Tensor) – a scalar or 1D tensor of quantile values in the range [0, 1]
dim (int) – the dimension to reduce.
keepdim (bool) – whether the output tensor has dim retained or not.   Keyword Arguments
out (Tensor, optional) – the output tensor.   Example: >>> a = torch.randn(2, 3)
>>> a
tensor([[ 0.0795, -1.2117,  0.9765],
        [ 1.1707,  0.6706,  0.4884]])
>>> q = torch.tensor([0.25, 0.5, 0.75])
>>> torch.quantile(a, q, dim=1, keepdim=True)
tensor([[[-0.5661],
        [ 0.5795]],

        [[ 0.0795],
        [ 0.6706]],

        [[ 0.5280],
        [ 0.9206]]])
>>> torch.quantile(a, q, dim=1, keepdim=True).shape
torch.Size([3, 2, 1])


torch.quantization This module implements the functions you call directly to convert your model from FP32 to quantized form. For example the prepare() is used in post training quantization to prepares your model for the calibration step and convert() actually converts the weights to int8 and replaces the operations with their quantized counterparts. There are other helper functions for things like quantizing the input to your model and performing critical fusions like conv+relu. Top-level quantization APIs
torch.quantization.quantize(model, run_fn, run_args, mapping=None, inplace=False) [source]

Quantize the input float model with post training static quantization. First it will prepare the model for calibration, then it calls run_fn which will run the calibration step, after that we will convert the model to a quantized model.  Parameters

model – input float model
run_fn – a calibration function for calibrating the prepared model
run_args – positional arguments for run_fn

inplace – carry out model transformations in-place, the original module is mutated
mapping – correspondence between original module types and quantized counterparts   Returns
Quantized model.

torch.quantization.quantize_dynamic(model, qconfig_spec=None, dtype=torch.qint8, mapping=None, inplace=False) [source]

Converts a float model to dynamic (i.e. weights-only) quantized model. Replaces specified modules with dynamic weight-only quantized versions and output the quantized model. For simplest usage provide dtype argument that can be float16 or qint8. Weight-only quantization by default is performed for layers with large weights size - i.e. Linear and RNN variants. Fine grained control is possible with qconfig and mapping that act similarly to quantize(). If qconfig is provided, the dtype argument is ignored.  Parameters

model – input model
qconfig_spec –
Either:  A dictionary that maps from name or type of submodule to quantization configuration, qconfig applies to all submodules of a given module unless qconfig for the submodules are specified (when the submodule already has qconfig attribute). Entries in the dictionary need to be QConfigDynamic instances. A set of types and/or submodule names to apply dynamic quantization to, in which case the dtype argument is used to specify the bit-width
inplace – carry out model transformations in-place, the original module is mutated
mapping – maps type of a submodule to a type of corresponding dynamically quantized version with which the submodule needs to be replaced

torch.quantization.quantize_qat(model, run_fn, run_args, inplace=False) [source]

Do quantization aware training and output a quantized model  Parameters

model – input model
run_fn – a function for evaluating the prepared model, can be a function that simply runs the prepared model or a training loop
run_args – positional arguments for run_fn
   Returns
Quantized model.

torch.quantization.prepare(model, inplace=False, allow_list=None, observer_non_leaf_module_list=None, prepare_custom_config_dict=None) [source]

Prepares a copy of the model for quantization calibration or quantization-aware training. Quantization configuration should be assigned preemptively to individual submodules in .qconfig attribute. The model will be attached with observer or fake quant modules, and qconfig will be propagated.  Parameters

model – input model to be modified in-place
inplace – carry out model transformations in-place, the original module is mutated
allow_list – list of quantizable modules
observer_non_leaf_module_list – list of non-leaf modules we want to add observer
prepare_custom_config_dict – customization configuration dictionary for prepare function    # Example of prepare_custom_config_dict:
prepare_custom_config_dict = {
    # user will manually define the corresponding observed
    # module class which has a from_float class method that converts
    # float custom module to observed custom module
    "float_to_observed_custom_module_class": {
        CustomModule: ObservedCustomModule
    }
 }

torch.quantization.prepare_qat(model, mapping=None, inplace=False) [source]

Prepares a copy of the model for quantization calibration or quantization-aware training and converts it to quantized version. Quantization configuration should be assigned preemptively to individual submodules in .qconfig attribute.  Parameters

model – input model to be modified in-place
mapping – dictionary that maps float modules to quantized modules to be replaced.
inplace – carry out model transformations in-place, the original module is mutated

torch.quantization.convert(module, mapping=None, inplace=False, remove_qconfig=True, convert_custom_config_dict=None) [source]

Converts submodules in input module to a different module according to mapping by calling from_float method on the target module class. And remove qconfig at the end if remove_qconfig is set to True.  Parameters

module – prepared and calibrated module
mapping – a dictionary that maps from source module type to target module type, can be overwritten to allow swapping user defined Modules
inplace – carry out model transformations in-place, the original module is mutated
convert_custom_config_dict – custom configuration dictionary for convert function    # Example of convert_custom_config_dict:
convert_custom_config_dict = {
    # user will manually define the corresponding quantized
    # module class which has a from_observed class method that converts
    # observed custom module to quantized custom module
    "observed_to_quantized_custom_module_class": {
        ObservedCustomModule: QuantizedCustomModule
    }
}

class torch.quantization.QConfig [source]

Describes how to quantize a layer or a part of the network by providing settings (observer classes) for activations and weights respectively. Note that QConfig needs to contain observer classes (like MinMaxObserver) or a callable that returns instances on invocation, not the concrete observer instances themselves. Quantization preparation function will instantiate observers multiple times for each of the layers. Observer classes have usually reasonable default arguments, but they can be overwritten with with_args method (that behaves like functools.partial): my_qconfig = QConfig(activation=MinMaxObserver.with_args(dtype=torch.qint8), weight=default_observer.with_args(dtype=torch.qint8))

class torch.quantization.QConfigDynamic [source]

Describes how to dynamically quantize a layer or a part of the network by providing settings (observer classes) for weights. It’s like QConfig, but for dynamic quantization. Note that QConfigDynamic needs to contain observer classes (like MinMaxObserver) or a callable that returns instances on invocation, not the concrete observer instances themselves. Quantization function will instantiate observers multiple times for each of the layers. Observer classes have usually reasonable default arguments, but they can be overwritten with with_args method (that behaves like functools.partial): my_qconfig = QConfigDynamic(weight=default_observer.with_args(dtype=torch.qint8))
 Preparing model for quantization
torch.quantization.fuse_modules(model, modules_to_fuse, inplace=False, fuser_func=<function fuse_known_modules>, fuse_custom_config_dict=None) [source]

Fuses a list of modules into a single module Fuses only the following sequence of modules: conv, bn conv, bn, relu conv, relu linear, relu bn, relu All other sequences are left unchanged. For these sequences, replaces the first item in the list with the fused module, replacing the rest of the modules with identity.  Parameters

model – Model containing the modules to be fused
modules_to_fuse – list of list of module names to fuse. Can also be a list of strings if there is only a single list of modules to fuse.
inplace – bool specifying if fusion happens in place on the model, by default a new model is returned
fuser_func – Function that takes in a list of modules and outputs a list of fused modules of the same length. For example, fuser_func([convModule, BNModule]) returns the list [ConvBNModule, nn.Identity()] Defaults to torch.quantization.fuse_known_modules
fuse_custom_config_dict – custom configuration for fusion    # Example of fuse_custom_config_dict
fuse_custom_config_dict = {
    # Additional fuser_method mapping
    "additional_fuser_method_mapping": {
        (torch.nn.Conv2d, torch.nn.BatchNorm2d): fuse_conv_bn
    },
}
  Returns
model with fused modules. A new copy is created if inplace=True.   Examples: >>> m = myModel()
>>> # m is a module containing  the sub-modules below
>>> modules_to_fuse = [ ['conv1', 'bn1', 'relu1'], ['submodule.conv', 'submodule.relu']]
>>> fused_m = torch.quantization.fuse_modules(m, modules_to_fuse)
>>> output = fused_m(input)

>>> m = myModel()
>>> # Alternately provide a single list of modules to fuse
>>> modules_to_fuse = ['conv1', 'bn1', 'relu1']
>>> fused_m = torch.quantization.fuse_modules(m, modules_to_fuse)
>>> output = fused_m(input)

class torch.quantization.QuantStub(qconfig=None) [source]

Quantize stub module, before calibration, this is same as an observer, it will be swapped as nnq.Quantize in convert.  Parameters
qconfig – quantization configuration for the tensor, if qconfig is not provided, we will get qconfig from parent modules

class torch.quantization.DeQuantStub [source]

Dequantize stub module, before calibration, this is same as identity, this will be swapped as nnq.DeQuantize in convert.

class torch.quantization.QuantWrapper(module) [source]

A wrapper class that wraps the input module, adds QuantStub and DeQuantStub and surround the call to module with call to quant and dequant modules. This is used by the quantization utility functions to add the quant and dequant modules, before convert function QuantStub will just be observer, it observes the input tensor, after convert, QuantStub will be swapped to nnq.Quantize which does actual quantization. Similarly for DeQuantStub.

torch.quantization.add_quant_dequant(module) [source]

Wrap the leaf child module in QuantWrapper if it has a valid qconfig Note that this function will modify the children of module inplace and it can return a new module which wraps the input module as well.  Parameters

module – input module with qconfig attributes for all the leaf modules
we want to quantize (that) –    Returns
Either the inplace modified module with submodules wrapped in QuantWrapper based on qconfig or a new QuantWrapper module which wraps the input module, the latter case only happens when the input module is a leaf module and we want to quantize it.
 Utility functions
torch.quantization.add_observer_(module, qconfig_propagation_list=None, non_leaf_module_list=None, device=None, custom_module_class_mapping=None) [source]

Add observer for the leaf child of the module. This function insert observer module to all leaf child module that has a valid qconfig attribute.  Parameters

module – input module with qconfig attributes for all the leaf modules that we want to quantize
device – parent device, if any
non_leaf_module_list – list of non-leaf modules we want to add observer   Returns
None, module is modified inplace with added observer modules and forward_hooks

torch.quantization.swap_module(mod, mapping, custom_module_class_mapping) [source]

Swaps the module if it has a quantized counterpart and it has an observer attached.  Parameters

mod – input module
mapping – a dictionary that maps from nn module to nnq module   Returns
The corresponding quantized module of mod

torch.quantization.propagate_qconfig_(module, qconfig_dict=None, allow_list=None) [source]

Propagate qconfig through the module hierarchy and assign qconfig attribute on each leaf module  Parameters

module – input module
qconfig_dict – dictionary that maps from name or type of submodule to quantization configuration, qconfig applies to all submodules of a given module unless qconfig for the submodules are specified (when the submodule already has qconfig attribute)   Returns
None, module is modified inplace with qconfig attached

torch.quantization.default_eval_fn(model, calib_data) [source]

Default evaluation function takes a torch.utils.data.Dataset or a list of input Tensors and run the model on the dataset
 Observers
class torch.quantization.ObserverBase(dtype) [source]

Base observer Module. Any observer implementation should derive from this class. Concrete observers should follow the same API. In forward, they will update the statistics of the observed Tensor. And they should provide a calculate_qparams function that computes the quantization parameters given the collected statistics.  Parameters
dtype – Quantized data type
classmethod with_args(**kwargs)
Wrapper that allows creation of class factories. This can be useful when there is a need to create classes with the same constructor arguments, but different instances. Example: >>> Foo.with_args = classmethod(_with_args)
>>> foo_builder = Foo.with_args(a=3, b=4).with_args(answer=42)
>>> foo_instance1 = foo_builder()
>>> foo_instance2 = foo_builder()
>>> id(foo_instance1) == id(foo_instance2)
False

class torch.quantization.MinMaxObserver(dtype=torch.quint8, qscheme=torch.per_tensor_affine, reduce_range=False, quant_min=None, quant_max=None) [source]

Observer module for computing the quantization parameters based on the running min and max values. This observer uses the tensor min/max statistics to compute the quantization parameters. The module records the running minimum and maximum of incoming tensors, and uses this statistic to compute the quantization parameters.  Parameters

dtype – Quantized data type
qscheme – Quantization scheme to be used
reduce_range – Reduces the range of the quantized data type by 1 bit
quant_min – Minimum quantization value. If unspecified, it will follow the 8-bit setup.
quant_max – Maximum quantization value. If unspecified, it will follow the 8-bit setup.    Given running min/max as xminx_\text{min}  and xmaxx_\text{max} , scale ss  and zero point zz  are computed as: The running minimum/maximum xmin/maxx_\text{min/max}  is computed as:  xmin={min(X)if xmin=Nonemin(xmin,min(X))otherwisexmax={max(X)if xmax=Nonemax(xmax,max(X))otherwise\begin{array}{ll} x_\text{min} &= \begin{cases} \min(X) & \text{if~}x_\text{min} = \text{None} \\ \min\left(x_\text{min}, \min(X)\right) & \text{otherwise} \end{cases}\\ x_\text{max} &= \begin{cases} \max(X) & \text{if~}x_\text{max} = \text{None} \\ \max\left(x_\text{max}, \max(X)\right) & \text{otherwise} \end{cases}\\ \end{array}
where XX  is the observed tensor. The scale ss  and zero point zz  are then computed as:  if Symmetric:s=2max(∣xmin∣,xmax)/(Qmax−Qmin)z={0if dtype is qint8128otherwiseOtherwise:s=(xmax−xmin)/(Qmax−Qmin)z=Qmin−round(xmin/s)\begin{aligned} \text{if Symmetric:}&\\ &s = 2 \max(|x_\text{min}|, x_\text{max}) / \left( Q_\text{max} - Q_\text{min} \right) \\ &z = \begin{cases} 0 & \text{if dtype is qint8} \\ 128 & \text{otherwise} \end{cases}\\ \text{Otherwise:}&\\ &s = \left( x_\text{max} - x_\text{min} \right ) / \left( Q_\text{max} - Q_\text{min} \right ) \\ &z = Q_\text{min} - \text{round}(x_\text{min} / s) \end{aligned}
where QminQ_\text{min}  and QmaxQ_\text{max}  are the minimum and maximum of the quantized data type.  Warning Only works with torch.per_tensor_symmetric quantization scheme   Warning dtype can only take torch.qint8 or torch.quint8.   Note If the running minimum equals to the running maximum, the scale and zero_point are set to 1.0 and 0.

class torch.quantization.MovingAverageMinMaxObserver(averaging_constant=0.01, dtype=torch.quint8, qscheme=torch.per_tensor_affine, reduce_range=False, quant_min=None, quant_max=None) [source]

Observer module for computing the quantization parameters based on the moving average of the min and max values. This observer computes the quantization parameters based on the moving averages of minimums and maximums of the incoming tensors. The module records the average minimum and maximum of incoming tensors, and uses this statistic to compute the quantization parameters.  Parameters

averaging_constant – Averaging constant for min/max.
dtype – Quantized data type
qscheme – Quantization scheme to be used
reduce_range – Reduces the range of the quantized data type by 1 bit
quant_min – Minimum quantization value. If unspecified, it will follow the 8-bit setup.
quant_max – Maximum quantization value. If unspecified, it will follow the 8-bit setup.    The moving average min/max is computed as follows  xmin={min(X)if xmin=None(1−c)xmin+cmin(X)otherwisexmax={max(X)if xmax=None(1−c)xmax+cmax(X)otherwise\begin{array}{ll} x_\text{min} = \begin{cases} \min(X) & \text{if~}x_\text{min} = \text{None} \\ (1 - c) x_\text{min} + c \min(X) & \text{otherwise} \end{cases}\\ x_\text{max} = \begin{cases} \max(X) & \text{if~}x_\text{max} = \text{None} \\ (1 - c) x_\text{max} + c \max(X) & \text{otherwise} \end{cases}\\ \end{array}
where xmin/maxx_\text{min/max}  is the running average min/max, XX  is is the incoming tensor, and cc  is the averaging_constant. The scale and zero point are then computed as in MinMaxObserver.  Note Only works with torch.per_tensor_affine quantization scheme.   Note If the running minimum equals to the running maximum, the scale and zero_point are set to 1.0 and 0.

class torch.quantization.PerChannelMinMaxObserver(ch_axis=0, dtype=torch.quint8, qscheme=torch.per_channel_affine, reduce_range=False, quant_min=None, quant_max=None) [source]

Observer module for computing the quantization parameters based on the running per channel min and max values. This observer uses the tensor min/max statistics to compute the per channel quantization parameters. The module records the running minimum and maximum of incoming tensors, and uses this statistic to compute the quantization parameters.  Parameters

ch_axis – Channel axis
dtype – Quantized data type
qscheme – Quantization scheme to be used
reduce_range – Reduces the range of the quantized data type by 1 bit
quant_min – Minimum quantization value. If unspecified, it will follow the 8-bit setup.
quant_max – Maximum quantization value. If unspecified, it will follow the 8-bit setup.    The quantization parameters are computed the same way as in MinMaxObserver, with the difference that the running min/max values are stored per channel. Scales and zero points are thus computed per channel as well.  Note If the running minimum equals to the running maximum, the scales and zero_points are set to 1.0 and 0.

class torch.quantization.MovingAveragePerChannelMinMaxObserver(averaging_constant=0.01, ch_axis=0, dtype=torch.quint8, qscheme=torch.per_channel_affine, reduce_range=False, quant_min=None, quant_max=None) [source]

Observer module for computing the quantization parameters based on the running per channel min and max values. This observer uses the tensor min/max statistics to compute the per channel quantization parameters. The module records the running minimum and maximum of incoming tensors, and uses this statistic to compute the quantization parameters.  Parameters

averaging_constant – Averaging constant for min/max.
ch_axis – Channel axis
dtype – Quantized data type
qscheme – Quantization scheme to be used
reduce_range – Reduces the range of the quantized data type by 1 bit
quant_min – Minimum quantization value. If unspecified, it will follow the 8-bit setup.
quant_max – Maximum quantization value. If unspecified, it will follow the 8-bit setup.    The quantization parameters are computed the same way as in MovingAverageMinMaxObserver, with the difference that the running min/max values are stored per channel. Scales and zero points are thus computed per channel as well.  Note If the running minimum equals to the running maximum, the scales and zero_points are set to 1.0 and 0.

class torch.quantization.HistogramObserver(bins=2048, upsample_rate=128, dtype=torch.quint8, qscheme=torch.per_tensor_affine, reduce_range=False) [source]

The module records the running histogram of tensor values along with min/max values. calculate_qparams will calculate scale and zero_point.  Parameters

bins – Number of bins to use for the histogram
upsample_rate – Factor by which the histograms are upsampled, this is used to interpolate histograms with varying ranges across observations
dtype – Quantized data type
qscheme – Quantization scheme to be used
reduce_range – Reduces the range of the quantized data type by 1 bit    The scale and zero point are computed as follows:
 Create the histogram of the incoming inputs.

The histogram is computed continuously, and the ranges per bin change with every new tensor observed.
 Search the distribution in the histogram for optimal min/max values.

The search for the min/max values ensures the minimization of the quantization error with respect to the floating point model.
 Compute the scale and zero point the same way as in the

MinMaxObserver

class torch.quantization.FakeQuantize(observer=<class 'torch.quantization.observer.MovingAverageMinMaxObserver'>, quant_min=0, quant_max=255, **observer_kwargs) [source]

Simulate the quantize and dequantize operations in training time. The output of this module is given by x_out = (clamp(round(x/scale + zero_point), quant_min, quant_max)-zero_point)*scale
scale defines the scale factor used for quantization.
zero_point specifies the quantized value to which 0 in floating point maps to
quant_min specifies the minimum allowable quantized value.
quant_max specifies the maximum allowable quantized value.
fake_quant_enable controls the application of fake quantization on tensors, note that statistics can still be updated.
observer_enable controls statistics collection on tensors

dtype specifies the quantized dtype that is being emulated with fake-quantization,

allowable values are torch.qint8 and torch.quint8. The values of quant_min and quant_max should be chosen to be consistent with the dtype      Parameters

observer (module) – Module for observing statistics on input tensors and calculating scale and zero-point.
quant_min (int) – The minimum allowable quantized value.
quant_max (int) – The maximum allowable quantized value.
observer_kwargs (optional) – Arguments for the observer module   Variables
~FakeQuantize.observer (Module) – User provided module that collects statistics on the input tensor and provides a method to calculate scale and zero-point.

class torch.quantization.NoopObserver(dtype=torch.float16, custom_op_name='') [source]

Observer that doesn’t do anything and just passes its configuration to the quantized module’s .from_float(). Primarily used for quantization to float16 which doesn’t require determining ranges.  Parameters

dtype – Quantized data type
custom_op_name – (temporary) specify this observer for an operator that doesn’t require any observation (Can be used in Graph Mode Passes for special case ops).
 Debugging utilities
torch.quantization.get_observer_dict(mod, target_dict, prefix='') [source]

Traverse the modules and save all observers into dict. This is mainly used for quantization accuracy debug :param mod: the top module we want to save all observers :param prefix: the prefix for the current module :param target_dict: the dictionary used to save all the observers

class torch.quantization.RecordingObserver(**kwargs) [source]

The module is mainly for debug and records the tensor values during runtime.  Parameters

dtype – Quantized data type
qscheme – Quantization scheme to be used
reduce_range – Reduces the range of the quantized data type by 1 bit

nn.intrinsic


torch.quantization.add_observer_(module, qconfig_propagation_list=None, non_leaf_module_list=None, device=None, custom_module_class_mapping=None) [source]

Add observer for the leaf child of the module. This function insert observer module to all leaf child module that has a valid qconfig attribute.  Parameters

module – input module with qconfig attributes for all the leaf modules that we want to quantize
device – parent device, if any
non_leaf_module_list – list of non-leaf modules we want to add observer   Returns
None, module is modified inplace with added observer modules and forward_hooks


torch.quantization.add_quant_dequant(module) [source]

Wrap the leaf child module in QuantWrapper if it has a valid qconfig Note that this function will modify the children of module inplace and it can return a new module which wraps the input module as well.  Parameters

module – input module with qconfig attributes for all the leaf modules
we want to quantize (that) –    Returns
Either the inplace modified module with submodules wrapped in QuantWrapper based on qconfig or a new QuantWrapper module which wraps the input module, the latter case only happens when the input module is a leaf module and we want to quantize it.


torch.quantization.convert(module, mapping=None, inplace=False, remove_qconfig=True, convert_custom_config_dict=None) [source]

Converts submodules in input module to a different module according to mapping by calling from_float method on the target module class. And remove qconfig at the end if remove_qconfig is set to True.  Parameters

module – prepared and calibrated module
mapping – a dictionary that maps from source module type to target module type, can be overwritten to allow swapping user defined Modules
inplace – carry out model transformations in-place, the original module is mutated
convert_custom_config_dict – custom configuration dictionary for convert function    # Example of convert_custom_config_dict:
convert_custom_config_dict = {
    # user will manually define the corresponding quantized
    # module class which has a from_observed class method that converts
    # observed custom module to quantized custom module
    "observed_to_quantized_custom_module_class": {
        ObservedCustomModule: QuantizedCustomModule
    }
}


torch.quantization.default_eval_fn(model, calib_data) [source]

Default evaluation function takes a torch.utils.data.Dataset or a list of input Tensors and run the model on the dataset


class torch.quantization.DeQuantStub [source]

Dequantize stub module, before calibration, this is same as identity, this will be swapped as nnq.DeQuantize in convert.


class torch.quantization.FakeQuantize(observer=<class 'torch.quantization.observer.MovingAverageMinMaxObserver'>, quant_min=0, quant_max=255, **observer_kwargs) [source]

Simulate the quantize and dequantize operations in training time. The output of this module is given by x_out = (clamp(round(x/scale + zero_point), quant_min, quant_max)-zero_point)*scale
scale defines the scale factor used for quantization.
zero_point specifies the quantized value to which 0 in floating point maps to
quant_min specifies the minimum allowable quantized value.
quant_max specifies the maximum allowable quantized value.
fake_quant_enable controls the application of fake quantization on tensors, note that statistics can still be updated.
observer_enable controls statistics collection on tensors

dtype specifies the quantized dtype that is being emulated with fake-quantization,

allowable values are torch.qint8 and torch.quint8. The values of quant_min and quant_max should be chosen to be consistent with the dtype      Parameters

observer (module) – Module for observing statistics on input tensors and calculating scale and zero-point.
quant_min (int) – The minimum allowable quantized value.
quant_max (int) – The maximum allowable quantized value.
observer_kwargs (optional) – Arguments for the observer module   Variables
~FakeQuantize.observer (Module) – User provided module that collects statistics on the input tensor and provides a method to calculate scale and zero-point.


torch.quantization.fuse_modules(model, modules_to_fuse, inplace=False, fuser_func=<function fuse_known_modules>, fuse_custom_config_dict=None) [source]

Fuses a list of modules into a single module Fuses only the following sequence of modules: conv, bn conv, bn, relu conv, relu linear, relu bn, relu All other sequences are left unchanged. For these sequences, replaces the first item in the list with the fused module, replacing the rest of the modules with identity.  Parameters

model – Model containing the modules to be fused
modules_to_fuse – list of list of module names to fuse. Can also be a list of strings if there is only a single list of modules to fuse.
inplace – bool specifying if fusion happens in place on the model, by default a new model is returned
fuser_func – Function that takes in a list of modules and outputs a list of fused modules of the same length. For example, fuser_func([convModule, BNModule]) returns the list [ConvBNModule, nn.Identity()] Defaults to torch.quantization.fuse_known_modules
fuse_custom_config_dict – custom configuration for fusion    # Example of fuse_custom_config_dict
fuse_custom_config_dict = {
    # Additional fuser_method mapping
    "additional_fuser_method_mapping": {
        (torch.nn.Conv2d, torch.nn.BatchNorm2d): fuse_conv_bn
    },
}
  Returns
model with fused modules. A new copy is created if inplace=True.   Examples: >>> m = myModel()
>>> # m is a module containing  the sub-modules below
>>> modules_to_fuse = [ ['conv1', 'bn1', 'relu1'], ['submodule.conv', 'submodule.relu']]
>>> fused_m = torch.quantization.fuse_modules(m, modules_to_fuse)
>>> output = fused_m(input)

>>> m = myModel()
>>> # Alternately provide a single list of modules to fuse
>>> modules_to_fuse = ['conv1', 'bn1', 'relu1']
>>> fused_m = torch.quantization.fuse_modules(m, modules_to_fuse)
>>> output = fused_m(input)


torch.quantization.get_observer_dict(mod, target_dict, prefix='') [source]

Traverse the modules and save all observers into dict. This is mainly used for quantization accuracy debug :param mod: the top module we want to save all observers :param prefix: the prefix for the current module :param target_dict: the dictionary used to save all the observers


class torch.quantization.HistogramObserver(bins=2048, upsample_rate=128, dtype=torch.quint8, qscheme=torch.per_tensor_affine, reduce_range=False) [source]

The module records the running histogram of tensor values along with min/max values. calculate_qparams will calculate scale and zero_point.  Parameters

bins – Number of bins to use for the histogram
upsample_rate – Factor by which the histograms are upsampled, this is used to interpolate histograms with varying ranges across observations
dtype – Quantized data type
qscheme – Quantization scheme to be used
reduce_range – Reduces the range of the quantized data type by 1 bit    The scale and zero point are computed as follows:
 Create the histogram of the incoming inputs.

The histogram is computed continuously, and the ranges per bin change with every new tensor observed.
 Search the distribution in the histogram for optimal min/max values.

The search for the min/max values ensures the minimization of the quantization error with respect to the floating point model.
 Compute the scale and zero point the same way as in the

MinMaxObserver


class torch.quantization.MinMaxObserver(dtype=torch.quint8, qscheme=torch.per_tensor_affine, reduce_range=False, quant_min=None, quant_max=None) [source]

Observer module for computing the quantization parameters based on the running min and max values. This observer uses the tensor min/max statistics to compute the quantization parameters. The module records the running minimum and maximum of incoming tensors, and uses this statistic to compute the quantization parameters.  Parameters

dtype – Quantized data type
qscheme – Quantization scheme to be used
reduce_range – Reduces the range of the quantized data type by 1 bit
quant_min – Minimum quantization value. If unspecified, it will follow the 8-bit setup.
quant_max – Maximum quantization value. If unspecified, it will follow the 8-bit setup.    Given running min/max as xminx_\text{min}  and xmaxx_\text{max} , scale ss  and zero point zz  are computed as: The running minimum/maximum xmin/maxx_\text{min/max}  is computed as:  xmin={min(X)if xmin=Nonemin(xmin,min(X))otherwisexmax={max(X)if xmax=Nonemax(xmax,max(X))otherwise\begin{array}{ll} x_\text{min} &= \begin{cases} \min(X) & \text{if~}x_\text{min} = \text{None} \\ \min\left(x_\text{min}, \min(X)\right) & \text{otherwise} \end{cases}\\ x_\text{max} &= \begin{cases} \max(X) & \text{if~}x_\text{max} = \text{None} \\ \max\left(x_\text{max}, \max(X)\right) & \text{otherwise} \end{cases}\\ \end{array}
where XX  is the observed tensor. The scale ss  and zero point zz  are then computed as:  if Symmetric:s=2max(∣xmin∣,xmax)/(Qmax−Qmin)z={0if dtype is qint8128otherwiseOtherwise:s=(xmax−xmin)/(Qmax−Qmin)z=Qmin−round(xmin/s)\begin{aligned} \text{if Symmetric:}&\\ &s = 2 \max(|x_\text{min}|, x_\text{max}) / \left( Q_\text{max} - Q_\text{min} \right) \\ &z = \begin{cases} 0 & \text{if dtype is qint8} \\ 128 & \text{otherwise} \end{cases}\\ \text{Otherwise:}&\\ &s = \left( x_\text{max} - x_\text{min} \right ) / \left( Q_\text{max} - Q_\text{min} \right ) \\ &z = Q_\text{min} - \text{round}(x_\text{min} / s) \end{aligned}
where QminQ_\text{min}  and QmaxQ_\text{max}  are the minimum and maximum of the quantized data type.  Warning Only works with torch.per_tensor_symmetric quantization scheme   Warning dtype can only take torch.qint8 or torch.quint8.   Note If the running minimum equals to the running maximum, the scale and zero_point are set to 1.0 and 0.


class torch.quantization.MovingAverageMinMaxObserver(averaging_constant=0.01, dtype=torch.quint8, qscheme=torch.per_tensor_affine, reduce_range=False, quant_min=None, quant_max=None) [source]

Observer module for computing the quantization parameters based on the moving average of the min and max values. This observer computes the quantization parameters based on the moving averages of minimums and maximums of the incoming tensors. The module records the average minimum and maximum of incoming tensors, and uses this statistic to compute the quantization parameters.  Parameters

averaging_constant – Averaging constant for min/max.
dtype – Quantized data type
qscheme – Quantization scheme to be used
reduce_range – Reduces the range of the quantized data type by 1 bit
quant_min – Minimum quantization value. If unspecified, it will follow the 8-bit setup.
quant_max – Maximum quantization value. If unspecified, it will follow the 8-bit setup.    The moving average min/max is computed as follows  xmin={min(X)if xmin=None(1−c)xmin+cmin(X)otherwisexmax={max(X)if xmax=None(1−c)xmax+cmax(X)otherwise\begin{array}{ll} x_\text{min} = \begin{cases} \min(X) & \text{if~}x_\text{min} = \text{None} \\ (1 - c) x_\text{min} + c \min(X) & \text{otherwise} \end{cases}\\ x_\text{max} = \begin{cases} \max(X) & \text{if~}x_\text{max} = \text{None} \\ (1 - c) x_\text{max} + c \max(X) & \text{otherwise} \end{cases}\\ \end{array}
where xmin/maxx_\text{min/max}  is the running average min/max, XX  is is the incoming tensor, and cc  is the averaging_constant. The scale and zero point are then computed as in MinMaxObserver.  Note Only works with torch.per_tensor_affine quantization scheme.   Note If the running minimum equals to the running maximum, the scale and zero_point are set to 1.0 and 0.


class torch.quantization.MovingAveragePerChannelMinMaxObserver(averaging_constant=0.01, ch_axis=0, dtype=torch.quint8, qscheme=torch.per_channel_affine, reduce_range=False, quant_min=None, quant_max=None) [source]

Observer module for computing the quantization parameters based on the running per channel min and max values. This observer uses the tensor min/max statistics to compute the per channel quantization parameters. The module records the running minimum and maximum of incoming tensors, and uses this statistic to compute the quantization parameters.  Parameters

averaging_constant – Averaging constant for min/max.
ch_axis – Channel axis
dtype – Quantized data type
qscheme – Quantization scheme to be used
reduce_range – Reduces the range of the quantized data type by 1 bit
quant_min – Minimum quantization value. If unspecified, it will follow the 8-bit setup.
quant_max – Maximum quantization value. If unspecified, it will follow the 8-bit setup.    The quantization parameters are computed the same way as in MovingAverageMinMaxObserver, with the difference that the running min/max values are stored per channel. Scales and zero points are thus computed per channel as well.  Note If the running minimum equals to the running maximum, the scales and zero_points are set to 1.0 and 0.


class torch.quantization.NoopObserver(dtype=torch.float16, custom_op_name='') [source]

Observer that doesn’t do anything and just passes its configuration to the quantized module’s .from_float(). Primarily used for quantization to float16 which doesn’t require determining ranges.  Parameters

dtype – Quantized data type
custom_op_name – (temporary) specify this observer for an operator that doesn’t require any observation (Can be used in Graph Mode Passes for special case ops).


class torch.quantization.ObserverBase(dtype) [source]

Base observer Module. Any observer implementation should derive from this class. Concrete observers should follow the same API. In forward, they will update the statistics of the observed Tensor. And they should provide a calculate_qparams function that computes the quantization parameters given the collected statistics.  Parameters
dtype – Quantized data type
classmethod with_args(**kwargs)
Wrapper that allows creation of class factories. This can be useful when there is a need to create classes with the same constructor arguments, but different instances. Example: >>> Foo.with_args = classmethod(_with_args)
>>> foo_builder = Foo.with_args(a=3, b=4).with_args(answer=42)
>>> foo_instance1 = foo_builder()
>>> foo_instance2 = foo_builder()
>>> id(foo_instance1) == id(foo_instance2)
False


classmethod with_args(**kwargs)
Wrapper that allows creation of class factories. This can be useful when there is a need to create classes with the same constructor arguments, but different instances. Example: >>> Foo.with_args = classmethod(_with_args)
>>> foo_builder = Foo.with_args(a=3, b=4).with_args(answer=42)
>>> foo_instance1 = foo_builder()
>>> foo_instance2 = foo_builder()
>>> id(foo_instance1) == id(foo_instance2)
False


class torch.quantization.PerChannelMinMaxObserver(ch_axis=0, dtype=torch.quint8, qscheme=torch.per_channel_affine, reduce_range=False, quant_min=None, quant_max=None) [source]

Observer module for computing the quantization parameters based on the running per channel min and max values. This observer uses the tensor min/max statistics to compute the per channel quantization parameters. The module records the running minimum and maximum of incoming tensors, and uses this statistic to compute the quantization parameters.  Parameters

ch_axis – Channel axis
dtype – Quantized data type
qscheme – Quantization scheme to be used
reduce_range – Reduces the range of the quantized data type by 1 bit
quant_min – Minimum quantization value. If unspecified, it will follow the 8-bit setup.
quant_max – Maximum quantization value. If unspecified, it will follow the 8-bit setup.    The quantization parameters are computed the same way as in MinMaxObserver, with the difference that the running min/max values are stored per channel. Scales and zero points are thus computed per channel as well.  Note If the running minimum equals to the running maximum, the scales and zero_points are set to 1.0 and 0.


torch.quantization.prepare(model, inplace=False, allow_list=None, observer_non_leaf_module_list=None, prepare_custom_config_dict=None) [source]

Prepares a copy of the model for quantization calibration or quantization-aware training. Quantization configuration should be assigned preemptively to individual submodules in .qconfig attribute. The model will be attached with observer or fake quant modules, and qconfig will be propagated.  Parameters

model – input model to be modified in-place
inplace – carry out model transformations in-place, the original module is mutated
allow_list – list of quantizable modules
observer_non_leaf_module_list – list of non-leaf modules we want to add observer
prepare_custom_config_dict – customization configuration dictionary for prepare function    # Example of prepare_custom_config_dict:
prepare_custom_config_dict = {
    # user will manually define the corresponding observed
    # module class which has a from_float class method that converts
    # float custom module to observed custom module
    "float_to_observed_custom_module_class": {
        CustomModule: ObservedCustomModule
    }
 }


torch.quantization.prepare_qat(model, mapping=None, inplace=False) [source]

Prepares a copy of the model for quantization calibration or quantization-aware training and converts it to quantized version. Quantization configuration should be assigned preemptively to individual submodules in .qconfig attribute.  Parameters

model – input model to be modified in-place
mapping – dictionary that maps float modules to quantized modules to be replaced.
inplace – carry out model transformations in-place, the original module is mutated


torch.quantization.propagate_qconfig_(module, qconfig_dict=None, allow_list=None) [source]

Propagate qconfig through the module hierarchy and assign qconfig attribute on each leaf module  Parameters

module – input module
qconfig_dict – dictionary that maps from name or type of submodule to quantization configuration, qconfig applies to all submodules of a given module unless qconfig for the submodules are specified (when the submodule already has qconfig attribute)   Returns
None, module is modified inplace with qconfig attached


class torch.quantization.QConfig [source]

Describes how to quantize a layer or a part of the network by providing settings (observer classes) for activations and weights respectively. Note that QConfig needs to contain observer classes (like MinMaxObserver) or a callable that returns instances on invocation, not the concrete observer instances themselves. Quantization preparation function will instantiate observers multiple times for each of the layers. Observer classes have usually reasonable default arguments, but they can be overwritten with with_args method (that behaves like functools.partial): my_qconfig = QConfig(activation=MinMaxObserver.with_args(dtype=torch.qint8), weight=default_observer.with_args(dtype=torch.qint8))


class torch.quantization.QConfigDynamic [source]

Describes how to dynamically quantize a layer or a part of the network by providing settings (observer classes) for weights. It’s like QConfig, but for dynamic quantization. Note that QConfigDynamic needs to contain observer classes (like MinMaxObserver) or a callable that returns instances on invocation, not the concrete observer instances themselves. Quantization function will instantiate observers multiple times for each of the layers. Observer classes have usually reasonable default arguments, but they can be overwritten with with_args method (that behaves like functools.partial): my_qconfig = QConfigDynamic(weight=default_observer.with_args(dtype=torch.qint8))


torch.quantization.quantize(model, run_fn, run_args, mapping=None, inplace=False) [source]

Quantize the input float model with post training static quantization. First it will prepare the model for calibration, then it calls run_fn which will run the calibration step, after that we will convert the model to a quantized model.  Parameters

model – input float model
run_fn – a calibration function for calibrating the prepared model
run_args – positional arguments for run_fn

inplace – carry out model transformations in-place, the original module is mutated
mapping – correspondence between original module types and quantized counterparts   Returns
Quantized model.


torch.quantization.quantize_dynamic(model, qconfig_spec=None, dtype=torch.qint8, mapping=None, inplace=False) [source]

Converts a float model to dynamic (i.e. weights-only) quantized model. Replaces specified modules with dynamic weight-only quantized versions and output the quantized model. For simplest usage provide dtype argument that can be float16 or qint8. Weight-only quantization by default is performed for layers with large weights size - i.e. Linear and RNN variants. Fine grained control is possible with qconfig and mapping that act similarly to quantize(). If qconfig is provided, the dtype argument is ignored.  Parameters

model – input model
qconfig_spec –
Either:  A dictionary that maps from name or type of submodule to quantization configuration, qconfig applies to all submodules of a given module unless qconfig for the submodules are specified (when the submodule already has qconfig attribute). Entries in the dictionary need to be QConfigDynamic instances. A set of types and/or submodule names to apply dynamic quantization to, in which case the dtype argument is used to specify the bit-width
inplace – carry out model transformations in-place, the original module is mutated
mapping – maps type of a submodule to a type of corresponding dynamically quantized version with which the submodule needs to be replaced


torch.quantization.quantize_qat(model, run_fn, run_args, inplace=False) [source]

Do quantization aware training and output a quantized model  Parameters

model – input model
run_fn – a function for evaluating the prepared model, can be a function that simply runs the prepared model or a training loop
run_args – positional arguments for run_fn
   Returns
Quantized model.


class torch.quantization.QuantStub(qconfig=None) [source]

Quantize stub module, before calibration, this is same as an observer, it will be swapped as nnq.Quantize in convert.  Parameters
qconfig – quantization configuration for the tensor, if qconfig is not provided, we will get qconfig from parent modules


class torch.quantization.QuantWrapper(module) [source]

A wrapper class that wraps the input module, adds QuantStub and DeQuantStub and surround the call to module with call to quant and dequant modules. This is used by the quantization utility functions to add the quant and dequant modules, before convert function QuantStub will just be observer, it observes the input tensor, after convert, QuantStub will be swapped to nnq.Quantize which does actual quantization. Similarly for DeQuantStub.


class torch.quantization.RecordingObserver(**kwargs) [source]

The module is mainly for debug and records the tensor values during runtime.  Parameters

dtype – Quantized data type
qscheme – Quantization scheme to be used
reduce_range – Reduces the range of the quantized data type by 1 bit


torch.quantization.swap_module(mod, mapping, custom_module_class_mapping) [source]

Swaps the module if it has a quantized counterpart and it has an observer attached.  Parameters

mod – input module
mapping – a dictionary that maps from nn module to nnq module   Returns
The corresponding quantized module of mod


torch.quantize_per_channel(input, scales, zero_points, axis, dtype) → Tensor
Converts a float tensor to a per-channel quantized tensor with given scales and zero points.  Parameters

input (Tensor) – float tensor to quantize
scales (Tensor) – float 1D tensor of scales to use, size should match input.size(axis)

zero_points (int) – integer 1D tensor of offset to use, size should match input.size(axis)

axis (int) – dimension on which apply per-channel quantization
dtype (torch.dtype) – the desired data type of returned tensor. Has to be one of the quantized dtypes: torch.quint8, torch.qint8, torch.qint32
   Returns
A newly quantized tensor  Return type
Tensor   Example: >>> x = torch.tensor([[-1.0, 0.0], [1.0, 2.0]])
>>> torch.quantize_per_channel(x, torch.tensor([0.1, 0.01]), torch.tensor([10, 0]), 0, torch.quint8)
tensor([[-1.,  0.],
        [ 1.,  2.]], size=(2, 2), dtype=torch.quint8,
       quantization_scheme=torch.per_channel_affine,
       scale=tensor([0.1000, 0.0100], dtype=torch.float64),
       zero_point=tensor([10,  0]), axis=0)
>>> torch.quantize_per_channel(x, torch.tensor([0.1, 0.01]), torch.tensor([10, 0]), 0, torch.quint8).int_repr()
tensor([[  0,  10],
        [100, 200]], dtype=torch.uint8)


torch.quantize_per_tensor(input, scale, zero_point, dtype) → Tensor
Converts a float tensor to a quantized tensor with given scale and zero point.  Parameters

input (Tensor) – float tensor to quantize
scale (float) – scale to apply in quantization formula
zero_point (int) – offset in integer value that maps to float zero
dtype (torch.dtype) – the desired data type of returned tensor. Has to be one of the quantized dtypes: torch.quint8, torch.qint8, torch.qint32
   Returns
A newly quantized tensor  Return type
Tensor   Example: >>> torch.quantize_per_tensor(torch.tensor([-1.0, 0.0, 1.0, 2.0]), 0.1, 10, torch.quint8)
tensor([-1.,  0.,  1.,  2.], size=(4,), dtype=torch.quint8,
       quantization_scheme=torch.per_tensor_affine, scale=0.1, zero_point=10)
>>> torch.quantize_per_tensor(torch.tensor([-1.0, 0.0, 1.0, 2.0]), 0.1, 10, torch.quint8).int_repr()
tensor([ 0, 10, 20, 30], dtype=torch.uint8)


class torch.quasirandom.SobolEngine(dimension, scramble=False, seed=None) [source]

The torch.quasirandom.SobolEngine is an engine for generating (scrambled) Sobol sequences. Sobol sequences are an example of low discrepancy quasi-random sequences. This implementation of an engine for Sobol sequences is capable of sampling sequences up to a maximum dimension of 21201. It uses direction numbers from https://web.maths.unsw.edu.au/~fkuo/sobol/ obtained using the search criterion D(6) up to the dimension 21201. This is the recommended choice by the authors. References  Art B. Owen. Scrambling Sobol and Niederreiter-Xing points. Journal of Complexity, 14(4):466-489, December 1998. I. M. Sobol. The distribution of points in a cube and the accurate evaluation of integrals. Zh. Vychisl. Mat. i Mat. Phys., 7:784-802, 1967.   Parameters

dimension (Int) – The dimensionality of the sequence to be drawn
scramble (bool, optional) – Setting this to True will produce scrambled Sobol sequences. Scrambling is capable of producing better Sobol sequences. Default: False.
seed (Int, optional) – This is the seed for the scrambling. The seed of the random number generator is set to this, if specified. Otherwise, it uses a random seed. Default: None
    Examples: >>> soboleng = torch.quasirandom.SobolEngine(dimension=5)
>>> soboleng.draw(3)
tensor([[0.5000, 0.5000, 0.5000, 0.5000, 0.5000],
        [0.7500, 0.2500, 0.7500, 0.2500, 0.7500],
        [0.2500, 0.7500, 0.2500, 0.7500, 0.2500]])

draw(n=1, out=None, dtype=torch.float32) [source]

Function to draw a sequence of n points from a Sobol sequence. Note that the samples are dependent on the previous samples. The size of the result is (n,dimension)(n, dimension) .  Parameters

n (Int, optional) – The length of sequence of points to draw. Default: 1
out (Tensor, optional) – The output tensor
dtype (torch.dtype, optional) – the desired data type of the returned tensor. Default: torch.float32

draw_base2(m, out=None, dtype=torch.float32) [source]

Function to draw a sequence of 2**m points from a Sobol sequence. Note that the samples are dependent on the previous samples. The size of the result is (2∗∗m,dimension)(2**m, dimension) .  Parameters

m (Int) – The (base2) exponent of the number of points to draw.
out (Tensor, optional) – The output tensor
dtype (torch.dtype, optional) – the desired data type of the returned tensor. Default: torch.float32

fast_forward(n) [source]

Function to fast-forward the state of the SobolEngine by n steps. This is equivalent to drawing n samples without using the samples.  Parameters
n (Int) – The number of steps to fast-forward by.

reset() [source]

Function to reset the SobolEngine to base state.


draw(n=1, out=None, dtype=torch.float32) [source]

Function to draw a sequence of n points from a Sobol sequence. Note that the samples are dependent on the previous samples. The size of the result is (n,dimension)(n, dimension) .  Parameters

n (Int, optional) – The length of sequence of points to draw. Default: 1
out (Tensor, optional) – The output tensor
dtype (torch.dtype, optional) – the desired data type of the returned tensor. Default: torch.float32


draw_base2(m, out=None, dtype=torch.float32) [source]

Function to draw a sequence of 2**m points from a Sobol sequence. Note that the samples are dependent on the previous samples. The size of the result is (2∗∗m,dimension)(2**m, dimension) .  Parameters

m (Int) – The (base2) exponent of the number of points to draw.
out (Tensor, optional) – The output tensor
dtype (torch.dtype, optional) – the desired data type of the returned tensor. Default: torch.float32


fast_forward(n) [source]

Function to fast-forward the state of the SobolEngine by n steps. This is equivalent to drawing n samples without using the samples.  Parameters
n (Int) – The number of steps to fast-forward by.


reset() [source]

Function to reset the SobolEngine to base state.


torch.rad2deg(input, *, out=None) → Tensor
Returns a new tensor with each of the elements of input converted from angles in radians to degrees.  Parameters
input (Tensor) – the input tensor.  Keyword Arguments
out (Tensor, optional) – the output tensor.   Example: >>> a = torch.tensor([[3.142, -3.142], [6.283, -6.283], [1.570, -1.570]])
>>> torch.rad2deg(a)
tensor([[ 180.0233, -180.0233],
        [ 359.9894, -359.9894],
        [  89.9544,  -89.9544]])


torch.rand(*size, *, out=None, dtype=None, layout=torch.strided, device=None, requires_grad=False) → Tensor
Returns a tensor filled with random numbers from a uniform distribution on the interval [0,1)[0, 1)  The shape of the tensor is defined by the variable argument size.  Parameters
size (int...) – a sequence of integers defining the shape of the output tensor. Can be a variable number of arguments or a collection like a list or tuple.  Keyword Arguments

out (Tensor, optional) – the output tensor.
dtype (torch.dtype, optional) – the desired data type of returned tensor. Default: if None, uses a global default (see torch.set_default_tensor_type()).
layout (torch.layout, optional) – the desired layout of returned Tensor. Default: torch.strided.
device (torch.device, optional) – the desired device of returned tensor. Default: if None, uses the current device for the default tensor type (see torch.set_default_tensor_type()). device will be the CPU for CPU tensor types and the current CUDA device for CUDA tensor types.
requires_grad (bool, optional) – If autograd should record operations on the returned tensor. Default: False.    Example: >>> torch.rand(4)
tensor([ 0.5204,  0.2503,  0.3525,  0.5673])
>>> torch.rand(2, 3)
tensor([[ 0.8237,  0.5781,  0.6879],
        [ 0.3816,  0.7249,  0.0998]])


torch.randint(low=0, high, size, *, generator=None, out=None, dtype=None, layout=torch.strided, device=None, requires_grad=False) → Tensor
Returns a tensor filled with random integers generated uniformly between low (inclusive) and high (exclusive). The shape of the tensor is defined by the variable argument size.  Note With the global dtype default (torch.float32), this function returns a tensor with dtype torch.int64.   Parameters

low (int, optional) – Lowest integer to be drawn from the distribution. Default: 0.
high (int) – One above the highest integer to be drawn from the distribution.
size (tuple) – a tuple defining the shape of the output tensor.   Keyword Arguments

generator (torch.Generator, optional) – a pseudorandom number generator for sampling
out (Tensor, optional) – the output tensor.
dtype (torch.dtype, optional) – the desired data type of returned tensor. Default: if None, uses a global default (see torch.set_default_tensor_type()).
layout (torch.layout, optional) – the desired layout of returned Tensor. Default: torch.strided.
device (torch.device, optional) – the desired device of returned tensor. Default: if None, uses the current device for the default tensor type (see torch.set_default_tensor_type()). device will be the CPU for CPU tensor types and the current CUDA device for CUDA tensor types.
requires_grad (bool, optional) – If autograd should record operations on the returned tensor. Default: False.    Example: >>> torch.randint(3, 5, (3,))
tensor([4, 3, 4])

>>> torch.randint(10, (2, 2))
tensor([[0, 2],
        [5, 5]])

>>> torch.randint(3, 10, (2, 2))
tensor([[4, 5],
        [6, 7]])


torch.randint_like(input, low=0, high, *, dtype=None, layout=torch.strided, device=None, requires_grad=False, memory_format=torch.preserve_format) → Tensor
Returns a tensor with the same shape as Tensor input filled with random integers generated uniformly between low (inclusive) and high (exclusive).  Parameters

input (Tensor) – the size of input will determine size of the output tensor.
low (int, optional) – Lowest integer to be drawn from the distribution. Default: 0.
high (int) – One above the highest integer to be drawn from the distribution.   Keyword Arguments

dtype (torch.dtype, optional) – the desired data type of returned Tensor. Default: if None, defaults to the dtype of input.
layout (torch.layout, optional) – the desired layout of returned tensor. Default: if None, defaults to the layout of input.
device (torch.device, optional) – the desired device of returned tensor. Default: if None, defaults to the device of input.
requires_grad (bool, optional) – If autograd should record operations on the returned tensor. Default: False.
memory_format (torch.memory_format, optional) – the desired memory format of returned Tensor. Default: torch.preserve_format.


torch.randn(*size, *, out=None, dtype=None, layout=torch.strided, device=None, requires_grad=False) → Tensor
Returns a tensor filled with random numbers from a normal distribution with mean 0 and variance 1 (also called the standard normal distribution).  outi∼N(0,1)\text{out}_{i} \sim \mathcal{N}(0, 1)
The shape of the tensor is defined by the variable argument size.  Parameters
size (int...) – a sequence of integers defining the shape of the output tensor. Can be a variable number of arguments or a collection like a list or tuple.  Keyword Arguments

out (Tensor, optional) – the output tensor.
dtype (torch.dtype, optional) – the desired data type of returned tensor. Default: if None, uses a global default (see torch.set_default_tensor_type()).
layout (torch.layout, optional) – the desired layout of returned Tensor. Default: torch.strided.
device (torch.device, optional) – the desired device of returned tensor. Default: if None, uses the current device for the default tensor type (see torch.set_default_tensor_type()). device will be the CPU for CPU tensor types and the current CUDA device for CUDA tensor types.
requires_grad (bool, optional) – If autograd should record operations on the returned tensor. Default: False.    Example: >>> torch.randn(4)
tensor([-2.1436,  0.9966,  2.3426, -0.6366])
>>> torch.randn(2, 3)
tensor([[ 1.5954,  2.8929, -1.0923],
        [ 1.1719, -0.4709, -0.1996]])


torch.randn_like(input, *, dtype=None, layout=None, device=None, requires_grad=False, memory_format=torch.preserve_format) → Tensor
Returns a tensor with the same size as input that is filled with random numbers from a normal distribution with mean 0 and variance 1. torch.randn_like(input) is equivalent to torch.randn(input.size(), dtype=input.dtype, layout=input.layout, device=input.device).  Parameters
input (Tensor) – the size of input will determine size of the output tensor.  Keyword Arguments

dtype (torch.dtype, optional) – the desired data type of returned Tensor. Default: if None, defaults to the dtype of input.
layout (torch.layout, optional) – the desired layout of returned tensor. Default: if None, defaults to the layout of input.
device (torch.device, optional) – the desired device of returned tensor. Default: if None, defaults to the device of input.
requires_grad (bool, optional) – If autograd should record operations on the returned tensor. Default: False.
memory_format (torch.memory_format, optional) – the desired memory format of returned Tensor. Default: torch.preserve_format.


torch.random
torch.random.fork_rng(devices=None, enabled=True, _caller='fork_rng', _devices_kw='devices') [source]

Forks the RNG, so that when you return, the RNG is reset to the state that it was previously in.  Parameters

devices (iterable of CUDA IDs) – CUDA devices for which to fork the RNG. CPU RNG state is always forked. By default, fork_rng() operates on all devices, but will emit a warning if your machine has a lot of devices, since this function will run very slowly in that case. If you explicitly specify devices, this warning will be suppressed
enabled (bool) – if False, the RNG is not forked. This is a convenience argument for easily disabling the context manager without having to delete it and unindent your Python code under it.

torch.random.get_rng_state() [source]

Returns the random number generator state as a torch.ByteTensor.

torch.random.initial_seed() [source]

Returns the initial seed for generating random numbers as a Python long.

torch.random.manual_seed(seed) [source]

Sets the seed for generating random numbers. Returns a torch.Generator object.  Parameters
seed (int) – The desired seed. Value must be within the inclusive range [-0x8000_0000_0000_0000, 0xffff_ffff_ffff_ffff]. Otherwise, a RuntimeError is raised. Negative inputs are remapped to positive values with the formula 0xffff_ffff_ffff_ffff + seed.

torch.random.seed() [source]

Sets the seed for generating random numbers to a non-deterministic random number. Returns a 64 bit number used to seed the RNG.

torch.random.set_rng_state(new_state) [source]

Sets the random number generator state.  Parameters
new_state (torch.ByteTensor) – The desired state


torch.random.fork_rng(devices=None, enabled=True, _caller='fork_rng', _devices_kw='devices') [source]

Forks the RNG, so that when you return, the RNG is reset to the state that it was previously in.  Parameters

devices (iterable of CUDA IDs) – CUDA devices for which to fork the RNG. CPU RNG state is always forked. By default, fork_rng() operates on all devices, but will emit a warning if your machine has a lot of devices, since this function will run very slowly in that case. If you explicitly specify devices, this warning will be suppressed
enabled (bool) – if False, the RNG is not forked. This is a convenience argument for easily disabling the context manager without having to delete it and unindent your Python code under it.


torch.random.get_rng_state() [source]

Returns the random number generator state as a torch.ByteTensor.


torch.random.initial_seed() [source]

Returns the initial seed for generating random numbers as a Python long.


torch.random.manual_seed(seed) [source]

Sets the seed for generating random numbers. Returns a torch.Generator object.  Parameters
seed (int) – The desired seed. Value must be within the inclusive range [-0x8000_0000_0000_0000, 0xffff_ffff_ffff_ffff]. Otherwise, a RuntimeError is raised. Negative inputs are remapped to positive values with the formula 0xffff_ffff_ffff_ffff + seed.


torch.random.seed() [source]

Sets the seed for generating random numbers to a non-deterministic random number. Returns a 64 bit number used to seed the RNG.


torch.random.set_rng_state(new_state) [source]

Sets the random number generator state.  Parameters
new_state (torch.ByteTensor) – The desired state


torch.randperm(n, *, generator=None, out=None, dtype=torch.int64, layout=torch.strided, device=None, requires_grad=False, pin_memory=False) → Tensor
Returns a random permutation of integers from 0 to n - 1.  Parameters
n (int) – the upper bound (exclusive)  Keyword Arguments

generator (torch.Generator, optional) – a pseudorandom number generator for sampling
out (Tensor, optional) – the output tensor.
dtype (torch.dtype, optional) – the desired data type of returned tensor. Default: torch.int64.
layout (torch.layout, optional) – the desired layout of returned Tensor. Default: torch.strided.
device (torch.device, optional) – the desired device of returned tensor. Default: if None, uses the current device for the default tensor type (see torch.set_default_tensor_type()). device will be the CPU for CPU tensor types and the current CUDA device for CUDA tensor types.
requires_grad (bool, optional) – If autograd should record operations on the returned tensor. Default: False.
pin_memory (bool, optional) – If set, returned tensor would be allocated in the pinned memory. Works only for CPU tensors. Default: False.    Example: >>> torch.randperm(4)
tensor([2, 1, 0, 3])


torch.rand_like(input, *, dtype=None, layout=None, device=None, requires_grad=False, memory_format=torch.preserve_format) → Tensor
Returns a tensor with the same size as input that is filled with random numbers from a uniform distribution on the interval [0,1)[0, 1) . torch.rand_like(input) is equivalent to torch.rand(input.size(), dtype=input.dtype, layout=input.layout, device=input.device).  Parameters
input (Tensor) – the size of input will determine size of the output tensor.  Keyword Arguments

dtype (torch.dtype, optional) – the desired data type of returned Tensor. Default: if None, defaults to the dtype of input.
layout (torch.layout, optional) – the desired layout of returned tensor. Default: if None, defaults to the layout of input.
device (torch.device, optional) – the desired device of returned tensor. Default: if None, defaults to the device of input.
requires_grad (bool, optional) – If autograd should record operations on the returned tensor. Default: False.
memory_format (torch.memory_format, optional) – the desired memory format of returned Tensor. Default: torch.preserve_format.


torch.range(start=0, end, step=1, *, out=None, dtype=None, layout=torch.strided, device=None, requires_grad=False) → Tensor
Returns a 1-D tensor of size ⌊end−startstep⌋+1\left\lfloor \frac{\text{end} - \text{start}}{\text{step}} \right\rfloor + 1  with values from start to end with step step. Step is the gap between two values in the tensor.  outi+1=outi+step.\text{out}_{i+1} = \text{out}_i + \text{step}.
 Warning This function is deprecated and will be removed in a future release because its behavior is inconsistent with Python’s range builtin. Instead, use torch.arange(), which produces values in [start, end).   Parameters

start (float) – the starting value for the set of points. Default: 0.
end (float) – the ending value for the set of points
step (float) – the gap between each pair of adjacent points. Default: 1.   Keyword Arguments

out (Tensor, optional) – the output tensor.
dtype (torch.dtype, optional) – the desired data type of returned tensor. Default: if None, uses a global default (see torch.set_default_tensor_type()). If dtype is not given, infer the data type from the other input arguments. If any of start, end, or stop are floating-point, the dtype is inferred to be the default dtype, see get_default_dtype(). Otherwise, the dtype is inferred to be torch.int64.
layout (torch.layout, optional) – the desired layout of returned Tensor. Default: torch.strided.
device (torch.device, optional) – the desired device of returned tensor. Default: if None, uses the current device for the default tensor type (see torch.set_default_tensor_type()). device will be the CPU for CPU tensor types and the current CUDA device for CUDA tensor types.
requires_grad (bool, optional) – If autograd should record operations on the returned tensor. Default: False.    Example: >>> torch.range(1, 4)
tensor([ 1.,  2.,  3.,  4.])
>>> torch.range(1, 4, 0.5)
tensor([ 1.0000,  1.5000,  2.0000,  2.5000,  3.0000,  3.5000,  4.0000])


torch.ravel(input) → Tensor
Return a contiguous flattened tensor. A copy is made only if needed.  Parameters
input (Tensor) – the input tensor.   Example: >>> t = torch.tensor([[[1, 2],
...                    [3, 4]],
...                   [[5, 6],
...                    [7, 8]]])
>>> torch.ravel(t)
tensor([1, 2, 3, 4, 5, 6, 7, 8])


torch.real(input) → Tensor
Returns a new tensor containing real values of the self tensor. The returned tensor and self share the same underlying storage.  Warning real() is only supported for tensors with complex dtypes.   Parameters
input (Tensor) – the input tensor.    Example::

>>> x=torch.randn(4, dtype=torch.cfloat)
>>> x
tensor([(0.3100+0.3553j), (-0.5445-0.7896j), (-1.6492-0.0633j), (-0.0638-0.8119j)])
>>> x.real
tensor([ 0.3100, -0.5445, -1.6492, -0.0638])


torch.reciprocal(input, *, out=None) → Tensor
Returns a new tensor with the reciprocal of the elements of input  Note Unlike NumPy’s reciprocal, torch.reciprocal supports integral inputs. Integral inputs to reciprocal are automatically promoted to the default scalar type.   outi=1inputi\text{out}_{i} = \frac{1}{\text{input}_{i}}
 Parameters
input (Tensor) – the input tensor.  Keyword Arguments
out (Tensor, optional) – the output tensor.   Example: >>> a = torch.randn(4)
>>> a
tensor([-0.4595, -2.1219, -1.4314,  0.7298])
>>> torch.reciprocal(a)
tensor([-2.1763, -0.4713, -0.6986,  1.3702])


torch.remainder(input, other, *, out=None) → Tensor
Computes the element-wise remainder of division. The dividend and divisor may contain both for integer and floating point numbers. The remainder has the same sign as the divisor other. Supports broadcasting to a common shape, type promotion, and integer and float inputs.  Note Complex inputs are not supported. In some cases, it is not mathematically possible to satisfy the definition of a modulo operation with complex numbers. See torch.fmod() for how division by zero is handled.   Parameters

input (Tensor) – the dividend
other (Tensor or Scalar) – the divisor   Keyword Arguments
out (Tensor, optional) – the output tensor.   Example: >>> torch.remainder(torch.tensor([-3., -2, -1, 1, 2, 3]), 2)
tensor([ 1.,  0.,  1.,  1.,  0.,  1.])
>>> torch.remainder(torch.tensor([1, 2, 3, 4, 5]), 1.5)
tensor([ 1.0000,  0.5000,  0.0000,  1.0000,  0.5000])
  See also torch.fmod(), which computes the element-wise remainder of division equivalently to the C library function fmod().


torch.renorm(input, p, dim, maxnorm, *, out=None) → Tensor
Returns a tensor where each sub-tensor of input along dimension dim is normalized such that the p-norm of the sub-tensor is lower than the value maxnorm  Note If the norm of a row is lower than maxnorm, the row is unchanged   Parameters

input (Tensor) – the input tensor.
p (float) – the power for the norm computation
dim (int) – the dimension to slice over to get the sub-tensors
maxnorm (float) – the maximum norm to keep each sub-tensor under   Keyword Arguments
out (Tensor, optional) – the output tensor.   Example: >>> x = torch.ones(3, 3)
>>> x[1].fill_(2)
tensor([ 2.,  2.,  2.])
>>> x[2].fill_(3)
tensor([ 3.,  3.,  3.])
>>> x
tensor([[ 1.,  1.,  1.],
        [ 2.,  2.,  2.],
        [ 3.,  3.,  3.]])
>>> torch.renorm(x, 1, 0, 5)
tensor([[ 1.0000,  1.0000,  1.0000],
        [ 1.6667,  1.6667,  1.6667],
        [ 1.6667,  1.6667,  1.6667]])


torch.repeat_interleave(input, repeats, dim=None) → Tensor
Repeat elements of a tensor.  Warning This is different from torch.Tensor.repeat() but similar to numpy.repeat.   Parameters

input (Tensor) – the input tensor.
repeats (Tensor or int) – The number of repetitions for each element. repeats is broadcasted to fit the shape of the given axis.
dim (int, optional) – The dimension along which to repeat values. By default, use the flattened input array, and return a flat output array.   Returns
Repeated tensor which has the same shape as input, except along the given axis.  Return type
Tensor   Example: >>> x = torch.tensor([1, 2, 3])
>>> x.repeat_interleave(2)
tensor([1, 1, 2, 2, 3, 3])
>>> y = torch.tensor([[1, 2], [3, 4]])
>>> torch.repeat_interleave(y, 2)
tensor([1, 1, 2, 2, 3, 3, 4, 4])
>>> torch.repeat_interleave(y, 3, dim=1)
tensor([[1, 1, 1, 2, 2, 2],
        [3, 3, 3, 4, 4, 4]])
>>> torch.repeat_interleave(y, torch.tensor([1, 2]), dim=0)
tensor([[1, 2],
        [3, 4],
        [3, 4]])

torch.repeat_interleave(repeats) → Tensor
 If the repeats is tensor([n1, n2, n3, …]), then the output will be tensor([0, 0, …, 1, 1, …, 2, 2, …, …]) where 0 appears n1 times, 1 appears n2 times, 2 appears n3 times, etc.


torch.reshape(input, shape) → Tensor
Returns a tensor with the same data and number of elements as input, but with the specified shape. When possible, the returned tensor will be a view of input. Otherwise, it will be a copy. Contiguous inputs and inputs with compatible strides can be reshaped without copying, but you should not depend on the copying vs. viewing behavior. See torch.Tensor.view() on when it is possible to return a view. A single dimension may be -1, in which case it’s inferred from the remaining dimensions and the number of elements in input.  Parameters

input (Tensor) – the tensor to be reshaped
shape (tuple of python:ints) – the new shape    Example: >>> a = torch.arange(4.)
>>> torch.reshape(a, (2, 2))
tensor([[ 0.,  1.],
        [ 2.,  3.]])
>>> b = torch.tensor([[0, 1], [2, 3]])
>>> torch.reshape(b, (-1,))
tensor([ 0,  1,  2,  3])


torch.result_type(tensor1, tensor2) → dtype
Returns the torch.dtype that would result from performing an arithmetic operation on the provided input tensors. See type promotion documentation for more information on the type promotion logic.  Parameters

tensor1 (Tensor or Number) – an input tensor or number
tensor2 (Tensor or Number) – an input tensor or number    Example: >>> torch.result_type(torch.tensor([1, 2], dtype=torch.int), 1.0)
torch.float32
>>> torch.result_type(torch.tensor([1, 2], dtype=torch.uint8), torch.tensor(1))
torch.uint8


torch.roll(input, shifts, dims=None) → Tensor
Roll the tensor along the given dimension(s). Elements that are shifted beyond the last position are re-introduced at the first position. If a dimension is not specified, the tensor will be flattened before rolling and then restored to the original shape.  Parameters

input (Tensor) – the input tensor.
shifts (int or tuple of python:ints) – The number of places by which the elements of the tensor are shifted. If shifts is a tuple, dims must be a tuple of the same size, and each dimension will be rolled by the corresponding value
dims (int or tuple of python:ints) – Axis along which to roll    Example: >>> x = torch.tensor([1, 2, 3, 4, 5, 6, 7, 8]).view(4, 2)
>>> x
tensor([[1, 2],
        [3, 4],
        [5, 6],
        [7, 8]])
>>> torch.roll(x, 1, 0)
tensor([[7, 8],
        [1, 2],
        [3, 4],
        [5, 6]])
>>> torch.roll(x, -1, 0)
tensor([[3, 4],
        [5, 6],
        [7, 8],
        [1, 2]])
>>> torch.roll(x, shifts=(2, 1), dims=(0, 1))
tensor([[6, 5],
        [8, 7],
        [2, 1],
        [4, 3]])


torch.rot90(input, k, dims) → Tensor
Rotate a n-D tensor by 90 degrees in the plane specified by dims axis. Rotation direction is from the first towards the second axis if k > 0, and from the second towards the first for k < 0.  Parameters

input (Tensor) – the input tensor.
k (int) – number of times to rotate
dims (a list or tuple) – axis to rotate    Example: >>> x = torch.arange(4).view(2, 2)
>>> x
tensor([[0, 1],
        [2, 3]])
>>> torch.rot90(x, 1, [0, 1])
tensor([[1, 3],
        [0, 2]])

>>> x = torch.arange(8).view(2, 2, 2)
>>> x
tensor([[[0, 1],
         [2, 3]],

        [[4, 5],
         [6, 7]]])
>>> torch.rot90(x, 1, [1, 2])
tensor([[[1, 3],
         [0, 2]],

        [[5, 7],
         [4, 6]]])


torch.round(input, *, out=None) → Tensor
Returns a new tensor with each of the elements of input rounded to the closest integer.  Parameters
input (Tensor) – the input tensor.  Keyword Arguments
out (Tensor, optional) – the output tensor.   Example: >>> a = torch.randn(4)
>>> a
tensor([ 0.9920,  0.6077,  0.9734, -1.0362])
>>> torch.round(a)
tensor([ 1.,  1.,  1., -1.])


torch.row_stack(tensors, *, out=None) → Tensor
Alias of torch.vstack().


torch.rsqrt(input, *, out=None) → Tensor
Returns a new tensor with the reciprocal of the square-root of each of the elements of input.  outi=1inputi\text{out}_{i} = \frac{1}{\sqrt{\text{input}_{i}}}
 Parameters
input (Tensor) – the input tensor.  Keyword Arguments
out (Tensor, optional) – the output tensor.   Example: >>> a = torch.randn(4)
>>> a
tensor([-0.0370,  0.2970,  1.5420, -0.9105])
>>> torch.rsqrt(a)
tensor([    nan,  1.8351,  0.8053,     nan])


torch.save(obj, f, pickle_module=<module 'pickle' from '/home/matti/miniconda3/lib/python3.7/pickle.py'>, pickle_protocol=2, _use_new_zipfile_serialization=True) [source]

Saves an object to a disk file. See also: saving-loading-tensors  Parameters

obj – saved object
f – a file-like object (has to implement write and flush) or a string or os.PathLike object containing a file name
pickle_module – module used for pickling metadata and objects
pickle_protocol – can be specified to override the default protocol     Note A common PyTorch convention is to save tensors using .pt file extension.   Note PyTorch preserves storage sharing across serialization. See preserve-storage-sharing for more details.   Note The 1.6 release of PyTorch switched torch.save to use a new zipfile-based file format. torch.load still retains the ability to load files in the old format. If for any reason you want torch.save to use the old format, pass the kwarg _use_new_zipfile_serialization=False.  Example >>> # Save to file
>>> x = torch.tensor([0, 1, 2, 3, 4])
>>> torch.save(x, 'tensor.pt')
>>> # Save to io.BytesIO buffer
>>> buffer = io.BytesIO()
>>> torch.save(x, buffer)


torch.scatter(input, dim, index, src) → Tensor
Out-of-place version of torch.Tensor.scatter_()


torch.scatter_add(input, dim, index, src) → Tensor
Out-of-place version of torch.Tensor.scatter_add_()


torch.searchsorted(sorted_sequence, values, *, out_int32=False, right=False, out=None) → Tensor
Find the indices from the innermost dimension of sorted_sequence such that, if the corresponding values in values were inserted before the indices, the order of the corresponding innermost dimension within sorted_sequence would be preserved. Return a new tensor with the same size as values. If right is False (default), then the left boundary of sorted_sequence is closed. More formally, the returned index satisfies the following rules:
sorted_sequence right returned index satisfies
1-D False sorted_sequence[i-1] < values[m][n]...[l][x] <= sorted_sequence[i]
1-D True sorted_sequence[i-1] <= values[m][n]...[l][x] < sorted_sequence[i]
N-D False sorted_sequence[m][n]...[l][i-1] < values[m][n]...[l][x] <= sorted_sequence[m][n]...[l][i]
N-D True sorted_sequence[m][n]...[l][i-1] <= values[m][n]...[l][x] < sorted_sequence[m][n]...[l][i]    Parameters

sorted_sequence (Tensor) – N-D or 1-D tensor, containing monotonically increasing sequence on the innermost dimension.
values (Tensor or Scalar) – N-D tensor or a Scalar containing the search value(s).   Keyword Arguments

out_int32 (bool, optional) – indicate the output data type. torch.int32 if True, torch.int64 otherwise. Default value is False, i.e. default output data type is torch.int64.
right (bool, optional) – if False, return the first suitable location that is found. If True, return the last such index. If no suitable index found, return 0 for non-numerical value (eg. nan, inf) or the size of innermost dimension within sorted_sequence (one pass the last index of the innermost dimension). In other words, if False, gets the lower bound index for each value in values on the corresponding innermost dimension of the sorted_sequence. If True, gets the upper bound index instead. Default value is False.
out (Tensor, optional) – the output tensor, must be the same size as values if provided.     Note If your use case is always 1-D sorted sequence, torch.bucketize() is preferred, because it has fewer dimension checks resulting in slightly better performance.  Example: >>> sorted_sequence = torch.tensor([[1, 3, 5, 7, 9], [2, 4, 6, 8, 10]])
>>> sorted_sequence
tensor([[ 1,  3,  5,  7,  9],
        [ 2,  4,  6,  8, 10]])
>>> values = torch.tensor([[3, 6, 9], [3, 6, 9]])
>>> values
tensor([[3, 6, 9],
        [3, 6, 9]])
>>> torch.searchsorted(sorted_sequence, values)
tensor([[1, 3, 4],
        [1, 2, 4]])
>>> torch.searchsorted(sorted_sequence, values, right=True)
tensor([[2, 3, 5],
        [1, 3, 4]])

>>> sorted_sequence_1d = torch.tensor([1, 3, 5, 7, 9])
>>> sorted_sequence_1d
tensor([1, 3, 5, 7, 9])
>>> torch.searchsorted(sorted_sequence_1d, values)
tensor([[1, 3, 4],
        [1, 3, 4]])


torch.seed() [source]

Sets the seed for generating random numbers to a non-deterministic random number. Returns a 64 bit number used to seed the RNG.


torch.set_default_dtype(d) [source]

Sets the default floating point dtype to d. This dtype is:  The inferred dtype for python floats in torch.tensor(). Used to infer dtype for python complex numbers. The default complex dtype is set to torch.complex128 if default floating point dtype is torch.float64, otherwise it’s set to torch.complex64
  The default floating point dtype is initially torch.float32.  Parameters
d (torch.dtype) – the floating point dtype to make the default   Example >>> # initial default for floating point is torch.float32
>>> torch.tensor([1.2, 3]).dtype
torch.float32
>>> # initial default for floating point is torch.complex64
>>> torch.tensor([1.2, 3j]).dtype
torch.complex64
>>> torch.set_default_dtype(torch.float64)
>>> torch.tensor([1.2, 3]).dtype    # a new floating point tensor
torch.float64
>>> torch.tensor([1.2, 3j]).dtype   # a new complex tensor
torch.complex128


torch.set_default_tensor_type(t) [source]

Sets the default torch.Tensor type to floating point tensor type t. This type will also be used as default floating point type for type inference in torch.tensor(). The default floating point tensor type is initially torch.FloatTensor.  Parameters
t (type or string) – the floating point tensor type or its name   Example: >>> torch.tensor([1.2, 3]).dtype    # initial default for floating point is torch.float32
torch.float32
>>> torch.set_default_tensor_type(torch.DoubleTensor)
>>> torch.tensor([1.2, 3]).dtype    # a new floating point tensor
torch.float64


torch.set_flush_denormal(mode) → bool
Disables denormal floating numbers on CPU. Returns True if your system supports flushing denormal numbers and it successfully configures flush denormal mode. set_flush_denormal() is only supported on x86 architectures supporting SSE3.  Parameters
mode (bool) – Controls whether to enable flush denormal mode or not   Example: >>> torch.set_flush_denormal(True)
True
>>> torch.tensor([1e-323], dtype=torch.float64)
tensor([ 0.], dtype=torch.float64)
>>> torch.set_flush_denormal(False)
True
>>> torch.tensor([1e-323], dtype=torch.float64)
tensor(9.88131e-324 *
       [ 1.0000], dtype=torch.float64)


class torch.set_grad_enabled(mode) [source]

Context-manager that sets gradient calculation to on or off. set_grad_enabled will enable or disable grads based on its argument mode. It can be used as a context-manager or as a function. This context manager is thread local; it will not affect computation in other threads.  Parameters
mode (bool) – Flag whether to enable grad (True), or disable (False). This can be used to conditionally enable gradients.   Example: >>> x = torch.tensor([1], requires_grad=True)
>>> is_train = False
>>> with torch.set_grad_enabled(is_train):
...   y = x * 2
>>> y.requires_grad
False
>>> torch.set_grad_enabled(True)
>>> y = x * 2
>>> y.requires_grad
True
>>> torch.set_grad_enabled(False)
>>> y = x * 2
>>> y.requires_grad
False


torch.set_num_interop_threads(int)
Sets the number of threads used for interop parallelism (e.g. in JIT interpreter) on CPU.  Warning Can only be called once and before any inter-op parallel work is started (e.g. JIT execution).


torch.set_num_threads(int)
Sets the number of threads used for intraop parallelism on CPU.  Warning To ensure that the correct number of threads is used, set_num_threads must be called before running eager, JIT or autograd code.


torch.set_printoptions(precision=None, threshold=None, edgeitems=None, linewidth=None, profile=None, sci_mode=None) [source]

Set options for printing. Items shamelessly taken from NumPy  Parameters

precision – Number of digits of precision for floating point output (default = 4).
threshold – Total number of array elements which trigger summarization rather than full repr (default = 1000).
edgeitems – Number of array items in summary at beginning and end of each dimension (default = 3).
linewidth – The number of characters per line for the purpose of inserting line breaks (default = 80). Thresholded matrices will ignore this parameter.
profile – Sane defaults for pretty printing. Can override with any of the above options. (any one of default, short, full)
sci_mode – Enable (True) or disable (False) scientific notation. If None (default) is specified, the value is defined by torch._tensor_str._Formatter. This value is automatically chosen by the framework.


torch.set_rng_state(new_state) [source]

Sets the random number generator state.  Parameters
new_state (torch.ByteTensor) – The desired state


torch.sgn(input, *, out=None) → Tensor
For complex tensors, this function returns a new tensor whose elemants have the same angle as that of the elements of input and absolute value 1. For a non-complex tensor, this function returns the signs of the elements of input (see torch.sign()). outi=0\text{out}_{i} = 0 , if ∣inputi∣==0|{\text{{input}}_i}| == 0  outi=inputi∣inputi∣\text{out}_{i} = \frac{{\text{{input}}_i}}{|{\text{{input}}_i}|} , otherwise  Parameters
input (Tensor) – the input tensor.  Keyword Arguments
out (Tensor, optional) – the output tensor.   Example: >>> x=torch.tensor([3+4j, 7-24j, 0, 1+2j])
>>> x.sgn()
tensor([0.6000+0.8000j, 0.2800-0.9600j, 0.0000+0.0000j, 0.4472+0.8944j])


torch.sigmoid(input, *, out=None) → Tensor
Returns a new tensor with the sigmoid of the elements of input.  outi=11+e−inputi\text{out}_{i} = \frac{1}{1 + e^{-\text{input}_{i}}}
 Parameters
input (Tensor) – the input tensor.  Keyword Arguments
out (Tensor, optional) – the output tensor.   Example: >>> a = torch.randn(4)
>>> a
tensor([ 0.9213,  1.0887, -0.8858, -1.7683])
>>> torch.sigmoid(a)
tensor([ 0.7153,  0.7481,  0.2920,  0.1458])


torch.sign(input, *, out=None) → Tensor
Returns a new tensor with the signs of the elements of input.  outi=sgn(inputi)\text{out}_{i} = \operatorname{sgn}(\text{input}_{i})
 Parameters
input (Tensor) – the input tensor.  Keyword Arguments
out (Tensor, optional) – the output tensor.   Example: >>> a = torch.tensor([0.7, -1.2, 0., 2.3])
>>> a
tensor([ 0.7000, -1.2000,  0.0000,  2.3000])
>>> torch.sign(a)
tensor([ 1., -1.,  0.,  1.])


torch.signbit(input, *, out=None) → Tensor
Tests if each element of input has its sign bit set (is less than zero) or not.  Parameters
input (Tensor) – the input tensor.  Keyword Arguments
out (Tensor, optional) – the output tensor.   Example: >>> a = torch.tensor([0.7, -1.2, 0., 2.3])
>>> torch.signbit(a)
tensor([ False, True,  False,  False])


torch.sin(input, *, out=None) → Tensor
Returns a new tensor with the sine of the elements of input.  outi=sin(inputi)\text{out}_{i} = \sin(\text{input}_{i})
 Parameters
input (Tensor) – the input tensor.  Keyword Arguments
out (Tensor, optional) – the output tensor.   Example: >>> a = torch.randn(4)
>>> a
tensor([-0.5461,  0.1347, -2.7266, -0.2746])
>>> torch.sin(a)
tensor([-0.5194,  0.1343, -0.4032, -0.2711])


torch.sinc(input, *, out=None) → Tensor
Computes the normalized sinc of input.  outi={1,if inputi=0sin(πinputi)/(πinputi),otherwise\text{out}_{i} = \begin{cases} 1, & \text{if}\ \text{input}_{i}=0 \\ \sin(\pi \text{input}_{i}) / (\pi \text{input}_{i}), & \text{otherwise} \end{cases}
 Parameters
input (Tensor) – the input tensor.  Keyword Arguments
out (Tensor, optional) – the output tensor.   Example: >>> a = torch.randn(4)
>>> a
tensor([ 0.2252, -0.2948,  1.0267, -1.1566])
>>> torch.sinc(a)
tensor([ 0.9186,  0.8631, -0.0259, -0.1300])


torch.sinh(input, *, out=None) → Tensor
Returns a new tensor with the hyperbolic sine of the elements of input.  outi=sinh(inputi)\text{out}_{i} = \sinh(\text{input}_{i})
 Parameters
input (Tensor) – the input tensor.  Keyword Arguments
out (Tensor, optional) – the output tensor.   Example: >>> a = torch.randn(4)
>>> a
tensor([ 0.5380, -0.8632, -0.1265,  0.9399])
>>> torch.sinh(a)
tensor([ 0.5644, -0.9744, -0.1268,  1.0845])
  Note When input is on the CPU, the implementation of torch.sinh may use the Sleef library, which rounds very large results to infinity or negative infinity. See here for details.


torch.slogdet(input) -> (Tensor, Tensor)
Calculates the sign and log absolute value of the determinant(s) of a square matrix or batches of square matrices.  Note torch.slogdet() is deprecated. Please use torch.linalg.slogdet() instead.   Note If input has zero determinant, this returns (0, -inf).   Note Backward through slogdet() internally uses SVD results when input is not invertible. In this case, double backward through slogdet() will be unstable in when input doesn’t have distinct singular values. See svd() for details.   Parameters
input (Tensor) – the input tensor of size (*, n, n) where * is zero or more batch dimensions.  Returns
A namedtuple (sign, logabsdet) containing the sign of the determinant, and the log value of the absolute determinant.   Example: >>> A = torch.randn(3, 3)
>>> A
tensor([[ 0.0032, -0.2239, -1.1219],
        [-0.6690,  0.1161,  0.4053],
        [-1.6218, -0.9273, -0.0082]])
>>> torch.det(A)
tensor(-0.7576)
>>> torch.logdet(A)
tensor(nan)
>>> torch.slogdet(A)
torch.return_types.slogdet(sign=tensor(-1.), logabsdet=tensor(-0.2776))


torch.smm(input, mat) → Tensor
Performs a matrix multiplication of the sparse matrix input with the dense matrix mat.  Parameters

input (Tensor) – a sparse matrix to be matrix multiplied
mat (Tensor) – a dense matrix to be matrix multiplied