torch.tril_indices
torch.tril_indices(row, col, offset=0, *, dtype=torch.long, device='cpu', layout=torch.strided) -> Tensor
Returns the indices of the lower triangular part of a "row"-by-
   "col" matrix in a 2-by-N Tensor, where the first row contains row
   coordinates of all indices and the second row contains column
   coordinates. Indices are ordered based on rows and then columns.
The lower triangular part of the matrix is defined as the elements
   on and below the diagonal.
The argument "offset" controls which diagonal to consider. If
   "offset" = 0, all elements on and below the main diagonal are
   retained. A positive value includes just as many diagonals above
   the main diagonal, and similarly a negative value excludes just as
   many diagonals below the main diagonal. The main diagonal are the
   set of indices \lbrace (i, i) \rbrace for i \in [0, \min{d_{1},
   d_{2}} - 1] where d_{1}, d_{2} are the dimensions of the matrix.
Note:


Note:
 When running on CUDA, "row * col" must be less than 2^{59} to
 prevent overflow during calculation.

Parameters:
      * row ("int") -- number of rows in the 2-D matrix.
  * **col** ("int") -- number of columns in the 2-D matrix.

  * **offset** ("int") -- diagonal offset from the main diagonal.
    Default: if not provided, 0.

Keyword Arguments:
      * dtype ("torch.dtype", optional) -- the desired data type
        of returned tensor. Default: if "None", "torch.long".
  * **device** ("torch.device", optional) -- the desired device of
    returned tensor. Default: if "None", uses the current device
    for the default tensor type (see
    "torch.set_default_tensor_type()"). "device" will be the CPU
    for CPU tensor types and the current CUDA device for CUDA
    tensor types.

  * **layout** ("torch.layout", optional) -- currently only
    support "torch.strided".

Example:
  >>> a = torch.tril_indices(3, 3)


a = torch.tril_indices(3, 3)
      >>> a
      tensor([[0, 1, 1, 2, 2, 2],
              [0, 0, 1, 0, 1, 2]])

  >>> a = torch.tril_indices(4, 3, -1)
  >>> a
  tensor([[1, 2, 2, 3, 3, 3],
          [0, 0, 1, 0, 1, 2]])

  >>> a = torch.tril_indices(4, 3, 1)
  >>> a
  tensor([[0, 0, 1, 1, 1, 2, 2, 2, 3, 3, 3],
          [0, 1, 0, 1, 2, 0, 1, 2, 0, 1, 2]])


torch.Tensor.copy_
Tensor.copy_(src, non_blocking=False) -> Tensor
Copies the elements from "src" into "self" tensor and returns
   "self".
The "src" tensor must be broadcastable with the "self" tensor. It
   may be of a different data type or reside on a different device.
Parameters:
      * src (Tensor) -- the source tensor to copy from
  * **non_blocking** (*bool*) -- if "True" and this copy is
    between CPU and GPU, the copy may occur asynchronously with
    respect to the host. For other cases, this argument has no
    effect.


torch.Tensor.chalf
Tensor.chalf(memory_format=torch.preserve_format) -> Tensor
"self.chalf()" is equivalent to "self.to(torch.complex32)". See
   "to()".
Parameters:
      memory_format ("torch.memory_format", optional) -- the
      desired memory format of returned Tensor. Default:
      "torch.preserve_format".


torch.nn.functional.fractional_max_pool3d
torch.nn.functional.fractional_max_pool3d(args, *kwargs)
Applies 3D fractional max pooling over an input signal composed of
   several input planes.
Fractional MaxPooling is described in detail in the paper
   Fractional MaxPooling by Ben Graham
The max-pooling operation is applied in kT \times kH \times kW
   regions by a stochastic step size determined by the target output
   size. The number of output features is equal to the number of input
   planes.
Parameters:
      * kernel_size -- the size of the window to take a max over.
        Can be a single number k (for a square kernel of k \times k
        \times k) or a tuple (kT, kH, kW)
  * **output_size** -- the target output size of the form oT
    \times oH \times oW. Can be a tuple *(oT, oH, oW)* or a single
    number oH for a cubic output oH \times oH \times oH

  * **output_ratio** -- If one wants to have an output size as a


ratio of the input size, this option can be given. This has to
        be a number or tuple in the range (0, 1)
  * **return_indices** -- if "True", will return the indices along
    with the outputs. Useful to pass to "max_unpool3d()".

Shape:
      * Input: (N, C, T_{in}, H_{in}, W_{in}) or (C, T_{in}, H_{in},
        W_{in}).
  * Output: (N, C, T_{out}, H_{out}, W_{out}) or (C, T_{out},
    H_{out}, W_{out}), where (T_{out}, H_{out},
    W_{out})=\text{output\_size} or (T_{out}, H_{out},
    W_{out})=\text{output\_ratio} \times (T_{in}, H_{in}, W_{in})

Examples::
      >>> input = torch.randn(20, 16, 50, 32, 16)
      >>> # pool of cubic window of size=3, and target output size 13x12x11
      >>> F.fractional_max_pool3d(input, 3, output_size=(13, 12, 11))
      >>> # pool of cubic window and target output size being half of input size
      >>> F.fractional_max_pool3d(input, 3, output_ratio=(0.5, 0.5, 0.5))


ConvBnReLU2d
class torch.ao.nn.intrinsic.qat.ConvBnReLU2d(in_channels, out_channels, kernel_size, stride=1, padding=0, dilation=1, groups=1, bias=None, padding_mode='zeros', eps=1e-05, momentum=0.1, freeze_bn=False, qconfig=None)
A ConvBnReLU2d module is a module fused from Conv2d, BatchNorm2d
   and ReLU, attached with FakeQuantize modules for weight, used in
   quantization aware training.
We combined the interface of "torch.nn.Conv2d" and
   "torch.nn.BatchNorm2d" and "torch.nn.ReLU".
Similar to torch.nn.Conv2d, with FakeQuantize modules initialized
   to default.
Variables:
      weight_fake_quant -- fake quant module for weight


ParametrizationList
class torch.nn.utils.parametrize.ParametrizationList(modules, original, unsafe=False)
A sequential container that holds and manages the "original" or
   "original0", "original1", ... parameters or buffers of a
   parametrized "torch.nn.Module".
It is the type of "module.parametrizations[tensor_name]" when
   "module[tensor_name]" has been parametrized with
   "register_parametrization()".
If the first registered parametrization has a "right_inverse" that
   returns one tensor or does not have a "right_inverse" (in which
   case we assume that "right_inverse" is the identity), it will hold
   the tensor under the name "original". If it has a "right_inverse"
   that returns more than one tensor, these will be registered as
   "original0", "original1", ...
Warning:
 This class is used internally by "register_parametrization()". It
 is documented here for completeness. It shall not be instantiated
 by the user.

Parameters:


by the user.
Parameters:
      * modules (sequence) -- sequence of modules representing
        the parametrizations
  * **original** (*Parameter** or **Tensor*) -- parameter or
    buffer that is parametrized

  * **unsafe** (*bool*) -- a boolean flag that denotes whether the
    parametrization may change the dtype and shape of the tensor.
    Default: *False* Warning: the parametrization is not checked
    for consistency upon registration. Enable this flag at your
    own risk.

right_inverse(value)
  Calls the methods "right_inverse" (see
  "register_parametrization()") of the parametrizations in the
  inverse order they were registered in. Then, it stores the
  result in "self.original" if "right_inverse" outputs one tensor
  or in "self.original0", "self.original1", ... if it outputs
  several.

  Parameters:
     **value** (*Tensor*) -- Value to which initialize the module


torch.nn.functional.cosine_embedding_loss
torch.nn.functional.cosine_embedding_loss(input1, input2, target, margin=0, size_average=None, reduce=None, reduction='mean') -> Tensor
See "CosineEmbeddingLoss" for details.
Return type:
      Tensor


torch._foreach_frac
torch._foreach_frac(self: List[Tensor]) -> List[Tensor]
Apply "torch.frac()" to each Tensor of the input list.


torch.stft
torch.stft(input, n_fft, hop_length=None, win_length=None, window=None, center=True, pad_mode='reflect', normalized=False, onesided=None, return_complex=None)
Short-time Fourier transform (STFT).
Warning:
 From version 1.8.0, "return_complex" must always be given
 explicitly for real inputs and *return_complex=False* has been
 deprecated. Strongly prefer *return_complex=True* as in a future
 pytorch release, this function will only return complex
 tensors.Note that "torch.view_as_real()" can be used to recover a
 real tensor with an extra last dimension for real and imaginary
 components.

The STFT computes the Fourier transform of short overlapping
   windows of the input. This giving frequency components of the
   signal as they change over time. The interface of this function is
   modeled after (but not a drop-in replacement for) librosa stft
   function.
Ignoring the optional batch dimension, this method computes the


following expression:
  X[\omega, m] = \sum_{k = 0}^{\text{win\_length-1}}%
  \text{window}[k]\ \text{input}[m \times \text{hop\_length} + k]\
  %                     \exp\left(- j \frac{2 \pi \cdot \omega
  k}{\text{win\_length}}\right),

where m is the index of the sliding window, and \omega is the
   frequency 0 \leq \omega < \text{n_fft} for "onesided=False", or 0
   \leq \omega < \lfloor \text{n_fft} / 2 \rfloor + 1 for
   "onesided=True".

"input" must be either a 1-D time sequence or a 2-D batch of time
     sequences.

If "hop_length" is "None" (default), it is treated as equal to
     "floor(n_fft / 4)".

If "win_length" is "None" (default), it is treated as equal to
     "n_fft".

"window" can be a 1-D tensor of size "win_length", e.g., from
     "torch.hann_window()". If "window" is "None" (default), it is
     treated as if having 1 everywhere in the window. If
     \text{win_length} < \text{n_fft}, "window" will be padded on


both sides to length "n_fft" before being applied.

If "center" is "True" (default), "input" will be padded on both
     sides so that the t-th frame is centered at time t \times
     \text{hop_length}. Otherwise, the t-th frame begins at time  t
     \times \text{hop_length}.

"pad_mode" determines the padding method used on "input" when
     "center" is "True". See "torch.nn.functional.pad()" for all
     available options. Default is ""reflect"".

If "onesided" is "True" (default for real input), only values for
     \omega in \left[0, 1, 2, \dots, \left\lfloor
     \frac{\text{n_fft}}{2} \right\rfloor + 1\right] are returned
     because the real-to-complex Fourier transform satisfies the
     conjugate symmetry, i.e., X[m, \omega] = X[m, \text{n_fft} -
     \omega]^*. Note if the input or window tensors are complex, then
     "onesided" output is not possible.

If "normalized" is "True" (default is "False"), the function


returns the normalized STFT results, i.e., multiplied by
     (\text{frame_length})^{-0.5}.

If "return_complex" is "True" (default if input is complex), the
     return is a "input.dim() + 1" dimensional complex tensor. If
     "False", the output is a "input.dim() + 2" dimensional real
     tensor where the last dimension represents the real and imaginary
     components.

Returns either a complex tensor of size ( \times N \times T) if
   "return_complex" is true, or a real tensor of size ( \times N
   \times T \times 2). Where * is the optional batch size of "input",
   N is the number of frequencies where STFT is applied and T is the
   total number of frames used.
Warning:
 This function changed signature at version 0.4.1. Calling with
 the previous signature may cause error or return incorrect
 result.

Parameters:
      * input (Tensor) -- the input tensor
  * **n_fft** (*int*) -- size of Fourier transform


hop_length (int, optional) -- the distance between
        neighboring sliding window frames. Default: "None" (treated as
        equal to "floor(n_fft / 4)")

win_length (int, optional) -- the size of window
    frame and STFT filter. Default: "None"  (treated as equal to
    "n_fft")

window (Tensor, optional) -- the optional window
    function. Default: "None" (treated as window of all 1 s)

center (bool, optional) -- whether to pad "input" on
    both sides so that the t-th frame is centered at time t \times
    \text{hop_length}. Default: "True"

pad_mode (str, optional) -- controls the padding
    method used when "center" is "True". Default: ""reflect""

normalized (bool, optional) -- controls whether to
    return the normalized STFT results Default: "False"

onesided (bool, optional) -- controls whether to


return half of results to avoid redundancy for real inputs.
        Default: "True" for real "input" and "window", "False"
        otherwise.
  * **return_complex** (*bool**, **optional*) --

    whether to return a complex tensor, or a real tensor with an
    extra last dimension for the real and imaginary components.

    Changed in version 2.0: "return_complex" is now a required
    argument for real inputs, as the default is being transitioned
    to "True".

    Deprecated since version 2.0: "return_complex=False" is
    deprecated, instead use "return_complex=True" Note that
    calling "torch.view_as_real()" on the output will recover the
    deprecated output format.

Returns:
      A tensor containing the STFT result with shape described above
Return type:
      Tensor


upsample_nearest
class torch.ao.nn.quantized.functional.upsample_nearest(input, size=None, scale_factor=None)
Upsamples the input, using nearest neighbours' pixel values.
Warning:
 This function is deprecated in favor of
 "torch.nn.quantized.functional.interpolate()". This is equivalent
 with "nn.quantized.functional.interpolate(..., mode='nearest')".

Note:
 The input quantization parameters propagate to the output.

Note:
 Only 2D inputs are supported

Parameters:
      * input (Tensor) -- quantized input
  * **size** (*int** or **Tuple**[**int**, **int**] or
    **Tuple**[**int**, **int**, **int**]*) -- output spatial size.

  * **scale_factor** (*int*) -- multiplier for spatial size. Has
    to be an integer.


torch.addmm
torch.addmm(input, mat1, mat2, *, beta=1, alpha=1, out=None) -> Tensor
Performs a matrix multiplication of the matrices "mat1" and "mat2".
   The matrix "input" is added to the final result.
If "mat1" is a (n \times m) tensor, "mat2" is a (m \times p)
   tensor, then "input" must be broadcastable with a (n \times p)
   tensor and "out" will be a (n \times p) tensor.
"alpha" and "beta" are scaling factors on matrix-vector product
   between "mat1" and "mat2" and the added matrix "input"
   respectively.
  \text{out} = \beta\ \text{input} + \alpha\ (\text{mat1}_i
  \mathbin{@} \text{mat2}_i)

If "beta" is 0, then "input" will be ignored, and nan and inf
   in it will not be propagated.
For inputs of type FloatTensor or DoubleTensor, arguments
   "beta" and "alpha" must be real numbers, otherwise they should be
   integers.
This operation has support for arguments with sparse layouts. If


"input" is sparse the result will have the same layout and if "out"
   is provided it must have the same layout as "input".
Warning:
 Sparse support is a beta feature and some layout(s)/dtype/device
 combinations may not be supported, or may not have autograd
 support. If you notice missing functionality please open a
 feature request.

This operator supports TensorFloat32.
On certain ROCm devices, when using float16 inputs this module will
   use different precision for backward.
Parameters:
      * input (Tensor) -- matrix to be added
  * **mat1** (*Tensor*) -- the first matrix to be matrix
    multiplied

  * **mat2** (*Tensor*) -- the second matrix to be matrix
    multiplied

Keyword Arguments:
      * beta (Number, optional) -- multiplier for "input"
        (\beta)
  * **alpha** (*Number**, **optional*) -- multiplier for mat1 @
    mat2 (\alpha)

  * **out** (*Tensor**, **optional*) -- the output tensor.

Example:


Example:
  >>> M = torch.randn(2, 3)
  >>> mat1 = torch.randn(2, 3)
  >>> mat2 = torch.randn(3, 3)
  >>> torch.addmm(M, mat1, mat2)
  tensor([[-4.8716,  1.4671, -1.3746],
          [ 0.7573, -3.9555, -2.8681]])


torch.Tensor.subtract_
Tensor.subtract_(other, *, alpha=1) -> Tensor
In-place version of "subtract()".


torch.Tensor.arcsin
Tensor.arcsin() -> Tensor
See "torch.arcsin()"


torch.quantized_max_pool2d
torch.quantized_max_pool2d(input, kernel_size, stride=[], padding=0, dilation=1, ceil_mode=False) -> Tensor
Applies a 2D max pooling over an input quantized tensor composed of
   several input planes.
Parameters:
      * input (Tensor) -- quantized tensor
  * **kernel_size** ("list of int") -- the size of the sliding
    window

  * **stride** ("list of int", optional) -- the stride of the
    sliding window

  * **padding** ("list of int", optional) -- padding to be added
    on both sides, must be >= 0 and <= kernel_size / 2

  * **dilation** ("list of int", optional) -- The stride between
    elements within a sliding window, must be > 0. Default 1

  * **ceil_mode** (*bool**, **optional*) -- If True, will use ceil
    instead of floor to compute the output shape. Defaults to
    False.

Returns:
      A quantized tensor with max_pool2d applied.
Return type:
      Tensor
Example:


Return type:
      Tensor
Example:
  >>> qx = torch.quantize_per_tensor(torch.rand(2, 2, 2, 2), 1.5, 3, torch.quint8)
  >>> torch.quantized_max_pool2d(qx, [2,2])
  tensor([[[[1.5000]],

          [[1.5000]]],

          [[[0.0000]],

          [[0.0000]]]], size=(2, 2, 1, 1), dtype=torch.quint8,
      quantization_scheme=torch.per_tensor_affine, scale=1.5, zero_point=3)


ReLU
class torch.nn.ReLU(inplace=False)
Applies the rectified linear unit function element-wise:
\text{ReLU}(x) = (x)^+ = \max(0, x)
Parameters:
      inplace (bool) -- can optionally do the operation in-
      place. Default: "False"
Shape:
      * Input: (*), where * means any number of dimensions.
  * Output: (*), same shape as the input.

[image]
Examples:
    >>> m = nn.ReLU()
    >>> input = torch.randn(2)
    >>> output = m(input)

  An implementation of CReLU - https://arxiv.org/abs/1603.05201

    >>> m = nn.ReLU()
    >>> input = torch.randn(2).unsqueeze(0)
    >>> output = torch.cat((m(input), m(-input)))


hardtanh
class torch.ao.nn.quantized.functional.hardtanh(input, min_val=- 1.0, max_val=1.0, inplace=False)
This is the quantized version of "hardtanh()".
Return type:
      Tensor


torch.Tensor.put_
Tensor.put_(index, source, accumulate=False) -> Tensor
Copies the elements from "source" into the positions specified by
   "index". For the purpose of indexing, the "self" tensor is treated
   as if it were a 1-D tensor.
"index" and "source" need to have the same number of elements, but
   not necessarily the same shape.
If "accumulate" is "True", the elements in "source" are added to
   "self". If accumulate is "False", the behavior is undefined if
   "index" contain duplicate elements.
Parameters:
      * index (LongTensor) -- the indices into self
  * **source** (*Tensor*) -- the tensor containing values to copy
    from

  * **accumulate** (*bool*) -- whether to accumulate into self

Example:
  >>> src = torch.tensor([[4, 3, 5],
  ...                     [6, 7, 8]])
  >>> src.put_(torch.tensor([1, 3]), torch.tensor([9, 10]))
  tensor([[  4,   9,   5],
          [ 10,   7,   8]])


torch._foreach_trunc
torch._foreach_trunc(self: List[Tensor]) -> List[Tensor]
Apply "torch.trunc()" to each Tensor of the input list.


torch.Tensor.acosh
Tensor.acosh() -> Tensor
See "torch.acosh()"


torch.Tensor.backward
Tensor.backward(gradient=None, retain_graph=None, create_graph=False, inputs=None)
Computes the gradient of current tensor w.r.t. graph leaves.
The graph is differentiated using the chain rule. If the tensor is
   non-scalar (i.e. its data has more than one element) and requires
   gradient, the function additionally requires specifying "gradient".
   It should be a tensor of matching type and location, that contains
   the gradient of the differentiated function w.r.t. "self".
This function accumulates gradients in the leaves - you might need
   to zero ".grad" attributes or set them to "None" before calling it.
   See Default gradient layouts for details on the memory layout of
   accumulated gradients.
Note:
 If you run any forward ops, create "gradient", and/or call
 "backward" in a user-specified CUDA stream context, see Stream
 semantics of backward passes.

Note:


semantics of backward passes.
Note:
 When "inputs" are provided and a given input is not a leaf, the
 current implementation will call its grad_fn (though it is not
 strictly needed to get this gradients). It is an implementation
 detail on which the user should not rely. See https://github.com
 /pytorch/pytorch/pull/60521#issuecomment-867061780 for more
 details.

Parameters:
      * gradient (Tensor or None) -- Gradient w.r.t. the
        tensor. If it is a tensor, it will be automatically converted
        to a Tensor that does not require grad unless "create_graph"
        is True. None values can be specified for scalar Tensors or
        ones that don't require grad. If a None value would be
        acceptable then this argument is optional.
  * **retain_graph** (*bool**, **optional*) -- If "False", the
    graph used to compute the grads will be freed. Note that in
    nearly all cases setting this option to True is not needed and


often can be worked around in a much more efficient way.
        Defaults to the value of "create_graph".
  * **create_graph** (*bool**, **optional*) -- If "True", graph of
    the derivative will be constructed, allowing to compute higher
    order derivative products. Defaults to "False".

  * **inputs** (*sequence of Tensor*) -- Inputs w.r.t. which the
    gradient will be accumulated into ".grad". All other Tensors
    will be ignored. If not provided, the gradient is accumulated
    into all the leaf Tensors that were used to compute the
    attr::tensors.


torch.pinverse
torch.pinverse(input, rcond=1e-15) -> Tensor
Alias for "torch.linalg.pinv()"


torch.Tensor.reshape_as
Tensor.reshape_as(other) -> Tensor
Returns this tensor as the same shape as "other".
   "self.reshape_as(other)" is equivalent to
   "self.reshape(other.sizes())". This method returns a view if
   "other.sizes()" is compatible with the current shape. See
   "torch.Tensor.view()" on when it is possible to return a view.
Please see "reshape()" for more information about "reshape".
Parameters:
      other ("torch.Tensor") -- The result tensor has the same
      shape as "other".


torch.nn.modules.module.register_module_forward_pre_hook
torch.nn.modules.module.register_module_forward_pre_hook(hook)
Registers a forward pre-hook common to all modules.
Warning:
 This adds global state to the *nn.module* module and it is only
 intended for debugging/profiling purposes.

The hook will be called every time before "forward()" is invoked.
   It should have the following signature:
  hook(module, input) -> None or modified input

The input contains only the positional arguments given to the
   module. Keyword arguments won't be passed to the hooks and only to
   the "forward". The hook can modify the input. User can either
   return a tuple or a single modified value in the hook. We will wrap
   the value into a tuple if a single value is returned(unless that
   value is already a tuple).
This hook has precedence over the specific module hooks registered
   with "register_forward_pre_hook".
Returns:


with "register_forward_pre_hook".
Returns:
      a handle that can be used to remove the added hook by calling
      "handle.remove()"
Return type:
      "torch.utils.hooks.RemovableHandle"


torch.nn.functional.prelu
torch.nn.functional.prelu(input, weight) -> Tensor
Applies element-wise the function \text{PReLU}(x) = \max(0,x) +
   \text{weight} * \min(0,x) where weight is a learnable parameter.
Note:
 *weight* is expected to be a scalar or 1-D tensor. If *weight* is
 1-D, its size must match the number of input channels, determined
 by *input.size(1)* when *input.dim() >= 2*, otherwise 1. In the
 1-D case, note that when *input* has dim > 2, *weight* can be
 expanded to the shape of *input* in a way that is not possible
 using normal broadcasting semantics.

See "PReLU" for more details.


default_per_channel_weight_fake_quant
torch.quantization.fake_quantize.default_per_channel_weight_fake_quant
alias of functools.partial(,
   observer=, quant_min=-128, quant_max=127,
   dtype=torch.qint8, qscheme=torch.per_channel_symmetric,
   reduce_range=False, ch_axis=0){}


torch.fft.irfft
torch.fft.irfft(input, n=None, dim=- 1, norm=None, *, out=None) -> Tensor
Computes the inverse of "rfft()".
"input" is interpreted as a one-sided Hermitian signal in the
   Fourier domain, as produced by "rfft()". By the Hermitian property,
   the output will be real-valued.
Note:
 Some input frequencies must be real-valued to satisfy the
 Hermitian property. In these cases the imaginary component will
 be ignored. For example, any imaginary component in the zero-
 frequency term cannot be represented in a real output and so will
 always be ignored.

Note:
 The correct interpretation of the Hermitian input depends on the
 length of the original data, as given by "n". This is because
 each input shape could correspond to either an odd or even length
 signal. By default, the signal is assumed to be even length and
 odd signals will not round-trip properly. So, it is recommended
 to always pass the signal length "n".


to always pass the signal length "n".
Note:
 Supports torch.half and torch.chalf on CUDA with GPU Architecture
 SM53 or greater. However it only supports powers of 2 signal
 length in every transformed dimension. With default arguments,
 size of the transformed dimension should be (2^n + 1) as argument
 *n* defaults to even output size = 2 * (transformed_dim_size - 1)

Parameters:
      * input (Tensor) -- the input tensor representing a half-
        Hermitian signal
  * **n** (*int**, **optional*) -- Output signal length. This
    determines the length of the output signal. If given, the
    input will either be zero-padded or trimmed to this length
    before computing the real IFFT. Defaults to even output:
    "n=2*(input.size(dim) - 1)".

  * **dim** (*int**, **optional*) -- The dimension along which to
    take the one dimensional real IFFT.

  * **norm** (*str**, **optional*) --


norm (str, optional) --Normalization mode. For the backward transform ("irfft()"),
these correspond to:

* ""forward"" - no normalization

* ""backward"" - normalize by "1/n"

* ""ortho"" - normalize by "1/sqrt(n)" (making the real IFFT
  orthonormal)

Calling the forward transform ("rfft()") with the same
normalization mode will apply an overall normalization of
"1/n" between the two transforms. This is required to make
"irfft()" the exact inverse.

Default is ""backward"" (normalize by "1/n").

Keyword Arguments:
      out (Tensor, optional) -- the output tensor.
-[ Example ]-

t = torch.linspace(0, 1, 5)
t
   tensor([0.0000, 0.2500, 0.5000, 0.7500, 1.0000])
T = torch.fft.rfft(t)
T
   tensor([ 2.5000+0.0000j, -0.6250+0.8602j, -0.6250+0.2031j])

Without specifying the output length to "irfft()", the output will


not round-trip properly because the input is odd-length:

torch.fft.irfft(T)
   tensor([0.1562, 0.3511, 0.7812, 1.2114])

So, it is recommended to always pass the signal length "n":

roundtrip = torch.fft.irfft(T, t.numel())
torch.testing.assert_close(roundtrip, t, check_stride=False)


torch.hamming_window
torch.hamming_window(window_length, periodic=True, alpha=0.54, beta=0.46, *, dtype=None, layout=torch.strided, device=None, requires_grad=False) -> Tensor
Hamming window function.
  w[n] = \alpha - \beta\ \cos \left( \frac{2 \pi n}{N - 1}
  \right),

where N is the full window size.
The input "window_length" is a positive integer controlling the
   returned window size. "periodic" flag determines whether the
   returned window trims off the last duplicate value from the
   symmetric window and is ready to be used as a periodic window with
   functions like "torch.stft()". Therefore, if "periodic" is true,
   the N in above formula is in fact \text{window_length} + 1. Also,
   we always have "torch.hamming_window(L, periodic=True)" equal to
   "torch.hamming_window(L + 1, periodic=False)[:-1])".
Note:
 If "window_length" =1, the returned window contains a single
 value 1.

Note:


value 1.
Note:
 This is a generalized version of "torch.hann_window()".

Parameters:
      * window_length (int) -- the size of returned window
  * **periodic** (*bool**, **optional*) -- If True, returns a
    window to be used as periodic function. If False, return a
    symmetric window.

  * **alpha** (*float**, **optional*) -- The coefficient \alpha in
    the equation above

  * **beta** (*float**, **optional*) -- The coefficient \beta in
    the equation above

Keyword Arguments:
      * dtype ("torch.dtype", optional) -- the desired data type
        of returned tensor. Default: if "None", uses a global default
        (see "torch.set_default_tensor_type()"). Only floating point
        types are supported.
  * **layout** ("torch.layout", optional) -- the desired layout of
    returned window tensor. Only "torch.strided" (dense layout) is
    supported.

  * **device** ("torch.device", optional) -- the desired device of


returned tensor. Default: if "None", uses the current device
        for the default tensor type (see
        "torch.set_default_tensor_type()"). "device" will be the CPU
        for CPU tensor types and the current CUDA device for CUDA
        tensor types.
  * **requires_grad** (*bool**, **optional*) -- If autograd should
    record operations on the returned tensor. Default: "False".

Returns:
      A 1-D tensor of size (\text{window_length},) containing the
      window.
Return type:
      Tensor


torch.Tensor.matrix_power
Tensor.matrix_power(n) -> Tensor
Note:
 "matrix_power()" is deprecated, use "torch.linalg.matrix_power()"
 instead.

Alias for "torch.linalg.matrix_power()"


BackendConfig
class torch.ao.quantization.backend_config.BackendConfig(name='')
Config that defines the set of patterns that can be quantized on a
   given backend, and how reference quantized models can be produced
   from these patterns.
A pattern in this context refers to a module, a functional, an
   operator, or a directed acyclic graph of the above. Each pattern
   supported on the target backend can be individually configured
   through "BackendPatternConfig" in terms of:

The supported input/output activation, weight, and bias data
      types

How observers and quant/dequant ops are inserted in order to
      construct the reference pattern, and

(Optionally) Fusion, QAT, and reference module mappings.

The format of the patterns is described in: https://github.com/pyt
   orch/pytorch/blob/master/torch/ao/quantization/backend_config/READ
   ME.md
Example usage:
  import torch
  from torch.ao.quantization.backend_config import (


BackendConfig,
          BackendPatternConfig,
          DTypeConfig,
          ObservationType,
      )
  weighted_int8_dtype_config = DTypeConfig(
      input_dtype=torch.quint8,
      output_dtype=torch.quint8,
      weight_dtype=torch.qint8,
      bias_dtype=torch.float)

  def fuse_conv2d_relu(is_qat, conv, relu):
      return torch.ao.nn.intrinsic.ConvReLU2d(conv, relu)

  # For quantizing Linear
  linear_config = BackendPatternConfig(torch.nn.Linear)             .set_observation_type(ObservationType.OUTPUT_USE_DIFFERENT_OBSERVER_AS_INPUT)             .add_dtype_config(weighted_int8_dtype_config)             .set_root_module(torch.nn.Linear)             .set_qat_module(torch.ao.nn.qat.Linear)             .set_reference_quantized_module(torch.ao.nn.quantized.reference.Linear)

  # For fusing Conv2d + ReLU into ConvReLU2d


For fusing Conv2d + ReLU into ConvReLU2d
  conv_relu_config = BackendPatternConfig((torch.nn.Conv2d, torch.nn.ReLU))             .set_observation_type(ObservationType.OUTPUT_USE_DIFFERENT_OBSERVER_AS_INPUT)             .add_dtype_config(weighted_int8_dtype_config)             .set_fused_module(torch.ao.nn.intrinsic.ConvReLU2d)             .set_fuser_method(fuse_conv2d_relu)

  # For quantizing ConvReLU2d
  fused_conv_relu_config = BackendPatternConfig(torch.ao.nn.intrinsic.ConvReLU2d)             .set_observation_type(ObservationType.OUTPUT_USE_DIFFERENT_OBSERVER_AS_INPUT)             .add_dtype_config(weighted_int8_dtype_config)             .set_root_module(torch.nn.Conv2d)             .set_qat_module(torch.ao.nn.intrinsic.qat.ConvReLU2d)             .set_reference_quantized_module(torch.ao.nn.quantized.reference.Conv2d)


backend_config = BackendConfig("my_backend")             .set_backend_pattern_config(linear_config)             .set_backend_pattern_config(conv_relu_config)             .set_backend_pattern_config(fused_conv_relu_config)
property configs: List[BackendPatternConfig]
  Return a copy of the list of configs set in this
  *BackendConfig*.

classmethod from_dict(backend_config_dict)
  Create a "BackendConfig" from a dictionary with the following
  items:

     "name": the name of the target backend

     "configs": a list of dictionaries that each represents a
     *BackendPatternConfig*

  Return type:
     *BackendConfig*

set_backend_pattern_config(config)
  Set the config for an pattern that can be run on the target
  backend. This overrides any existing config for the given
  pattern.

  Return type:
     *BackendConfig*

set_backend_pattern_configs(configs)
  Set the configs for patterns that can be run on the target


backend. This overrides any existing config for a given pattern
      if it was previously registered already.
  Return type:
     *BackendConfig*

set_name(name)
  Set the name of the target backend.

  Return type:
     *BackendConfig*

to_dict()
  Convert this "BackendConfig" to a dictionary with the items
  described in "from_dict()".

  Return type:
     *Dict*[str, *Any*]


torch.Tensor.as_subclass
Tensor.as_subclass(cls) -> Tensor
Makes a "cls" instance with the same data pointer as "self".
   Changes in the output mirror changes in "self", and the output
   stays attached to the autograd graph. "cls" must be a subclass of
   "Tensor".


torch.Tensor.cumprod_
Tensor.cumprod_(dim, dtype=None) -> Tensor
In-place version of "cumprod()"


torch.Tensor.flipud
Tensor.flipud() -> Tensor
See "torch.flipud()"


torch.zeros
torch.zeros(size, , out=None, dtype=None, layout=torch.strided, device=None, requires_grad=False) -> Tensor
Returns a tensor filled with the scalar value 0, with the shape
   defined by the variable argument "size".
Parameters:
      size (int...) -- a sequence of integers defining the
      shape of the output tensor. Can be a variable number of
      arguments or a collection like a list or tuple.
Keyword Arguments:
      * out (Tensor, optional) -- the output tensor.
  * **dtype** ("torch.dtype", optional) -- the desired data type
    of returned tensor. Default: if "None", uses a global default
    (see "torch.set_default_tensor_type()").

  * **layout** ("torch.layout", optional) -- the desired layout of
    returned Tensor. Default: "torch.strided".

  * **device** ("torch.device", optional) -- the desired device of
    returned tensor. Default: if "None", uses the current device


for the default tensor type (see
        "torch.set_default_tensor_type()"). "device" will be the CPU
        for CPU tensor types and the current CUDA device for CUDA
        tensor types.
  * **requires_grad** (*bool**, **optional*) -- If autograd should
    record operations on the returned tensor. Default: "False".

Example:
  >>> torch.zeros(2, 3)
  tensor([[ 0.,  0.,  0.],
          [ 0.,  0.,  0.]])

  >>> torch.zeros(5)
  tensor([ 0.,  0.,  0.,  0.,  0.])


torch.Tensor.swapaxes
Tensor.swapaxes(axis0, axis1) -> Tensor
See "torch.swapaxes()"


torch.jit.save
torch.jit.save(m, f, _extra_files=None)
Save an offline version of this module for use in a separate
   process. The saved module serializes all of the methods,
   submodules, parameters, and attributes of this module. It can be
   loaded into the C++ API using "torch::jit::load(filename)" or into
   the Python API with "torch.jit.load".
To be able to save a module, it must not make any calls to native
   Python functions.  This means that all submodules must be
   subclasses of "ScriptModule" as well.
Danger:
 All modules, no matter their device, are always loaded onto the
 CPU during loading.  This is different from "torch.load()"'s
 semantics and may change in the future.

Parameters:
      * m -- A "ScriptModule" to save.
  * **f** -- A file-like object (has to implement write and flush)
    or a string containing a file name.

  * **_extra_files** -- Map from filename to contents which will
    be stored as part of *f*.


be stored as part of f.
Note:
 torch.jit.save attempts to preserve the behavior of some
 operators across versions. For example, dividing two integer
 tensors in PyTorch 1.5 performed floor division, and if the
 module containing that code is saved in PyTorch 1.5 and loaded in
 PyTorch 1.6 its division behavior will be preserved. The same
 module saved in PyTorch 1.6 will fail to load in PyTorch 1.5,
 however, since the behavior of division changed in 1.6, and 1.5
 does not know how to replicate the 1.6 behavior.

Example:
  import torch
  import io

  class MyModule(torch.nn.Module):
      def forward(self, x):
          return x + 10

  m = torch.jit.script(MyModule())

  # Save to file
  torch.jit.save(m, 'scriptmodule.pt')
  # This line is equivalent to the previous
  m.save("scriptmodule.pt")

  # Save to io.BytesIO buffer
  buffer = io.BytesIO()
  torch.jit.save(m, buffer)


torch.jit.save(m, buffer)
  # Save with extra files
  extra_files = {'foo.txt': b'bar'}
  torch.jit.save(m, 'scriptmodule.pt', _extra_files=extra_files)


Sequential
class torch.nn.Sequential(*args: Module)
class torch.nn.Sequential(arg: OrderedDict[str, Module])
A sequential container. Modules will be added to it in the order
   they are passed in the constructor. Alternatively, an "OrderedDict"
   of modules can be passed in. The "forward()" method of "Sequential"
   accepts any input and forwards it to the first module it contains.
   It then "chains" outputs to inputs sequentially for each subsequent
   module, finally returning the output of the last module.
The value a "Sequential" provides over manually calling a sequence
   of modules is that it allows treating the whole container as a
   single module, such that performing a transformation on the
   "Sequential" applies to each of the modules it stores (which are
   each a registered submodule of the "Sequential").
What's the difference between a "Sequential" and a
   "torch.nn.ModuleList"? A "ModuleList" is exactly what it sounds


like--a list for storing "Module" s! On the other hand, the layers
   in a "Sequential" are connected in a cascading way.
Example:
  # Using Sequential to create a small model. When `model` is run,
  # input will first be passed to `Conv2d(1,20,5)`. The output of
  # `Conv2d(1,20,5)` will be used as the input to the first
  # `ReLU`; the output of the first `ReLU` will become the input
  # for `Conv2d(20,64,5)`. Finally, the output of
  # `Conv2d(20,64,5)` will be used as input to the second `ReLU`
  model = nn.Sequential(
            nn.Conv2d(1,20,5),
            nn.ReLU(),
            nn.Conv2d(20,64,5),
            nn.ReLU()
          )

  # Using Sequential with OrderedDict. This is functionally the
  # same as the above code
  model = nn.Sequential(OrderedDict([
            ('conv1', nn.Conv2d(1,20,5)),
            ('relu1', nn.ReLU()),
            ('conv2', nn.Conv2d(20,64,5)),
            ('relu2', nn.ReLU())


('relu2', nn.ReLU())
              ]))
append(module)
  Appends a given module to the end.

  Parameters:
     **module** (*nn.Module*) -- module to append

  Return type:
     *Sequential*


torch.ones
torch.ones(size, , out=None, dtype=None, layout=torch.strided, device=None, requires_grad=False) -> Tensor
Returns a tensor filled with the scalar value 1, with the shape
   defined by the variable argument "size".
Parameters:
      size (int...) -- a sequence of integers defining the
      shape of the output tensor. Can be a variable number of
      arguments or a collection like a list or tuple.
Keyword Arguments:
      * out (Tensor, optional) -- the output tensor.
  * **dtype** ("torch.dtype", optional) -- the desired data type
    of returned tensor. Default: if "None", uses a global default
    (see "torch.set_default_tensor_type()").

  * **layout** ("torch.layout", optional) -- the desired layout of
    returned Tensor. Default: "torch.strided".

  * **device** ("torch.device", optional) -- the desired device of
    returned tensor. Default: if "None", uses the current device
    for the default tensor type (see


for the default tensor type (see
        "torch.set_default_tensor_type()"). "device" will be the CPU
        for CPU tensor types and the current CUDA device for CUDA
        tensor types.
  * **requires_grad** (*bool**, **optional*) -- If autograd should
    record operations on the returned tensor. Default: "False".

Example:
  >>> torch.ones(2, 3)
  tensor([[ 1.,  1.,  1.],
          [ 1.,  1.,  1.]])

  >>> torch.ones(5)
  tensor([ 1.,  1.,  1.,  1.,  1.])


torch.arcsin
torch.arcsin(input, *, out=None) -> Tensor
Alias for "torch.asin()".


torch.mean
torch.mean(input, *, dtype=None) -> Tensor
Returns the mean value of all elements in the "input" tensor.
Parameters:
      input (Tensor) -- the input tensor.
Keyword Arguments:
      dtype ("torch.dtype", optional) -- the desired data type of
      returned tensor. If specified, the input tensor is casted to
      "dtype" before the operation is performed. This is useful for
      preventing data type overflows. Default: None.
Example:
  >>> a = torch.randn(1, 3)
  >>> a
  tensor([[ 0.2294, -0.5481,  1.3288]])
  >>> torch.mean(a)
  tensor(0.3367)

torch.mean(input, dim, keepdim=False, *, dtype=None, out=None) -> Tensor
Returns the mean value of each row of the "input" tensor in the
   given dimension "dim". If "dim" is a list of dimensions, reduce
   over all of them.
If "keepdim" is "True", the output tensor is of the same size as
   "input" except in the dimension(s) "dim" where it is of size 1.


Otherwise, "dim" is squeezed (see "torch.squeeze()"), resulting in
   the output tensor having 1 (or "len(dim)") fewer dimension(s).
Parameters:
      * input (Tensor) -- the input tensor.
  * **dim** (*int** or **tuple of ints*) -- the dimension or
    dimensions to reduce.

  * **keepdim** (*bool*) -- whether the output tensor has "dim"
    retained or not.

Keyword Arguments:
      * dtype ("torch.dtype", optional) -- the desired data type
        of returned tensor. If specified, the input tensor is casted
        to "dtype" before the operation is performed. This is useful
        for preventing data type overflows. Default: None.
  * **out** (*Tensor**, **optional*) -- the output tensor.

See also:
 "torch.nanmean()" computes the mean value of *non-NaN* elements.

Example:
  >>> a = torch.randn(4, 4)
  >>> a
  tensor([[-0.3841,  0.6320,  0.4254, -0.7384],
          [-0.9644,  1.0131, -0.6549, -1.4279],


[-0.2951, -1.3350, -0.7694,  0.5600],
              [ 1.0842, -0.9580,  0.3623,  0.2343]])
      >>> torch.mean(a, 1)
      tensor([-0.0163, -0.5085, -0.4599,  0.1807])
      >>> torch.mean(a, 1, True)
      tensor([[-0.0163],
              [-0.5085],
              [-0.4599],
              [ 0.1807]])


torch.fft.fft
torch.fft.fft(input, n=None, dim=- 1, norm=None, *, out=None) -> Tensor
Computes the one dimensional discrete Fourier transform of "input".
Note:
 The Fourier domain representation of any real signal satisfies
 the Hermitian property: *X[i] = conj(X[-i])*. This function
 always returns both the positive and negative frequency terms
 even though, for real inputs, the negative frequencies are
 redundant. "rfft()" returns the more compact one-sided
 representation where only the positive frequencies are returned.

Note:
 Supports torch.half and torch.chalf on CUDA with GPU Architecture
 SM53 or greater. However it only supports powers of 2 signal
 length in every transformed dimension.

Parameters:
      * input (Tensor) -- the input tensor
  * **n** (*int**, **optional*) -- Signal length. If given, the
    input will either be zero-padded or trimmed to this length
    before computing the FFT.


before computing the FFT.
  * **dim** (*int**, **optional*) -- The dimension along which to
    take the one dimensional FFT.

  * **norm** (*str**, **optional*) --

    Normalization mode. For the forward transform ("fft()"), these
    correspond to:

    * ""forward"" - normalize by "1/n"

    * ""backward"" - no normalization

    * ""ortho"" - normalize by "1/sqrt(n)" (making the FFT
      orthonormal)

    Calling the backward transform ("ifft()") with the same
    normalization mode will apply an overall normalization of
    "1/n" between the two transforms. This is required to make
    "ifft()" the exact inverse.

    Default is ""backward"" (no normalization).

Keyword Arguments:
      out (Tensor, optional) -- the output tensor.
-[ Example ]-

t = torch.arange(4)
t
   tensor([0, 1, 2, 3])
torch.fft.fft(t)
   tensor([ 6.+0.j, -2.+2.j, -2.+0.j, -2.-2.j])


tensor([ 6.+0.j, -2.+2.j, -2.+0.j, -2.-2.j])

t = torch.tensor([0.+1.j, 2.+3.j, 4.+5.j, 6.+7.j])
torch.fft.fft(t)
   tensor([12.+16.j, -8.+0.j, -4.-4.j,  0.-8.j])


torch.Tensor.var
Tensor.var(dim=None, *, correction=1, keepdim=False) -> Tensor
See "torch.var()"


torch.erfc
torch.erfc(input, *, out=None) -> Tensor
Alias for "torch.special.erfc()".


torch.nn.functional.one_hot
torch.nn.functional.one_hot(tensor, num_classes=- 1) -> LongTensor
Takes LongTensor with index values of shape "()" and returns a
   tensor of shape "(, num_classes)" that have zeros everywhere
   except where the index of last dimension matches the corresponding
   value of the input tensor, in which case it will be 1.
See also One-hot on Wikipedia .
Parameters:
      * tensor (LongTensor) -- class values of any shape.
  * **num_classes** (*int*) -- Total number of classes. If set to
    -1, the number of classes will be inferred as one greater than
    the largest class value in the input tensor.

Returns:
      LongTensor that has one more dimension with 1 values at the
      index of last dimension indicated by the input, and 0 everywhere
      else.
-[ Examples ]-

F.one_hot(torch.arange(0, 5) % 3)
   tensor([[1, 0, 0],
           [0, 1, 0],
           [0, 0, 1],
           [1, 0, 0],


[0, 0, 1],
           [1, 0, 0],
           [0, 1, 0]])

F.one_hot(torch.arange(0, 5) % 3, num_classes=5)
   tensor([[1, 0, 0, 0, 0],
           [0, 1, 0, 0, 0],
           [0, 0, 1, 0, 0],
           [1, 0, 0, 0, 0],
           [0, 1, 0, 0, 0]])
F.one_hot(torch.arange(0, 6).view(3,2) % 3)
   tensor([[[1, 0, 0],
            [0, 1, 0]],
           [[0, 0, 1],
            [1, 0, 0]],
           [[0, 1, 0],
            [0, 0, 1]]])


torch.Tensor.tile
Tensor.tile(*reps) -> Tensor
See "torch.tile()"


torch.Tensor.log2
Tensor.log2() -> Tensor
See "torch.log2()"


torch.Tensor.lcm_
Tensor.lcm_(other) -> Tensor
In-place version of "lcm()"


torch.cholesky_solve
torch.cholesky_solve(input, input2, upper=False, *, out=None) -> Tensor
Solves a linear system of equations with a positive semidefinite
   matrix to be inverted given its Cholesky factor matrix u.
If "upper" is "False", u is and lower triangular and c is
   returned such that:
  c = (u u^T)^{{-1}} b

If "upper" is "True" or not provided, u is upper triangular and c
   is returned such that:
  c = (u^T u)^{{-1}} b

torch.cholesky_solve(b, u) can take in 2D inputs b, u or inputs
   that are batches of 2D matrices. If the inputs are batches, then
   returns batched outputs c
Supports real-valued and complex-valued inputs. For the complex-
   valued inputs the transpose operator above is the conjugate
   transpose.
Parameters:
      * input (Tensor) -- input matrix b of size (*, m, k),
        where * is zero or more batch dimensions
  * **input2** (*Tensor*) -- input matrix u of size (*, m, m),


where * is zero of more batch dimensions composed of upper or
        lower triangular Cholesky factor
  * **upper** (*bool**, **optional*) -- whether to consider the
    Cholesky factor as a lower or upper triangular matrix.
    Default: "False".

Keyword Arguments:
      out (Tensor, optional) -- the output tensor for c
Example:
  >>> a = torch.randn(3, 3)
  >>> a = torch.mm(a, a.t()) # make symmetric positive definite
  >>> u = torch.linalg.cholesky(a)
  >>> a
  tensor([[ 0.7747, -1.9549,  1.3086],
          [-1.9549,  6.7546, -5.4114],
          [ 1.3086, -5.4114,  4.8733]])
  >>> b = torch.randn(3, 2)
  >>> b
  tensor([[-0.6355,  0.9891],
          [ 0.1974,  1.4706],
          [-0.4115, -0.6225]])
  >>> torch.cholesky_solve(b, u)
  tensor([[ -8.1625,  19.6097],
          [ -5.8398,  14.2387],
          [ -4.3771,  10.4173]])
  >>> torch.mm(a.inverse(), b)
  tensor([[ -8.1626,  19.6097],


tensor([[ -8.1626,  19.6097],
              [ -5.8398,  14.2387],
              [ -4.3771,  10.4173]])


torch.tensor_split
torch.tensor_split(input, indices_or_sections, dim=0) -> List of Tensors
Splits a tensor into multiple sub-tensors, all of which are views
   of "input", along dimension "dim" according to the indices or
   number of sections specified by "indices_or_sections". This
   function is based on NumPy's "numpy.array_split()".
Parameters:
      * input (Tensor) -- the tensor to split
  * **indices_or_sections** (*Tensor**, **int** or **list** or
    **tuple of ints*) --

    If "indices_or_sections" is an integer "n" or a zero
    dimensional long tensor with value "n", "input" is split into
    "n" sections along dimension "dim". If "input" is divisible by
    "n" along dimension "dim", each section will be of equal size,
    "input.size(dim) / n". If "input" is not divisible by "n", the
    sizes of the first "int(input.size(dim) % n)" sections will
    have size "int(input.size(dim) / n) + 1", and the rest will


have size "int(input.size(dim) / n)".
    If "indices_or_sections" is a list or tuple of ints, or a one-
    dimensional long tensor, then "input" is split along dimension
    "dim" at each of the indices in the list, tuple or tensor. For
    instance, "indices_or_sections=[2, 3]" and "dim=0" would
    result in the tensors "input[:2]", "input[2:3]", and
    "input[3:]".

    If "indices_or_sections" is a tensor, it must be a zero-
    dimensional or one-dimensional long tensor on the CPU.

  * **dim** (*int**, **optional*) -- dimension along which to
    split the tensor. Default: "0"

Example:
  >>> x = torch.arange(8)
  >>> torch.tensor_split(x, 3)
  (tensor([0, 1, 2]), tensor([3, 4, 5]), tensor([6, 7]))

  >>> x = torch.arange(7)
  >>> torch.tensor_split(x, 3)
  (tensor([0, 1, 2]), tensor([3, 4]), tensor([5, 6]))
  >>> torch.tensor_split(x, (1, 6))
  (tensor([0]), tensor([1, 2, 3, 4, 5]), tensor([6]))


x = torch.arange(14).reshape(2, 7)
      >>> x
      tensor([[ 0,  1,  2,  3,  4,  5,  6],
              [ 7,  8,  9, 10, 11, 12, 13]])
      >>> torch.tensor_split(x, 3, dim=1)
      (tensor([[0, 1, 2],
              [7, 8, 9]]),
       tensor([[ 3,  4],
              [10, 11]]),
       tensor([[ 5,  6],
              [12, 13]]))
      >>> torch.tensor_split(x, (1, 6), dim=1)
      (tensor([[0],
              [7]]),
       tensor([[ 1,  2,  3,  4,  5],
              [ 8,  9, 10, 11, 12]]),
       tensor([[ 6],
              [13]]))


torch.fft.rfftn
torch.fft.rfftn(input, s=None, dim=None, norm=None, *, out=None) -> Tensor
Computes the N-dimensional discrete Fourier transform of real
   "input".
The FFT of a real signal is Hermitian-symmetric, "X[i_1, ..., i_n]
   = conj(X[-i_1, ..., -i_n])" so the full "fftn()" output contains
   redundant information. "rfftn()" instead omits the negative
   frequencies in the last dimension.
Note:
 Supports torch.half on CUDA with GPU Architecture SM53 or
 greater. However it only supports powers of 2 signal length in
 every transformed dimensions.

Parameters:
      * input (Tensor) -- the input tensor
  * **s** (*Tuple**[**int**]**, **optional*) -- Signal size in the
    transformed dimensions. If given, each dimension "dim[i]" will
    either be zero-padded or trimmed to the length "s[i]" before
    computing the real FFT. If a length "-1" is specified, no
    padding is done in that dimension. Default: "s =


[input.size(d) for d in dim]"
  * **dim** (*Tuple**[**int**]**, **optional*) -- Dimensions to be
    transformed. Default: all dimensions, or the last "len(s)"
    dimensions if "s" is given.

  * **norm** (*str**, **optional*) --

    Normalization mode. For the forward transform ("rfftn()"),
    these correspond to:

    * ""forward"" - normalize by "1/n"

    * ""backward"" - no normalization

    * ""ortho"" - normalize by "1/sqrt(n)" (making the real FFT
      orthonormal)

    Where "n = prod(s)" is the logical FFT size. Calling the
    backward transform ("irfftn()") with the same normalization
    mode will apply an overall normalization of "1/n" between the
    two transforms. This is required to make "irfftn()" the exact
    inverse.

    Default is ""backward"" (no normalization).

Keyword Arguments:
      out (Tensor, optional) -- the output tensor.
-[ Example ]-

t = torch.rand(10, 10)


-[ Example ]-

t = torch.rand(10, 10)
rfftn = torch.fft.rfftn(t)
rfftn.size()
   torch.Size([10, 6])

Compared against the full output from "fftn()", we have all
   elements up to the Nyquist frequency.

fftn = torch.fft.fftn(t)
torch.testing.assert_close(fftn[..., :6], rfftn, check_stride=False)

The discrete Fourier transform is separable, so "rfftn()" here is
   equivalent to a combination of "fft()" and "rfft()":

two_ffts = torch.fft.fft(torch.fft.rfft(t, dim=1), dim=0)
torch.testing.assert_close(rfftn, two_ffts, check_stride=False)


torch.randperm
torch.randperm(n, *, generator=None, out=None, dtype=torch.int64, layout=torch.strided, device=None, requires_grad=False, pin_memory=False) -> Tensor
Returns a random permutation of integers from "0" to "n - 1".
Parameters:
      n (int) -- the upper bound (exclusive)
Keyword Arguments:
      * generator ("torch.Generator", optional) -- a pseudorandom
        number generator for sampling
  * **out** (*Tensor**, **optional*) -- the output tensor.

  * **dtype** ("torch.dtype", optional) -- the desired data type
    of returned tensor. Default: "torch.int64".

  * **layout** ("torch.layout", optional) -- the desired layout of
    returned Tensor. Default: "torch.strided".

  * **device** ("torch.device", optional) -- the desired device of
    returned tensor. Default: if "None", uses the current device
    for the default tensor type (see
    "torch.set_default_tensor_type()"). "device" will be the CPU


for CPU tensor types and the current CUDA device for CUDA
        tensor types.
  * **requires_grad** (*bool**, **optional*) -- If autograd should
    record operations on the returned tensor. Default: "False".

  * **pin_memory** (*bool**, **optional*) -- If set, returned
    tensor would be allocated in the pinned memory. Works only for
    CPU tensors. Default: "False".

Example:
  >>> torch.randperm(4)
  tensor([2, 1, 0, 3])


torch.nn.functional.tanhshrink
torch.nn.functional.tanhshrink(input) -> Tensor
Applies element-wise, \text{Tanhshrink}(x) = x - \text{Tanh}(x)
See "Tanhshrink" for more details.


torch.func.replace_all_batch_norm_modules_
torch.func.replace_all_batch_norm_modules_(root)
In place updates "root" by setting the "running_mean" and
   "running_var" to be None and setting track_running_stats to be
   False for any nn.BatchNorm module in "root"
Return type:
      Module


hardswish
class torch.ao.nn.quantized.functional.hardswish(input, scale, zero_point)
This is the quantized version of "hardswish()".
Parameters:
      * input (Tensor) -- quantized input
  * **scale** (*float*) -- quantization scale of the output tensor

  * **zero_point** (*int*) -- quantization zero point of the
    output tensor

Return type:
      Tensor


Transformer
class torch.nn.Transformer(d_model=512, nhead=8, num_encoder_layers=6, num_decoder_layers=6, dim_feedforward=2048, dropout=0.1, activation=, custom_encoder=None, custom_decoder=None, layer_norm_eps=1e-05, batch_first=False, norm_first=False, device=None, dtype=None)
A transformer model. User is able to modify the attributes as
   needed. The architecture is based on the paper "Attention Is All
   You Need". Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob
   Uszkoreit, Llion Jones, Aidan N Gomez, Lukasz Kaiser, and Illia
   Polosukhin. 2017. Attention is all you need. In Advances in Neural
   Information Processing Systems, pages 6000-6010.
Parameters:
      * d_model (int) -- the number of expected features in the
        encoder/decoder inputs (default=512).
  * **nhead** (*int*) -- the number of heads in the
    multiheadattention models (default=8).

  * **num_encoder_layers** (*int*) -- the number of sub-encoder-


layers in the encoder (default=6).
  * **num_decoder_layers** (*int*) -- the number of sub-decoder-
    layers in the decoder (default=6).

  * **dim_feedforward** (*int*) -- the dimension of the
    feedforward network model (default=2048).

  * **dropout** (*float*) -- the dropout value (default=0.1).

  * **activation** (*Union**[**str**,
    **Callable**[**[**Tensor**]**, **Tensor**]**]*) -- the
    activation function of encoder/decoder intermediate layer, can
    be a string ("relu" or "gelu") or a unary callable. Default:
    relu

  * **custom_encoder** (*Optional**[**Any**]*) -- custom encoder
    (default=None).

  * **custom_decoder** (*Optional**[**Any**]*) -- custom decoder
    (default=None).

  * **layer_norm_eps** (*float*) -- the eps value in layer
    normalization components (default=1e-5).

  * **batch_first** (*bool*) -- If "True", then the input and


output tensors are provided as (batch, seq, feature). Default:
        "False" (seq, batch, feature).
  * **norm_first** (*bool*) -- if "True", encoder and decoder
    layers will perform LayerNorms before other attention and
    feedforward operations, otherwise after. Default: "False"
    (after).

Examples::
      >>> transformer_model = nn.Transformer(nhead=16, num_encoder_layers=12)
      >>> src = torch.rand((10, 32, 512))
      >>> tgt = torch.rand((20, 32, 512))
      >>> out = transformer_model(src, tgt)
Note: A full example to apply nn.Transformer module for the word
   language model is available in
   https://github.com/pytorch/examples/tree/master/word_language_model
forward(src, tgt, src_mask=None, tgt_mask=None, memory_mask=None, src_key_padding_mask=None, tgt_key_padding_mask=None, memory_key_padding_mask=None)
  Take in and process masked source/target sequences.

  Parameters:
     * **src** (*Tensor*) -- the sequence to the encoder


(required).
     * **tgt** (*Tensor*) -- the sequence to the decoder
       (required).

     * **src_mask** (*Optional**[**Tensor**]*) -- the additive
       mask for the src sequence (optional).

     * **tgt_mask** (*Optional**[**Tensor**]*) -- the additive
       mask for the tgt sequence (optional).

     * **memory_mask** (*Optional**[**Tensor**]*) -- the additive
       mask for the encoder output (optional).

     * **src_key_padding_mask** (*Optional**[**Tensor**]*) -- the
       ByteTensor mask for src keys per batch (optional).

     * **tgt_key_padding_mask** (*Optional**[**Tensor**]*) -- the
       ByteTensor mask for tgt keys per batch (optional).

     * **memory_key_padding_mask** (*Optional**[**Tensor**]*) --
       the ByteTensor mask for memory keys per batch (optional).

  Return type:
     *Tensor*

  Shape:
     * src: (S, E) for unbatched input, (S, N, E) if


batch_first=False or (N, S, E) if batch_first=True.
     * tgt: (T, E) for unbatched input, (T, N, E) if
       *batch_first=False* or *(N, T, E)* if *batch_first=True*.

     * src_mask: (S, S) or (N\cdot\text{num\_heads}, S, S).

     * tgt_mask: (T, T) or (N\cdot\text{num\_heads}, T, T).

     * memory_mask: (T, S).

     * src_key_padding_mask: (S) for unbatched input otherwise (N,
       S).

     * tgt_key_padding_mask: (T) for unbatched input otherwise (N,
       T).

     * memory_key_padding_mask: (S) for unbatched input otherwise
       (N, S).

     Note: [src/tgt/memory]_mask ensures that position i is
     allowed to attend the unmasked positions. If a ByteTensor is
     provided, the non-zero positions are not allowed to attend
     while the zero positions will be unchanged. If a BoolTensor
     is provided, positions with "True" are not allowed to attend


while "False" values will be unchanged. If a FloatTensor is
         provided, it will be added to the attention weight.
         [src/tgt/memory]_key_padding_mask provides specified elements
         in the key to be ignored by the attention. If a ByteTensor is
         provided, the non-zero positions will be ignored while the
         zero positions will be unchanged. If a BoolTensor is
         provided, the positions with the value of "True" will be
         ignored while the position with the value of "False" will be
         unchanged.
     * output: (T, E) for unbatched input, (T, N, E) if
       *batch_first=False* or *(N, T, E)* if *batch_first=True*.

     Note: Due to the multi-head attention architecture in the
     transformer model, the output sequence length of a
     transformer is same as the input sequence (i.e. target)
     length of the decoder.

     where S is the source sequence length, T is the target


sequence length, N is the batch size, E is the feature number
  -[ Examples ]-

  >>> output = transformer_model(src, tgt, src_mask=src_mask, tgt_mask=tgt_mask)

static generate_square_subsequent_mask(sz, device='cpu')
  Generate a square mask for the sequence. The masked positions
  are filled with float('-inf'). Unmasked positions are filled
  with float(0.0).

  Return type:
     *Tensor*


default_placeholder_observer
torch.quantization.observer.default_placeholder_observer
alias of "PlaceholderObserver"


torch.sparse_csr_tensor
torch.sparse_csr_tensor(crow_indices, col_indices, values, size=None, *, dtype=None, device=None, requires_grad=False, check_invariants=None) -> Tensor
Constructs a sparse tensor in CSR (Compressed Sparse Row) with
   specified values at the given "crow_indices" and "col_indices".
   Sparse matrix multiplication operations in CSR format are typically
   faster than that for sparse tensors in COO format. Make you have a
   look at the note on the data type of the indices.
Note:
 If the "device" argument is not specified the device of the given
 "values" and indices tensor(s) must match. If, however, the
 argument is specified the input Tensors will be converted to the
 given device and in turn determine the device of the constructed
 sparse tensor.

Parameters:
      * crow_indices (array_like) -- (B+1)-dimensional array of
        size "(*batchsize, nrows + 1)".  The last element of each


batch is the number of non-zeros. This tensor encodes the
        index in values and col_indices depending on where the given
        row starts. Each successive number in the tensor subtracted by
        the number before it denotes the number of elements in a given
        row.
  * **col_indices** (*array_like*) -- Column co-ordinates of each
    element in values. (B+1)-dimensional tensor with the same
    length as values.

  * **values** (*array_list*) -- Initial values for the tensor.
    Can be a list, tuple, NumPy "ndarray", scalar, and other types
    that represents a (1+K)-dimensional tensor where "K" is the
    number of dense dimensions.

  * **size** (list, tuple, "torch.Size", optional) -- Size of the
    sparse tensor: "(*batchsize, nrows, ncols, *densesize)". If
    not provided, the size will be inferred as the minimum size
    big enough to hold all non-zero elements.

Keyword Arguments:


Keyword Arguments:
      * dtype ("torch.dtype", optional) -- the desired data type
        of returned tensor.  Default: if None, infers data type from
        "values".
  * **device** ("torch.device", optional) -- the desired device of
    returned tensor.  Default: if None, uses the current device
    for the default tensor type (see
    "torch.set_default_tensor_type()"). "device" will be the CPU
    for CPU tensor types and the current CUDA device for CUDA
    tensor types.

  * **requires_grad** (*bool**, **optional*) -- If autograd should
    record operations on the returned tensor. Default: "False".

  * **check_invariants** (*bool**, **optional*) -- If sparse
    tensor invariants are checked. Default: as returned by
    "torch.sparse.check_sparse_tensor_invariants.is_enabled()",
    initially False.

Example::
      >>> crow_indices = [0, 2, 4]
      >>> col_indices = [0, 1, 0, 1]
      >>> values = [1, 2, 3, 4]


values = [1, 2, 3, 4]
      >>> torch.sparse_csr_tensor(torch.tensor(crow_indices, dtype=torch.int64),
      ...                         torch.tensor(col_indices, dtype=torch.int64),
      ...                         torch.tensor(values), dtype=torch.double)
      tensor(crow_indices=tensor([0, 2, 4]),
             col_indices=tensor([0, 1, 0, 1]),
             values=tensor([1., 2., 3., 4.]), size=(2, 2), nnz=4,
             dtype=torch.float64, layout=torch.sparse_csr)


torch.column_stack
torch.column_stack(tensors, *, out=None) -> Tensor
Creates a new tensor by horizontally stacking the tensors in
   "tensors".
Equivalent to "torch.hstack(tensors)", except each zero or one
   dimensional tensor "t" in "tensors" is first reshaped into a
   "(t.numel(), 1)" column before being stacked horizontally.
Parameters:
      tensors (sequence of Tensors) -- sequence of tensors to
      concatenate
Keyword Arguments:
      out (Tensor, optional) -- the output tensor.
Example:
  >>> a = torch.tensor([1, 2, 3])
  >>> b = torch.tensor([4, 5, 6])
  >>> torch.column_stack((a, b))
  tensor([[1, 4],
      [2, 5],
      [3, 6]])
  >>> a = torch.arange(5)
  >>> b = torch.arange(10).reshape(5, 2)
  >>> torch.column_stack((a, b, b))
  tensor([[0, 0, 1, 0, 1],
          [1, 2, 3, 2, 3],
          [2, 4, 5, 4, 5],
          [3, 6, 7, 6, 7],
          [4, 8, 9, 8, 9]])


torch.index_reduce
torch.index_reduce(input, dim, index, source, reduce, *, include_self=True, out=None) -> Tensor
See "index_reduce_()" for function description.


torch.Tensor.neg
Tensor.neg() -> Tensor
See "torch.neg()"


torch.Tensor.is_complex
Tensor.is_complex() -> bool
Returns True if the data type of "self" is a complex data type.


torch.nn.utils.parametrizations.orthogonal
torch.nn.utils.parametrizations.orthogonal(module, name='weight', orthogonal_map=None, *, use_trivialization=True)
Applies an orthogonal or unitary parametrization to a matrix or a
   batch of matrices.
Letting \mathbb{K} be \mathbb{R} or \mathbb{C}, the parametrized
   matrix Q \in \mathbb{K}^{m \times n} is orthogonal as
  \begin{align*}     Q^{\text{H}}Q &= \mathrm{I}_n
  \mathrlap{\qquad \text{if }m \geq n}\\     QQ^{\text{H}} &=
  \mathrm{I}_m \mathrlap{\qquad \text{if }m < n} \end{align*}

where Q^{\text{H}} is the conjugate transpose when Q is complex and
   the transpose when Q is real-valued, and \mathrm{I}_n is the
   n-dimensional identity matrix. In plain words, Q will have
   orthonormal columns whenever m \geq n and orthonormal rows
   otherwise.
If the tensor has more than two dimensions, we consider it as a
   batch of matrices of shape (..., m, n).


batch of matrices of shape (..., m, n).
The matrix Q may be parametrized via three different
   "orthogonal_map" in terms of the original tensor:

""matrix_exp""/""cayley"": the "matrix_exp()" Q = \exp(A) and the
     Cayley map Q = (\mathrm{I}_n + A/2)(\mathrm{I}_n - A/2)^{-1} are
     applied to a skew-symmetric A to give an orthogonal matrix.

""householder"": computes a product of Householder reflectors
     ("householder_product()").

""matrix_exp""/""cayley"" often make the parametrized weight
   converge faster than ""householder"", but they are slower to
   compute for very thin or very wide matrices.
If "use_trivialization=True" (default), the parametrization
   implements the "Dynamic Trivialization Framework", where an extra
   matrix B \in \mathbb{K}^{n \times n} is stored under
   "module.parametrizations.weight[0].base". This helps the
   convergence of the parametrized layer at the expense of some extra


memory use. See Trivializations for Gradient-Based Optimization on
   Manifolds .
Initial value of Q: If the original tensor is not parametrized and
   "use_trivialization=True" (default), the initial value of Q is that
   of the original tensor if it is orthogonal (or unitary in the
   complex case) and it is orthogonalized via the QR decomposition
   otherwise (see "torch.linalg.qr()"). Same happens when it is not
   parametrized and "orthogonal_map="householder"" even when
   "use_trivialization=False". Otherwise, the initial value is the
   result of the composition of all the registered parametrizations
   applied to the original tensor.
Note:
 This function is implemented using the parametrization
 functionality in "register_parametrization()".

Parameters:
      * module (nn.Module) -- module on which to register the
        parametrization.
  * **name** (*str**, **optional*) -- name of the tensor to make
    orthogonal. Default: ""weight"".


orthogonal. Default: ""weight"".
  * **orthogonal_map** (*str**, **optional*) -- One of the
    following: ""matrix_exp"", ""cayley"", ""householder"".
    Default: ""matrix_exp"" if the matrix is square or complex,
    ""householder"" otherwise.

  * **use_trivialization** (*bool**, **optional*) -- whether to
    use the dynamic trivialization framework. Default: "True".

Returns:
      The original module with an orthogonal parametrization
      registered to the specified weight
Return type:
      Module
Example:
  >>> orth_linear = orthogonal(nn.Linear(20, 40))
  >>> orth_linear
  ParametrizedLinear(
  in_features=20, out_features=40, bias=True
  (parametrizations): ModuleDict(
      (weight): ParametrizationList(
      (0): _Orthogonal()
      )
  )
  )
  >>> Q = orth_linear.weight
  >>> torch.dist(Q.T @ Q, torch.eye(20))
  tensor(4.9332e-07)


torch.Tensor.any
Tensor.any(dim=None, keepdim=False) -> Tensor
See "torch.any()"


torch.Tensor.dist
Tensor.dist(other, p=2) -> Tensor
See "torch.dist()"


torch.cuda.device_count
torch.cuda.device_count()
Returns the number of GPUs available.
Return type:
      int


SyncBatchNorm
class torch.nn.SyncBatchNorm(num_features, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True, process_group=None, device=None, dtype=None)
Applies Batch Normalization over a N-Dimensional input (a mini-
   batch of [N-2]D inputs with additional channel dimension) as
   described in the paper Batch Normalization: Accelerating Deep
   Network Training by Reducing Internal Covariate Shift .
  y = \frac{x - \mathrm{E}[x]}{ \sqrt{\mathrm{Var}[x] + \epsilon}}
  * \gamma + \beta

The mean and standard-deviation are calculated per-dimension over
   all mini-batches of the same process groups. \gamma and \beta are
   learnable parameter vectors of size C (where C is the input
   size). By default, the elements of \gamma are sampled from
   \mathcal{U}(0, 1) and the elements of \beta are set to 0. The
   standard-deviation is calculated via the biased estimator,
   equivalent to torch.var(input, unbiased=False).


Also by default, during training this layer keeps running estimates
   of its computed mean and variance, which are then used for
   normalization during evaluation. The running estimates are kept
   with a default "momentum" of 0.1.
If "track_running_stats" is set to "False", this layer then does
   not keep running estimates, and batch statistics are instead used
   during evaluation time as well.
Note:
 This "momentum" argument is different from one used in optimizer
 classes and the conventional notion of momentum. Mathematically,
 the update rule for running statistics here is \hat{x}_\text{new}
 = (1 - \text{momentum}) \times \hat{x} + \text{momentum} \times
 x_t, where \hat{x} is the estimated statistic and x_t is the new
 observed value.

Because the Batch Normalization is done for each channel in the "C"
   dimension, computing statistics on "(N, +)" slices, it's common
   terminology to call this Volumetric Batch Normalization or Spatio-


temporal Batch Normalization.
Currently "SyncBatchNorm" only supports "DistributedDataParallel"
   (DDP) with single GPU per process. Use
   "torch.nn.SyncBatchNorm.convert_sync_batchnorm()" to convert
   "BatchNorm*D" layer to "SyncBatchNorm" before wrapping Network with
   DDP.
Parameters:
      * num_features (int) -- C from an expected input of size
        (N, C, +)
  * **eps** (*float*) -- a value added to the denominator for
    numerical stability. Default: "1e-5"

  * **momentum** (*float*) -- the value used for the running_mean
    and running_var computation. Can be set to "None" for
    cumulative moving average (i.e. simple average). Default: 0.1

  * **affine** (*bool*) -- a boolean value that when set to
    "True", this module has learnable affine parameters. Default:
    "True"

  * **track_running_stats** (*bool*) -- a boolean value that when
    set to "True", this module tracks the running mean and


variance, and when set to "False", this module does not track
        such statistics, and initializes statistics buffers
        "running_mean" and "running_var" as "None". When these buffers
        are "None", this module always uses batch statistics. in both
        training and eval modes. Default: "True"
  * **process_group** (*Optional**[**Any**]*) -- synchronization
    of stats happen within each process group individually.
    Default behavior is synchronization across the whole world

Shape:
      * Input: (N, C, +)
  * Output: (N, C, +) (same shape as input)

Note:
 Synchronization of batchnorm statistics occurs only while
 training, i.e. synchronization is disabled when "model.eval()" is
 set or if "self.training" is otherwise "False".

Examples:
  >>> # With Learnable Parameters
  >>> m = nn.SyncBatchNorm(100)
  >>> # creating process group (optional)
  >>> # ranks is a list of int identifying rank ids.


ranks = list(range(8))
      >>> r1, r2 = ranks[:4], ranks[4:]
      >>> # Note: every rank calls into new_group for every
      >>> # process group created, even if that rank is not
      >>> # part of the group.
      >>> process_groups = [torch.distributed.new_group(pids) for pids in [r1, r2]]
      >>> process_group = process_groups[0 if dist.get_rank() <= 3 else 1]
      >>> # Without Learnable Parameters
      >>> m = nn.BatchNorm3d(100, affine=False, process_group=process_group)
      >>> input = torch.randn(20, 100, 35, 45, 10)
      >>> output = m(input)

  >>> # network is nn.BatchNorm layer
  >>> sync_bn_network = nn.SyncBatchNorm.convert_sync_batchnorm(network, process_group)
  >>> # only single gpu per process is currently supported
  >>> ddp_sync_bn_network = torch.nn.parallel.DistributedDataParallel(
  >>>                         sync_bn_network,
  >>>                         device_ids=[args.local_rank],


                    output_device=args.local_rank)

classmethod convert_sync_batchnorm(module, process_group=None)
  Helper function to convert all "BatchNorm*D" layers in the model
  to "torch.nn.SyncBatchNorm" layers.

  Parameters:
     * **module** (*nn.Module*) -- module containing one or more
       "BatchNorm*D" layers

     * **process_group** (*optional*) -- process group to scope
       synchronization, default is the whole world

  Returns:
     The original "module" with the converted
     "torch.nn.SyncBatchNorm" layers. If the original "module" is
     a "BatchNorm*D" layer, a new "torch.nn.SyncBatchNorm" layer
     object will be returned instead.

  Example:

     >>> # Network with nn.BatchNorm layer
     >>> module = torch.nn.Sequential(
     >>>            torch.nn.Linear(20, 100),
     >>>            torch.nn.BatchNorm1d(100),
     >>>          ).cuda()
     >>> # creating process group (optional)


creating process group (optional)
     >>> # ranks is a list of int identifying rank ids.
     >>> ranks = list(range(8))
     >>> r1, r2 = ranks[:4], ranks[4:]
     >>> # Note: every rank calls into new_group for every
     >>> # process group created, even if that rank is not
     >>> # part of the group.
     >>> process_groups = [torch.distributed.new_group(pids) for pids in [r1, r2]]
     >>> process_group = process_groups[0 if dist.get_rank() <= 3 else 1]
     >>> sync_bn_module = torch.nn.SyncBatchNorm.convert_sync_batchnorm(module, process_group)


LambdaLR
class torch.optim.lr_scheduler.LambdaLR(optimizer, lr_lambda, last_epoch=- 1, verbose=False)
Sets the learning rate of each parameter group to the initial lr
   times a given function. When last_epoch=-1, sets initial lr as lr.
Parameters:
      * optimizer (Optimizer) -- Wrapped optimizer.
  * **lr_lambda** (*function** or **list*) -- A function which
    computes a multiplicative factor given an integer parameter
    epoch, or a list of such functions, one for each group in
    optimizer.param_groups.

  * **last_epoch** (*int*) -- The index of last epoch. Default:
    -1.

  * **verbose** (*bool*) -- If "True", prints a message to stdout
    for each update. Default: "False".

-[ Example ]-

Assuming optimizer has two groups.
lambda1 = lambda epoch: epoch // 30
lambda2 = lambda epoch: 0.95 ** epoch
scheduler = LambdaLR(optimizer, lr_lambda=[lambda1, lambda2])
for epoch in range(100):


for epoch in range(100):
    train(...)
    validate(...)
    scheduler.step()

get_last_lr()
  Return last computed learning rate by current scheduler.

load_state_dict(state_dict)
  Loads the schedulers state.

  When saving or loading the scheduler, please make sure to also
  save or load the state of the optimizer.

  Parameters:
     **state_dict** (*dict*) -- scheduler state. Should be an
     object returned from a call to "state_dict()".

print_lr(is_verbose, group, lr, epoch=None)
  Display the current learning rate.

state_dict()
  Returns the state of the scheduler as a "dict".

  It contains an entry for every variable in self.__dict__ which
  is not the optimizer. The learning rate lambda functions will
  only be saved if they are callable objects and not if they are
  functions or lambdas.

  When saving or loading the scheduler, please make sure to also


save or load the state of the optimizer.


torch.eye
torch.eye(n, m=None, *, out=None, dtype=None, layout=torch.strided, device=None, requires_grad=False) -> Tensor
Returns a 2-D tensor with ones on the diagonal and zeros elsewhere.
Parameters:
      * n (int) -- the number of rows
  * **m** (*int**, **optional*) -- the number of columns with
    default being "n"

Keyword Arguments:
      * out (Tensor, optional) -- the output tensor.
  * **dtype** ("torch.dtype", optional) -- the desired data type
    of returned tensor. Default: if "None", uses a global default
    (see "torch.set_default_tensor_type()").

  * **layout** ("torch.layout", optional) -- the desired layout of
    returned Tensor. Default: "torch.strided".

  * **device** ("torch.device", optional) -- the desired device of
    returned tensor. Default: if "None", uses the current device
    for the default tensor type (see
    "torch.set_default_tensor_type()"). "device" will be the CPU


for CPU tensor types and the current CUDA device for CUDA
        tensor types.
  * **requires_grad** (*bool**, **optional*) -- If autograd should
    record operations on the returned tensor. Default: "False".

Returns:
      A 2-D tensor with ones on the diagonal and zeros elsewhere
Return type:
      Tensor
Example:
  >>> torch.eye(3)
  tensor([[ 1.,  0.,  0.],
          [ 0.,  1.,  0.],
          [ 0.,  0.,  1.]])


Adagrad
class torch.optim.Adagrad(params, lr=0.01, lr_decay=0, weight_decay=0, initial_accumulator_value=0, eps=1e-10, foreach=None, *, maximize=False, differentiable=False)
Implements Adagrad algorithm.
  \begin{aligned}      &\rule{110mm}{0.4pt}
  \\      &\textbf{input}      : \gamma \text{ (lr)}, \: \theta_0
  \text{ (params)}, \: f(\theta)          \text{ (objective)}, \:
  \lambda \text{ (weight decay)},                          \\
  &\hspace{12mm}    \tau \text{ (initial accumulator value)}, \:
  \eta\text{ (lr decay)}\\      &\textbf{initialize} :
  state\_sum_0 \leftarrow 0                             \\[-1.ex]
  &\rule{110mm}{0.4pt}
  \\      &\textbf{for} \: t=1 \: \textbf{to} \: \ldots \:
  \textbf{do}                         \\      &\hspace{5mm}g_t
  \leftarrow   \nabla_{\theta} f_t (\theta_{t-1})           \\
  &\hspace{5mm} \tilde{\gamma}    \leftarrow \gamma / (1 +(t-1)


\eta)                  \      &\hspace{5mm} \textbf{if} \:
      \lambda \neq 0                                          \
      &\hspace{10mm} g_t \leftarrow g_t + \lambda \theta_{t-1}
      \      &\hspace{5mm}state_sum_t  \leftarrow  state_sum_{t-1}
      + g^2_t                      \      &\hspace{5mm}\theta_t
      \leftarrow          \theta_{t-1}- \tilde{\gamma}
      \frac{g_t}{\sqrt{state_sum_t}+\epsilon}            \
      &\rule{110mm}{0.4pt}
      \[-1.ex]      &\bf{return} \:  \theta_t
      \[-1.ex]      &\rule{110mm}{0.4pt}
      \[-1.ex] \end{aligned}
For further details regarding the algorithm we refer to Adaptive
   Subgradient Methods for Online Learning and Stochastic
   Optimization.
Parameters:
      * params (iterable) -- iterable of parameters to optimize
        or dicts defining parameter groups
  * **lr** (*float**, **optional*) -- learning rate (default:
    1e-2)

  * **lr_decay** (*float**, **optional*) -- learning rate decay


(default: 0)
  * **weight_decay** (*float**, **optional*) -- weight decay (L2
    penalty) (default: 0)

  * **eps** (*float**, **optional*) -- term added to the
    denominator to improve numerical stability (default: 1e-10)

  * **foreach** (*bool**, **optional*) -- whether foreach
    implementation of optimizer is used. If unspecified by the
    user (so foreach is None), we will try to use foreach over the
    for-loop implementation on CUDA, since it is usually
    significantly more performant. (default: None)

  * **maximize** (*bool**, **optional*) -- maximize the params
    based on the objective, instead of minimizing (default: False)

  * **differentiable** (*bool**, **optional*) -- whether autograd
    should occur through the optimizer step in training.
    Otherwise, the step() function runs in a torch.no_grad()
    context. Setting to True can impair performance, so leave it


False if you don't intend to run autograd through this
        instance (default: False)
add_param_group(param_group)
  Add a param group to the "Optimizer" s *param_groups*.

  This can be useful when fine tuning a pre-trained network as
  frozen layers can be made trainable and added to the "Optimizer"
  as training progresses.

  Parameters:
     **param_group** (*dict*) -- Specifies what Tensors should be
     optimized along with group specific optimization options.

load_state_dict(state_dict)
  Loads the optimizer state.

  Parameters:
     **state_dict** (*dict*) -- optimizer state. Should be an
     object returned from a call to "state_dict()".

register_step_post_hook(hook)
  Register an optimizer step post hook which will be called after
  optimizer step. It should have the following signature:

     hook(optimizer, args, kwargs) -> None

  The "optimizer" argument is the optimizer instance being used.


Parameters:
         hook (Callable) -- The user defined hook to be
         registered.
  Returns:
     a handle that can be used to remove the added hook by calling
     "handle.remove()"

  Return type:
     "torch.utils.hooks.RemoveableHandle"

register_step_pre_hook(hook)
  Register an optimizer step pre hook which will be called before
  optimizer step. It should have the following signature:

     hook(optimizer, args, kwargs) -> None or modified args and kwargs

  The "optimizer" argument is the optimizer instance being used.
  If args and kwargs are modified by the pre-hook, then the
  transformed values are returned as a tuple containing the
  new_args and new_kwargs.

  Parameters:
     **hook** (*Callable*) -- The user defined hook to be
     registered.

  Returns:
     a handle that can be used to remove the added hook by calling
     "handle.remove()"

  Return type:


"handle.remove()"
  Return type:
     "torch.utils.hooks.RemoveableHandle"

state_dict()
  Returns the state of the optimizer as a "dict".

  It contains two entries:

  * state - a dict holding current optimization state. Its content
       differs between optimizer classes.

  * param_groups - a list containing all parameter groups where
    each
       parameter group is a dict

zero_grad(set_to_none=False)
  Sets the gradients of all optimized "torch.Tensor" s to zero.

  Parameters:
     **set_to_none** (*bool*) -- instead of setting to zero, set
     the grads to None. This will in general have lower memory
     footprint, and can modestly improve performance. However, it
     changes certain behaviors. For example: 1. When the user
     tries to access a gradient and perform manual ops on it, a
     None attribute or a Tensor full of 0s will behave
     differently. 2. If the user requests


differently. 2. If the user requests
         "zero_grad(set_to_none=True)" followed by a backward pass,
         ".grad"s are guaranteed to be None for params that did not
         receive a gradient. 3. "torch.optim" optimizers have a
         different behavior if the gradient is 0 or None (in one case
         it does the step with a gradient of 0 and in the other it
         skips the step altogether).


torch.nn.functional.tanh
torch.nn.functional.tanh(input) -> Tensor
Applies element-wise, \text{Tanh}(x) = \tanh(x) = \frac{\exp(x) -
   \exp(-x)}{\exp(x) + \exp(-x)}
See "Tanh" for more details.


torch.Tensor.cholesky_inverse
Tensor.cholesky_inverse(upper=False) -> Tensor
See "torch.cholesky_inverse()"


torch.Tensor.new_empty
Tensor.new_empty(size, *, dtype=None, device=None, requires_grad=False, layout=torch.strided, pin_memory=False) -> Tensor
Returns a Tensor of size "size" filled with uninitialized data. By
   default, the returned Tensor has the same "torch.dtype" and
   "torch.device" as this tensor.
Parameters:
      size (int...) -- a list, tuple, or "torch.Size" of
      integers defining the shape of the output tensor.
Keyword Arguments:
      * dtype ("torch.dtype", optional) -- the desired type of
        returned tensor. Default: if None, same "torch.dtype" as this
        tensor.
  * **device** ("torch.device", optional) -- the desired device of
    returned tensor. Default: if None, same "torch.device" as this
    tensor.

  * **requires_grad** (*bool**, **optional*) -- If autograd should
    record operations on the returned tensor. Default: "False".

  * **layout** ("torch.layout", optional) -- the desired layout of


returned Tensor. Default: "torch.strided".
  * **pin_memory** (*bool**, **optional*) -- If set, returned
    tensor would be allocated in the pinned memory. Works only for
    CPU tensors. Default: "False".

Example:
  >>> tensor = torch.ones(())
  >>> tensor.new_empty((2, 3))
  tensor([[ 5.8182e-18,  4.5765e-41, -1.0545e+30],
          [ 3.0949e-41,  4.4842e-44,  0.0000e+00]])


default_debug_observer
torch.quantization.observer.default_debug_observer
alias of "RecordingObserver"


default_per_channel_weight_observer
torch.quantization.observer.default_per_channel_weight_observer
alias of functools.partial(,
   dtype=torch.qint8, qscheme=torch.per_channel_symmetric){}


torch.Tensor.transpose
Tensor.transpose(dim0, dim1) -> Tensor
See "torch.transpose()"


InstanceNorm2d
class torch.ao.nn.quantized.InstanceNorm2d(num_features, weight, bias, scale, zero_point, eps=1e-05, momentum=0.1, affine=False, track_running_stats=False, device=None, dtype=None)
This is the quantized version of "InstanceNorm2d".
Additional args:
      * scale - quantization scale of the output, type: double.
  * **zero_point** - quantization zero point of the output, type:
    long.


torch.Tensor.lerp
Tensor.lerp(end, weight) -> Tensor
See "torch.lerp()"


torch.Tensor.div_
Tensor.div_(value, *, rounding_mode=None) -> Tensor
In-place version of "div()"


torch.diag
torch.diag(input, diagonal=0, *, out=None) -> Tensor

If "input" is a vector (1-D tensor), then returns a 2-D square
     tensor with the elements of "input" as the diagonal.

If "input" is a matrix (2-D tensor), then returns a 1-D tensor
     with the diagonal elements of "input".

The argument "diagonal" controls which diagonal to consider:

If "diagonal" = 0, it is the main diagonal.

If "diagonal" > 0, it is above the main diagonal.

If "diagonal" < 0, it is below the main diagonal.

Parameters:
      * input (Tensor) -- the input tensor.
  * **diagonal** (*int**, **optional*) -- the diagonal to consider

Keyword Arguments:
      out (Tensor, optional) -- the output tensor.
See also:
 "torch.diagonal()" always returns the diagonal of its input.

 "torch.diagflat()" always constructs a tensor with diagonal
 elements specified by the input.

Examples:


Examples:
Get the square matrix where the input vector is the diagonal:
  >>> a = torch.randn(3)
  >>> a
  tensor([ 0.5950,-0.0872, 2.3298])
  >>> torch.diag(a)
  tensor([[ 0.5950, 0.0000, 0.0000],
          [ 0.0000,-0.0872, 0.0000],
          [ 0.0000, 0.0000, 2.3298]])
  >>> torch.diag(a, 1)
  tensor([[ 0.0000, 0.5950, 0.0000, 0.0000],
          [ 0.0000, 0.0000,-0.0872, 0.0000],
          [ 0.0000, 0.0000, 0.0000, 2.3298],
          [ 0.0000, 0.0000, 0.0000, 0.0000]])

Get the k-th diagonal of a given matrix:
  >>> a = torch.randn(3, 3)
  >>> a
  tensor([[-0.4264, 0.0255,-0.1064],
          [ 0.8795,-0.2429, 0.1374],
          [ 0.1029,-0.6482,-1.6300]])
  >>> torch.diag(a, 0)
  tensor([-0.4264,-0.2429,-1.6300])
  >>> torch.diag(a, 1)
  tensor([ 0.0255, 0.1374])


torch.nn.functional.multilabel_soft_margin_loss
torch.nn.functional.multilabel_soft_margin_loss(input, target, weight=None, size_average=None, reduce=None, reduction='mean') -> Tensor
See "MultiLabelSoftMarginLoss" for details.
Return type:
      Tensor


ConvBnReLU1d
class torch.ao.nn.intrinsic.ConvBnReLU1d(conv, bn, relu)
This is a sequential container which calls the Conv 1d, Batch Norm
   1d, and ReLU modules. During quantization this will be replaced
   with the corresponding fused module.


torch.Tensor.isclose
Tensor.isclose(other, rtol=1e-05, atol=1e-08, equal_nan=False) -> Tensor
See "torch.isclose()"


torch.fft.hfft2
torch.fft.hfft2(input, s=None, dim=(- 2, - 1), norm=None, *, out=None) -> Tensor
Computes the 2-dimensional discrete Fourier transform of a
   Hermitian symmetric "input" signal. Equivalent to "hfftn()" but
   only transforms the last two dimensions by default.
"input" is interpreted as a one-sided Hermitian signal in the time
   domain. By the Hermitian property, the Fourier transform will be
   real-valued.
Note:
 Supports torch.half and torch.chalf on CUDA with GPU Architecture
 SM53 or greater. However it only supports powers of 2 signal
 length in every transformed dimensions. With default arguments,
 the size of last dimension should be (2^n + 1) as argument *s*
 defaults to even output size = 2 * (last_dim_size - 1)

Parameters:
      * input (Tensor) -- the input tensor
  * **s** (*Tuple**[**int**]**, **optional*) -- Signal size in the
    transformed dimensions. If given, each dimension "dim[i]" will


either be zero-padded or trimmed to the length "s[i]" before
        computing the Hermitian FFT. If a length "-1" is specified, no
        padding is done in that dimension. Defaults to even output in
        the last dimension: "s[-1] = 2*(input.size(dim[-1]) - 1)".
  * **dim** (*Tuple**[**int**]**, **optional*) -- Dimensions to be
    transformed. The last dimension must be the half-Hermitian
    compressed dimension. Default: last two dimensions.

  * **norm** (*str**, **optional*) --

    Normalization mode. For the forward transform ("hfft2()"),
    these correspond to:

    * ""forward"" - normalize by "1/n"

    * ""backward"" - no normalization

    * ""ortho"" - normalize by "1/sqrt(n)" (making the Hermitian
      FFT orthonormal)

    Where "n = prod(s)" is the logical FFT size. Calling the
    backward transform ("ihfft2()") with the same normalization
    mode will apply an overall normalization of "1/n" between the


two transforms. This is required to make "ihfft2()" the exact
        inverse.
    Default is ""backward"" (no normalization).

Keyword Arguments:
      out (Tensor, optional) -- the output tensor.
-[ Example ]-
Starting from a real frequency-space signal, we can generate a
   Hermitian-symmetric time-domain signal: >>> T = torch.rand(10, 9)

t = torch.fft.ihfft2(T)

Without specifying the output length to "hfftn()", the output will
   not round-trip properly because the input is odd-length in the last
   dimension:

torch.fft.hfft2(t).size()
   torch.Size([10, 10])

So, it is recommended to always pass the signal shape "s".

roundtrip = torch.fft.hfft2(t, T.size())
roundtrip.size()
   torch.Size([10, 9])
torch.allclose(roundtrip, T)
   True


torch.Tensor.bitwise_and
Tensor.bitwise_and() -> Tensor
See "torch.bitwise_and()"


torch.topk
torch.topk(input, k, dim=None, largest=True, sorted=True, *, out=None)
Returns the "k" largest elements of the given "input" tensor along
   a given dimension.
If "dim" is not given, the last dimension of the input is chosen.
If "largest" is "False" then the k smallest elements are
   returned.
A namedtuple of (values, indices) is returned with the values
   and indices of the largest k elements of each row of the
   input tensor in the given dimension dim.
The boolean option "sorted" if "True", will make sure that the
   returned k elements are themselves sorted
Parameters:
      * input (Tensor) -- the input tensor.
  * **k** (*int*) -- the k in "top-k"

  * **dim** (*int**, **optional*) -- the dimension to sort along

  * **largest** (*bool**, **optional*) -- controls whether to
    return largest or smallest elements

  * **sorted** (*bool**, **optional*) -- controls whether to


return the elements in sorted order
Keyword Arguments:
      out (tuple, optional) -- the output tuple of (Tensor,
      LongTensor) that can be optionally given to be used as output
      buffers
Example:
  >>> x = torch.arange(1., 6.)
  >>> x
  tensor([ 1.,  2.,  3.,  4.,  5.])
  >>> torch.topk(x, 3)
  torch.return_types.topk(values=tensor([5., 4., 3.]), indices=tensor([4, 3, 2]))


SmoothL1Loss
class torch.nn.SmoothL1Loss(size_average=None, reduce=None, reduction='mean', beta=1.0)
Creates a criterion that uses a squared term if the absolute
   element-wise error falls below beta and an L1 term otherwise. It is
   less sensitive to outliers than "torch.nn.MSELoss" and in some
   cases prevents exploding gradients (e.g. see the paper Fast R-CNN
   by Ross Girshick).
For a batch of size N, the unreduced loss can be described as:
  \ell(x, y) = L = \{l_1, ..., l_N\}^T

with
  l_n = \begin{cases} 0.5 (x_n - y_n)^2 / beta, & \text{if } |x_n
  - y_n| < beta \\ |x_n - y_n| - 0.5 * beta, & \text{otherwise }
  \end{cases}

If reduction is not none, then:
  \ell(x, y) = \begin{cases}     \operatorname{mean}(L), &
  \text{if reduction} = \text{`mean';}\\
  \operatorname{sum}(L),  &  \text{if reduction} = \text{`sum'.}
  \end{cases}

Note:
 Smooth L1 loss can be seen as exactly "L1Loss", but with the |x -


y| < beta portion replaced with a quadratic function such that
     its slope is 1 at |x - y| = beta. The quadratic segment smooths
     the L1 loss near |x - y| = 0.
Note:
 Smooth L1 loss is closely related to "HuberLoss", being
 equivalent to huber(x, y) / beta (note that Smooth L1's beta
 hyper-parameter is also known as delta for Huber). This leads to
 the following differences:

 * As beta -> 0, Smooth L1 loss converges to "L1Loss", while
   "HuberLoss" converges to a constant 0 loss. When beta is 0,
   Smooth L1 loss is equivalent to L1 loss.

 * As beta -> +\infty, Smooth L1 loss converges to a constant 0
   loss, while "HuberLoss" converges to "MSELoss".

 * For Smooth L1 loss, as beta varies, the L1 segment of the loss
   has a constant slope of 1. For "HuberLoss", the slope of the L1
   segment is beta.

Parameters:
      * size_average (bool, optional) -- Deprecated (see


"reduction"). By default, the losses are averaged over each
        loss element in the batch. Note that for some losses, there
        are multiple elements per sample. If the field "size_average"
        is set to "False", the losses are instead summed for each
        minibatch. Ignored when "reduce" is "False". Default: "True"
  * **reduce** (*bool**, **optional*) -- Deprecated (see
    "reduction"). By default, the losses are averaged or summed
    over observations for each minibatch depending on
    "size_average". When "reduce" is "False", returns a loss per
    batch element instead and ignores "size_average". Default:
    "True"

  * **reduction** (*str**, **optional*) -- Specifies the reduction
    to apply to the output: "'none'" | "'mean'" | "'sum'".
    "'none'": no reduction will be applied, "'mean'": the sum of
    the output will be divided by the number of elements in the
    output, "'sum'": the output will be summed. Note:


"size_average" and "reduce" are in the process of being
        deprecated, and in the meantime, specifying either of those
        two args will override "reduction". Default: "'mean'"
  * **beta** (*float**, **optional*) -- Specifies the threshold at
    which to change between L1 and L2 loss. The value must be non-
    negative. Default: 1.0

Shape:
      * Input: (*), where * means any number of dimensions.
  * Target: (*), same shape as the input.

  * Output: scalar. If "reduction" is "'none'", then (*), same
    shape as the input.


AdaptiveAvgPool2d
class torch.nn.AdaptiveAvgPool2d(output_size)
Applies a 2D adaptive average pooling over an input signal composed
   of several input planes.
The output is of size H x W, for any input size. The number of
   output features is equal to the number of input planes.
Parameters:
      output_size (Union[int, None,
      Tuple[Optional[int], Optional[int]]])
      -- the target output size of the image of the form H x W. Can be
      a tuple (H, W) or a single H for a square image H x H. H and W
      can be either a "int", or "None" which means the size will be
      the same as that of the input.
Shape:
      * Input: (N, C, H_{in}, W_{in}) or (C, H_{in}, W_{in}).
  * Output: (N, C, S_{0}, S_{1}) or (C, S_{0}, S_{1}), where
    S=\text{output\_size}.

-[ Examples ]-

target output size of 5x7
m = nn.AdaptiveAvgPool2d((5, 7))
input = torch.randn(1, 64, 8, 9)
output = m(input)


output = m(input)
target output size of 7x7 (square)
m = nn.AdaptiveAvgPool2d(7)
input = torch.randn(1, 64, 10, 9)
output = m(input)
target output size of 10x7
m = nn.AdaptiveAvgPool2d((None, 7))
input = torch.randn(1, 64, 10, 9)
output = m(input)