Adagradclass torch.optim.Adagrad(params, lr=0.01, lr_decay=0, weight_decay=0, initial_accumulator_value=0, eps=1e-10, foreach=None, *, maximize=False, differentiable=False)
   Implements Adagrad algorithm.
      \begin{aligned}      &\rule{110mm}{0.4pt}
      \      &\textbf{input}      : \gamma \text{ (lr)}, \: \theta_0
      \text{ (params)}, \: f(\theta)          \text{ (objective)}, \:
      \lambda \text{ (weight decay)},                          \
      &\hspace{12mm}    \tau \text{ (initial accumulator value)}, \:
      \eta\text{ (lr decay)}\      &\textbf{initialize} :
      state_sum_0 \leftarrow 0                             \[-1.ex]
      &\rule{110mm}{0.4pt}
      \      &\textbf{for} \: t=1 \: \textbf{to} \: \ldots \:
      \textbf{do}                         \      &\hspace{5mm}g_t
      \leftarrow   \nabla_{\theta} f_t (\theta_{t-1})           \
      &\hspace{5mm} \tilde{\gamma}    \leftarrow \gamma / (1 +(t-1)
      \eta)                  \      &\hspace{5mm} \textbf{if} \:


\lambda \neq 0                                          \
      &\hspace{10mm} g_t \leftarrow g_t + \lambda \theta_{t-1}
      \      &\hspace{5mm}state_sum_t  \leftarrow  state_sum_{t-1}
      + g^2_t                      \      &\hspace{5mm}\theta_t
      \leftarrow          \theta_{t-1}- \tilde{\gamma}
      \frac{g_t}{\sqrt{state_sum_t}+\epsilon}            \
      &\rule{110mm}{0.4pt}
      \[-1.ex]      &\bf{return} \:  \theta_t
      \[-1.ex]      &\rule{110mm}{0.4pt}
      \[-1.ex] \end{aligned}
   For further details regarding the algorithm we refer to Adaptive
   Subgradient Methods for Online Learning and Stochastic
   Optimization.
   Parameters:
      * params (iterable) -- iterable of parameters to optimize
        or dicts defining parameter groups
      * lr (float, optional) -- learning rate (default:
        1e-2)
      * lr_decay (float, optional) -- learning rate decay
        (default: 0)


(default: 0)
      * weight_decay (float, optional) -- weight decay (L2
        penalty) (default: 0)
      * eps (float, optional) -- term added to the
        denominator to improve numerical stability (default: 1e-10)
      * foreach (bool, optional) -- whether foreach
        implementation of optimizer is used. If unspecified by the
        user (so foreach is None), we will try to use foreach over the
        for-loop implementation on CUDA, since it is usually
        significantly more performant. (default: None)
      * maximize (bool, optional) -- maximize the params
        based on the objective, instead of minimizing (default: False)
      * differentiable (bool, optional) -- whether autograd
        should occur through the optimizer step in training.
        Otherwise, the step() function runs in a torch.no_grad()
        context. Setting to True can impair performance, so leave it


False if you don't intend to run autograd through this
        instance (default: False)
   add_param_group(param_group)
      Add a param group to the "Optimizer" s param_groups.
      This can be useful when fine tuning a pre-trained network as
      frozen layers can be made trainable and added to the "Optimizer"
      as training progresses.
      Parameters:
         param_group (dict) -- Specifies what Tensors should be
         optimized along with group specific optimization options.
   load_state_dict(state_dict)
      Loads the optimizer state.
      Parameters:
         state_dict (dict) -- optimizer state. Should be an
         object returned from a call to "state_dict()".
   register_step_post_hook(hook)
      Register an optimizer step post hook which will be called after
      optimizer step. It should have the following signature:
         hook(optimizer, args, kwargs) -> None
      The "optimizer" argument is the optimizer instance being used.
      Parameters:


Parameters:
         hook (Callable) -- The user defined hook to be
         registered.
      Returns:
         a handle that can be used to remove the added hook by calling
         "handle.remove()"
      Return type:
         "torch.utils.hooks.RemoveableHandle"
   register_step_pre_hook(hook)
      Register an optimizer step pre hook which will be called before
      optimizer step. It should have the following signature:
         hook(optimizer, args, kwargs) -> None or modified args and kwargs
      The "optimizer" argument is the optimizer instance being used.
      If args and kwargs are modified by the pre-hook, then the
      transformed values are returned as a tuple containing the
      new_args and new_kwargs.
      Parameters:
         hook (Callable) -- The user defined hook to be
         registered.
      Returns:
         a handle that can be used to remove the added hook by calling
         "handle.remove()"
      Return type:
         "torch.utils.hooks.RemoveableHandle"


"torch.utils.hooks.RemoveableHandle"
   state_dict()
      Returns the state of the optimizer as a "dict".
      It contains two entries:
      * state - a dict holding current optimization state. Its content
           differs between optimizer classes.
      * param_groups - a list containing all parameter groups where
        each
           parameter group is a dict
   zero_grad(set_to_none=False)
      Sets the gradients of all optimized "torch.Tensor" s to zero.
      Parameters:
         set_to_none (bool) -- instead of setting to zero, set
         the grads to None. This will in general have lower memory
         footprint, and can modestly improve performance. However, it
         changes certain behaviors. For example: 1. When the user
         tries to access a gradient and perform manual ops on it, a
         None attribute or a Tensor full of 0s will behave
         differently. 2. If the user requests
         "zero_grad(set_to_none=True)" followed by a backward pass,


".grad"s are guaranteed to be None for params that did not
         receive a gradient. 3. "torch.optim" optimizers have a
         different behavior if the gradient is 0 or None (in one case
         it does the step with a gradient of 0 and in the other it
         skips the step altogether).


torch.nn.functional.tanhtorch.nn.functional.tanh(input) -> Tensor
   Applies element-wise, \text{Tanh}(x) = \tanh(x) = \frac{\exp(x) -
   \exp(-x)}{\exp(x) + \exp(-x)}
   See "Tanh" for more details.


torch.Tensor.cholesky_inverseTensor.cholesky_inverse(upper=False) -> Tensor
   See "torch.cholesky_inverse()"


torch.Tensor.new_emptyTensor.new_empty(size, , dtype=None, device=None, requires_grad=False, layout=torch.strided, pin_memory=False) -> Tensor
   Returns a Tensor of size "size" filled with uninitialized data. By
   default, the returned Tensor has the same "torch.dtype" and
   "torch.device" as this tensor.
   Parameters:
      size (int...*) -- a list, tuple, or "torch.Size" of
      integers defining the shape of the output tensor.
   Keyword Arguments:
      * dtype ("torch.dtype", optional) -- the desired type of
        returned tensor. Default: if None, same "torch.dtype" as this
        tensor.
      * device ("torch.device", optional) -- the desired device of
        returned tensor. Default: if None, same "torch.device" as this
        tensor.
      * requires_grad (*bool, optional*) -- If autograd should
        record operations on the returned tensor. Default: "False".
      * layout** ("torch.layout", optional) -- the desired layout of


returned Tensor. Default: "torch.strided".
      * pin_memory (bool, optional) -- If set, returned
        tensor would be allocated in the pinned memory. Works only for
        CPU tensors. Default: "False".
   Example:
      >>> tensor = torch.ones(())
      >>> tensor.new_empty((2, 3))
      tensor([[ 5.8182e-18,  4.5765e-41, -1.0545e+30],
              [ 3.0949e-41,  4.4842e-44,  0.0000e+00]])


default_debug_observertorch.quantization.observer.default_debug_observer
   alias of "RecordingObserver"


default_per_channel_weight_observertorch.quantization.observer.default_per_channel_weight_observer
   alias of functools.partial(,
   dtype=torch.qint8, qscheme=torch.per_channel_symmetric){}


torch.Tensor.transposeTensor.transpose(dim0, dim1) -> Tensor
   See "torch.transpose()"


InstanceNorm2dclass torch.ao.nn.quantized.InstanceNorm2d(num_features, weight, bias, scale, zero_point, eps=1e-05, momentum=0.1, affine=False, track_running_stats=False, device=None, dtype=None)
   This is the quantized version of "InstanceNorm2d".
   Additional args:
      * scale - quantization scale of the output, type: double.
      * zero_point - quantization zero point of the output, type:
        long.


torch.Tensor.lerpTensor.lerp(end, weight) -> Tensor
   See "torch.lerp()"


torch.Tensor.div_Tensor.div_(value, *, rounding_mode=None) -> Tensor
   In-place version of "div()"


torch.diagtorch.diag(input, diagonal=0, , out=None) -> Tensor
   * If "input" is a vector (1-D tensor), then returns a 2-D square
     tensor with the elements of "input" as the diagonal.
   * If "input" is a matrix (2-D tensor), then returns a 1-D tensor
     with the diagonal elements of "input".
   The argument "diagonal" controls which diagonal to consider:
   * If "diagonal" = 0, it is the main diagonal.
   * If "diagonal" > 0, it is above the main diagonal.
   * If "diagonal" < 0, it is below the main diagonal.
   Parameters:
      * input (Tensor) -- the input tensor.
      * diagonal (int, optional) -- the diagonal to consider
   Keyword Arguments:
      out (Tensor, optional*) -- the output tensor.
   See also:
     "torch.diagonal()" always returns the diagonal of its input.
     "torch.diagflat()" always constructs a tensor with diagonal
     elements specified by the input.
   Examples:
   Get the square matrix where the input vector is the diagonal:


a = torch.randn(3)
      >>> a
      tensor([ 0.5950,-0.0872, 2.3298])
      >>> torch.diag(a)
      tensor([[ 0.5950, 0.0000, 0.0000],
              [ 0.0000,-0.0872, 0.0000],
              [ 0.0000, 0.0000, 2.3298]])
      >>> torch.diag(a, 1)
      tensor([[ 0.0000, 0.5950, 0.0000, 0.0000],
              [ 0.0000, 0.0000,-0.0872, 0.0000],
              [ 0.0000, 0.0000, 0.0000, 2.3298],
              [ 0.0000, 0.0000, 0.0000, 0.0000]])
   Get the k-th diagonal of a given matrix:
      >>> a = torch.randn(3, 3)
      >>> a
      tensor([[-0.4264, 0.0255,-0.1064],
              [ 0.8795,-0.2429, 0.1374],
              [ 0.1029,-0.6482,-1.6300]])
      >>> torch.diag(a, 0)
      tensor([-0.4264,-0.2429,-1.6300])
      >>> torch.diag(a, 1)
      tensor([ 0.0255, 0.1374])


torch.nn.functional.multilabel_soft_margin_losstorch.nn.functional.multilabel_soft_margin_loss(input, target, weight=None, size_average=None, reduce=None, reduction='mean') -> Tensor
   See "MultiLabelSoftMarginLoss" for details.
   Return type:
      Tensor


ConvBnReLU1dclass torch.ao.nn.intrinsic.ConvBnReLU1d(conv, bn, relu)
   This is a sequential container which calls the Conv 1d, Batch Norm
   1d, and ReLU modules. During quantization this will be replaced
   with the corresponding fused module.


torch.Tensor.iscloseTensor.isclose(other, rtol=1e-05, atol=1e-08, equal_nan=False) -> Tensor
   See "torch.isclose()"


torch.fft.hfft2torch.fft.hfft2(input, s=None, dim=(- 2, - 1), norm=None, , out=None) -> Tensor
   Computes the 2-dimensional discrete Fourier transform of a
   Hermitian symmetric "input" signal. Equivalent to "hfftn()" but
   only transforms the last two dimensions by default.
   "input" is interpreted as a one-sided Hermitian signal in the time
   domain. By the Hermitian property, the Fourier transform will be
   real-valued.
   Note:
     Supports torch.half and torch.chalf on CUDA with GPU Architecture
     SM53 or greater. However it only supports powers of 2 signal
     length in every transformed dimensions. With default arguments,
     the size of last dimension should be (2^n + 1) as argument s
     defaults to even output size = 2 * (last_dim_size - 1)
   Parameters:
      * input (Tensor) -- the input tensor
      * s (Tuple[int], *optional) -- Signal size in the
        transformed dimensions. If given, each dimension "dim[i]" will


either be zero-padded or trimmed to the length "s[i]" before
        computing the Hermitian FFT. If a length "-1" is specified, no
        padding is done in that dimension. Defaults to even output in
        the last dimension: "s[-1] = 2(input.size(dim[-1]) - 1)".
      * dim (Tuple[int], optional*) -- Dimensions to be
        transformed. The last dimension must be the half-Hermitian
        compressed dimension. Default: last two dimensions.
      * norm (*str, *optional) --
        Normalization mode. For the forward transform ("hfft2()"),
        these correspond to:
        * ""forward"" - normalize by "1/n"
        * ""backward"" - no normalization
        * ""ortho"" - normalize by "1/sqrt(n)" (making the Hermitian
          FFT orthonormal)
        Where "n = prod(s)" is the logical FFT size. Calling the
        backward transform ("ihfft2()") with the same normalization
        mode will apply an overall normalization of "1/n" between the


two transforms. This is required to make "ihfft2()" the exact
        inverse.
        Default is ""backward"" (no normalization).
   Keyword Arguments:
      out (Tensor, optional) -- the output tensor.
   -[ Example ]-
   Starting from a real frequency-space signal, we can generate a
   Hermitian-symmetric time-domain signal: >>> T = torch.rand(10, 9)

t = torch.fft.ihfft2(T)
   Without specifying the output length to "hfftn()", the output will
   not round-trip properly because the input is odd-length in the last
   dimension:
torch.fft.hfft2(t).size()
   torch.Size([10, 10])
   So, it is recommended to always pass the signal shape "s".
roundtrip = torch.fft.hfft2(t, T.size())
roundtrip.size()
   torch.Size([10, 9])
torch.allclose(roundtrip, T)
   True


torch.Tensor.bitwise_andTensor.bitwise_and() -> Tensor
   See "torch.bitwise_and()"


torch.topktorch.topk(input, k, dim=None, largest=True, sorted=True, , out=None)
   Returns the "k" largest elements of the given "input" tensor along
   a given dimension.
   If "dim" is not given, the last dimension of the input is chosen.
   If "largest" is "False" then the k smallest elements are
   returned.
   A namedtuple of (values, indices) is returned with the values
   and indices of the largest k elements of each row of the
   input tensor in the given dimension dim.
   The boolean option "sorted" if "True", will make sure that the
   returned k elements are themselves sorted
   Parameters:
      * input (Tensor) -- the input tensor.
      * k (int) -- the k in "top-k"
      * dim (int, optional) -- the dimension to sort along
      * largest (bool, optional) -- controls whether to
        return largest or smallest elements
      * sorted (bool, optional*) -- controls whether to
        return the elements in sorted order


return the elements in sorted order
   Keyword Arguments:
      out (tuple, optional) -- the output tuple of (Tensor,
      LongTensor) that can be optionally given to be used as output
      buffers
   Example:
      >>> x = torch.arange(1., 6.)
      >>> x
      tensor([ 1.,  2.,  3.,  4.,  5.])
      >>> torch.topk(x, 3)
      torch.return_types.topk(values=tensor([5., 4., 3.]), indices=tensor([4, 3, 2]))


SmoothL1Lossclass torch.nn.SmoothL1Loss(size_average=None, reduce=None, reduction='mean', beta=1.0)
   Creates a criterion that uses a squared term if the absolute
   element-wise error falls below beta and an L1 term otherwise. It is
   less sensitive to outliers than "torch.nn.MSELoss" and in some
   cases prevents exploding gradients (e.g. see the paper Fast R-CNN
   by Ross Girshick).
   For a batch of size N, the unreduced loss can be described as:
      \ell(x, y) = L = {l_1, ..., l_N}^T
   with
      l_n = \begin{cases} 0.5 (x_n - y_n)^2 / beta, & \text{if } |x_n
      - y_n| < beta \ |x_n - y_n| - 0.5 * beta, & \text{otherwise }
      \end{cases}
   If reduction is not none, then:
      \ell(x, y) = \begin{cases}     \operatorname{mean}(L), &
      \text{if reduction} = \text{mean';}\\
      \operatorname{sum}(L),  &  \text{if reduction} = \text{sum'.}
      \end{cases}
   Note:
     Smooth L1 loss can be seen as exactly "L1Loss", but with the |x -


y| < beta portion replaced with a quadratic function such that
     its slope is 1 at |x - y| = beta. The quadratic segment smooths
     the L1 loss near |x - y| = 0.
   Note:
     Smooth L1 loss is closely related to "HuberLoss", being
     equivalent to huber(x, y) / beta (note that Smooth L1's beta
     hyper-parameter is also known as delta for Huber). This leads to
     the following differences:
     * As beta -> 0, Smooth L1 loss converges to "L1Loss", while
       "HuberLoss" converges to a constant 0 loss. When beta is 0,
       Smooth L1 loss is equivalent to L1 loss.
     * As beta -> +\infty, Smooth L1 loss converges to a constant 0
       loss, while "HuberLoss" converges to "MSELoss".
     * For Smooth L1 loss, as beta varies, the L1 segment of the loss
       has a constant slope of 1. For "HuberLoss", the slope of the L1
       segment is beta.
   Parameters:
      * size_average (bool, optional) -- Deprecated (see


"reduction"). By default, the losses are averaged over each
        loss element in the batch. Note that for some losses, there
        are multiple elements per sample. If the field "size_average"
        is set to "False", the losses are instead summed for each
        minibatch. Ignored when "reduce" is "False". Default: "True"
      * reduce (bool, optional) -- Deprecated (see
        "reduction"). By default, the losses are averaged or summed
        over observations for each minibatch depending on
        "size_average". When "reduce" is "False", returns a loss per
        batch element instead and ignores "size_average". Default:
        "True"
      * reduction (str, optional) -- Specifies the reduction
        to apply to the output: "'none'" | "'mean'" | "'sum'".
        "'none'": no reduction will be applied, "'mean'": the sum of
        the output will be divided by the number of elements in the
        output, "'sum'": the output will be summed. Note:


"size_average" and "reduce" are in the process of being
        deprecated, and in the meantime, specifying either of those
        two args will override "reduction". Default: "'mean'"
      * beta (float, optional) -- Specifies the threshold at
        which to change between L1 and L2 loss. The value must be non-
        negative. Default: 1.0
   Shape:
      * Input: (), where * means any number of dimensions.
      * Target: (), same shape as the input.
      * Output: scalar. If "reduction" is "'none'", then (*), same
        shape as the input.


AdaptiveAvgPool2dclass torch.nn.AdaptiveAvgPool2d(output_size)
   Applies a 2D adaptive average pooling over an input signal composed
   of several input planes.
   The output is of size H x W, for any input size. The number of
   output features is equal to the number of input planes.
   Parameters:
      output_size (Union[int, None,
      Tuple[Optional[int], Optional[int]]])
      -- the target output size of the image of the form H x W. Can be
      a tuple (H, W) or a single H for a square image H x H. H and W
      can be either a "int", or "None" which means the size will be
      the same as that of the input.
   Shape:
      * Input: (N, C, H_{in}, W_{in}) or (C, H_{in}, W_{in}).
      * Output: (N, C, S_{0}, S_{1}) or (C, S_{0}, S_{1}), where
        S=\text{output_size}.
   -[ Examples ]-

target output size of 5x7
m = nn.AdaptiveAvgPool2d((5, 7))
input = torch.randn(1, 64, 8, 9)
output = m(input)


output = m(input)
target output size of 7x7 (square)
m = nn.AdaptiveAvgPool2d(7)
input = torch.randn(1, 64, 10, 9)
output = m(input)
target output size of 10x7
m = nn.AdaptiveAvgPool2d((None, 7))
input = torch.randn(1, 64, 10, 9)
output = m(input)


torch.quantized_max_pool1dtorch.quantized_max_pool1d(input, kernel_size, stride=[], padding=0, dilation=1, ceil_mode=False) -> Tensor
   Applies a 1D max pooling over an input quantized tensor composed of
   several input planes.
   Parameters:
      * input (Tensor) -- quantized tensor
      * kernel_size (list of python:int) -- the size of the
        sliding window
      * stride ("list of int", optional) -- the stride of the
        sliding window
      * padding ("list of int", optional) -- padding to be added
        on both sides, must be >= 0 and <= kernel_size / 2
      * dilation ("list of int", optional) -- The stride between
        elements within a sliding window, must be > 0. Default 1
      * ceil_mode (bool, optional) -- If True, will use ceil
        instead of floor to compute the output shape. Defaults to
        False.
   Returns:
      A quantized tensor with max_pool1d applied.
   Return type:
      Tensor
   Example:


Return type:
      Tensor
   Example:
      >>> qx = torch.quantize_per_tensor(torch.rand(2, 2), 1.5, 3, torch.quint8)
      >>> torch.quantized_max_pool1d(qx, [2])
      tensor([[0.0000],
              [1.5000]], size=(2, 1), dtype=torch.quint8,
          quantization_scheme=torch.per_tensor_affine, scale=1.5, zero_point=3)


torch.Tensor.rsqrt_Tensor.rsqrt_() -> Tensor
   In-place version of "rsqrt()"


torch.sorttorch.sort(input, dim=- 1, descending=False, stable=False, , out=None)
   Sorts the elements of the "input" tensor along a given dimension in
   ascending order by value.
   If "dim" is not given, the last dimension of the input is chosen.
   If "descending" is "True" then the elements are sorted in
   descending order by value.
   If "stable" is "True" then the sorting routine becomes stable,
   preserving the order of equivalent elements.
   A namedtuple of (values, indices) is returned, where the values
   are the sorted values and indices are the indices of the elements
   in the original input tensor.
   Parameters:
      * input (Tensor) -- the input tensor.
      * dim (int, optional) -- the dimension to sort along
      * descending (bool, optional) -- controls the sorting
        order (ascending or descending)
      * stable (bool, optional*) -- makes the sorting routine


stable, which guarantees that the order of equivalent elements
        is preserved.
   Keyword Arguments:
      out (tuple, optional) -- the output tuple of
      (Tensor, LongTensor) that can be optionally given to be used
      as output buffers
   Example:
      >>> x = torch.randn(3, 4)
      >>> sorted, indices = torch.sort(x)
      >>> sorted
      tensor([[-0.2162,  0.0608,  0.6719,  2.3332],
              [-0.5793,  0.0061,  0.6058,  0.9497],
              [-0.5071,  0.3343,  0.9553,  1.0960]])
      >>> indices
      tensor([[ 1,  0,  2,  3],
              [ 3,  1,  0,  2],
              [ 0,  3,  1,  2]])
      >>> sorted, indices = torch.sort(x, 0)
      >>> sorted
      tensor([[-0.5071, -0.2162,  0.6719, -0.5793],
              [ 0.0608,  0.0061,  0.9497,  0.3343],
              [ 0.6058,  0.9553,  1.0960,  2.3332]])
      >>> indices
      tensor([[ 2,  0,  0,  1],
              [ 0,  1,  1,  2],
              [ 1,  2,  2,  0]])
      >>> x = torch.tensor([0, 1] * 9)


x = torch.tensor([0, 1] * 9)
      >>> x.sort()
      torch.return_types.sort(
          values=tensor([0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 1, 1, 1, 1, 1, 1, 1]),
          indices=tensor([ 2, 16,  4,  6, 14,  8,  0, 10, 12,  9, 17, 15, 13, 11,  7,  5,  3,  1]))
      >>> x.sort(stable=True)
      torch.return_types.sort(
          values=tensor([0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 1, 1, 1, 1, 1, 1, 1]),
          indices=tensor([ 0,  2,  4,  6,  8, 10, 12, 14, 16,  1,  3,  5,  7,  9, 11, 13, 15, 17]))


torch.sparse.addmmtorch.sparse.addmm(mat, mat1, mat2, , beta=1., alpha=1.) -> Tensor
   This function does exact same thing as "torch.addmm()" in the
   forward, except that it supports backward for sparse COO matrix
   "mat1". When "mat1" is a COO tensor it must have sparse_dim = 2.
   When inputs are COO tensors, this function also supports backward
   for both inputs.
   Supports both CSR and COO storage formats.
   Note:
     This function doesn't support computing derivaties with respect
     to CSR matrices.
   Parameters:
      * mat (Tensor) -- a dense matrix to be added
      * mat1 (Tensor) -- a sparse matrix to be multiplied
      * mat2 (Tensor) -- a dense matrix to be multiplied
      * beta (Number, optional) -- multiplier for "mat"
        (\beta)
      * alpha (Number, optional*) -- multiplier for mat1 @
        mat2 (\alpha)


torch.is_nonzerotorch.is_nonzero(input)
   Returns True if the "input" is a single element tensor which is not
   equal to zero after type conversions. i.e. not equal to
   "torch.tensor([0.])" or "torch.tensor([0])" or
   "torch.tensor([False])". Throws a "RuntimeError" if "torch.numel()
   != 1" (even in case of sparse tensors).
   Parameters:
      input (Tensor) -- the input tensor.
   Examples:
      >>> torch.is_nonzero(torch.tensor([0.]))
      False
      >>> torch.is_nonzero(torch.tensor([1.5]))
      True
      >>> torch.is_nonzero(torch.tensor([False]))
      False
      >>> torch.is_nonzero(torch.tensor([3]))
      True
      >>> torch.is_nonzero(torch.tensor([1, 3, 5]))
      Traceback (most recent call last):
      ...
      RuntimeError: bool value of Tensor with more than one value is ambiguous
      >>> torch.is_nonzero(torch.tensor([]))
      Traceback (most recent call last):
      ...
      RuntimeError: bool value of Tensor with no values is ambiguous


torch.signbittorch.signbit(input, , out=None) -> Tensor
   Tests if each element of "input" has its sign bit set or not.
   Parameters:
      input (Tensor) -- the input tensor.
   Keyword Arguments:
      out (Tensor, optional*) -- the output tensor.
   Example:
      >>> a = torch.tensor([0.7, -1.2, 0., 2.3])
      >>> torch.signbit(a)
      tensor([ False, True,  False,  False])
      >>> a = torch.tensor([-0.0, 0.0])
      >>> torch.signbit(a)
      tensor([ True,  False])
   Note:
     signbit handles signed zeros, so negative zero (-0) returns True.


torch.kaiser_windowtorch.kaiser_window(window_length, periodic=True, beta=12.0, *, dtype=None, layout=torch.strided, device=None, requires_grad=False) -> Tensor
   Computes the Kaiser window with window length "window_length" and
   shape parameter "beta".
   Let I_0 be the zeroth order modified Bessel function of the first
   kind (see "torch.i0()") and "N = L - 1" if "periodic" is False and
   "L" if "periodic" is True, where "L" is the "window_length". This
   function computes:
      out_i = I_0 \left( \beta \sqrt{1 - \left( {\frac{i - N/2}{N/2}}
      \right) ^2 } \right) / I_0( \beta )
   Calling "torch.kaiser_window(L, B, periodic=True)" is equivalent to
   calling "torch.kaiser_window(L + 1, B, periodic=False)[:-1])". The
   "periodic" argument is intended as a helpful shorthand to produce a
   periodic window as input to functions like "torch.stft()".
   Note:
     If "window_length" is one, then the returned window is a single
     element tensor containing a one.
   Parameters:


Parameters:
      * window_length (int) -- length of the window.
      * periodic (bool, optional) -- If True, returns a
        periodic window suitable for use in spectral analysis. If
        False, returns a symmetric window suitable for use in filter
        design.
      * beta (float, optional) -- shape parameter for the
        window.
   Keyword Arguments:
      * dtype ("torch.dtype", optional) -- the desired data type
        of returned tensor. Default: if "None", uses a global default
        (see "torch.set_default_tensor_type()").
      * layout ("torch.layout", optional) -- the desired layout of
        returned window tensor. Only "torch.strided" (dense layout) is
        supported.
      * device ("torch.device", optional) -- the desired device of
        returned tensor. Default: if "None", uses the current device
        for the default tensor type (see
        "torch.set_default_tensor_type()"). "device" will be the CPU


for CPU tensor types and the current CUDA device for CUDA
        tensor types.
      * requires_grad (bool, optional) -- If autograd should
        record operations on the returned tensor. Default: "False".


torch.prodtorch.prod(input, , dtype=None) -> Tensor
   Returns the product of all elements in the "input" tensor.
   Parameters:
      input (Tensor) -- the input tensor.
   Keyword Arguments:
      dtype ("torch.dtype", optional) -- the desired data type of
      returned tensor. If specified, the input tensor is casted to
      "dtype" before the operation is performed. This is useful for
      preventing data type overflows. Default: None.
   Example:
      >>> a = torch.randn(1, 3)
      >>> a
      tensor([[-0.8020,  0.5428, -1.5854]])
      >>> torch.prod(a)
      tensor(0.6902)
   torch.prod(input, dim, keepdim=False, , dtype=None) -> Tensor
   Returns the product of each row of the "input" tensor in the given
   dimension "dim".
   If "keepdim" is "True", the output tensor is of the same size as
   "input" except in the dimension "dim" where it is of size 1.
   Otherwise, "dim" is squeezed (see "torch.squeeze()"), resulting in


the output tensor having 1 fewer dimension than "input".
   Parameters:
      * input (Tensor) -- the input tensor.
      * dim (int) -- the dimension to reduce.
      * keepdim (bool) -- whether the output tensor has "dim"
        retained or not.
   Keyword Arguments:
      dtype ("torch.dtype", optional) -- the desired data type of
      returned tensor. If specified, the input tensor is casted to
      "dtype" before the operation is performed. This is useful for
      preventing data type overflows. Default: None.
   Example:
      >>> a = torch.randn(4, 2)
      >>> a
      tensor([[ 0.5261, -0.3837],
              [ 1.1857, -0.2498],
              [-1.1646,  0.0705],
              [ 1.1131, -1.0629]])
      >>> torch.prod(a, 1)
      tensor([-0.2018, -0.2962, -0.0821, -1.1831])


torch.Tensor.stftTensor.stft(n_fft, hop_length=None, win_length=None, window=None, center=True, pad_mode='reflect', normalized=False, onesided=None, return_complex=None)
   See "torch.stft()"
   Warning:
     This function changed signature at version 0.4.1. Calling with
     the previous signature may cause error or return incorrect
     result.


torch.fft.hfftntorch.fft.hfftn(input, s=None, dim=None, norm=None, *, out=None) -> Tensor
   Computes the n-dimensional discrete Fourier transform of a
   Hermitian symmetric "input" signal.
   "input" is interpreted as a one-sided Hermitian signal in the time
   domain. By the Hermitian property, the Fourier transform will be
   real-valued.
   Note:
     "hfftn()"/"ihfftn()" are analogous to "rfftn()"/"irfftn()". The
     real FFT expects a real signal in the time-domain and gives
     Hermitian symmetry in the frequency-domain. The Hermitian FFT is
     the opposite; Hermitian symmetric in the time-domain and real-
     valued in the frequency-domain. For this reason, special care
     needs to be taken with the shape argument "s", in the same way as
     with "irfftn()".
   Note:
     Some input frequencies must be real-valued to satisfy the
     Hermitian property. In these cases the imaginary component will
     be ignored. For example, any imaginary component in the zero-


frequency term cannot be represented in a real output and so will
     always be ignored.
   Note:
     The correct interpretation of the Hermitian input depends on the
     length of the original data, as given by "s". This is because
     each input shape could correspond to either an odd or even length
     signal. By default, the signal is assumed to be even length and
     odd signals will not round-trip properly. It is recommended to
     always pass the signal shape "s".
   Note:
     Supports torch.half and torch.chalf on CUDA with GPU Architecture
     SM53 or greater. However it only supports powers of 2 signal
     length in every transformed dimensions. With default arguments,
     the size of last dimension should be (2^n + 1) as argument s
     defaults to even output size = 2 * (last_dim_size - 1)
   Parameters:
      * input (Tensor) -- the input tensor
      * s (Tuple[int], optional) -- Signal size in the


transformed dimensions. If given, each dimension "dim[i]" will
        either be zero-padded or trimmed to the length "s[i]" before
        computing the real FFT. If a length "-1" is specified, no
        padding is done in that dimension. Defaults to even output in
        the last dimension: "s[-1] = 2(input.size(dim[-1]) - 1)".
      * dim (Tuple[int], optional*) -- Dimensions to be
        transformed. The last dimension must be the half-Hermitian
        compressed dimension. Default: all dimensions, or the last
        "len(s)" dimensions if "s" is given.
      * norm (*str, *optional) --
        Normalization mode. For the forward transform ("hfftn()"),
        these correspond to:
        * ""forward"" - normalize by "1/n"
        * ""backward"" - no normalization
        * ""ortho"" - normalize by "1/sqrt(n)" (making the Hermitian
          FFT orthonormal)
        Where "n = prod(s)" is the logical FFT size. Calling the


backward transform ("ihfftn()") with the same normalization
        mode will apply an overall normalization of "1/n" between the
        two transforms. This is required to make "ihfftn()" the exact
        inverse.
        Default is ""backward"" (no normalization).
   Keyword Arguments:
      out (Tensor, optional) -- the output tensor.
   -[ Example ]-
   Starting from a real frequency-space signal, we can generate a
   Hermitian-symmetric time-domain signal: >>> T = torch.rand(10, 9)

t = torch.fft.ihfftn(T)
   Without specifying the output length to "hfftn()", the output will
   not round-trip properly because the input is odd-length in the last
   dimension:
torch.fft.hfftn(t).size()
   torch.Size([10, 10])
   So, it is recommended to always pass the signal shape "s".
roundtrip = torch.fft.hfftn(t, T.size())
roundtrip.size()
   torch.Size([10, 9])
torch.allclose(roundtrip, T)
   True


MultiMarginLossclass torch.nn.MultiMarginLoss(p=1, margin=1.0, weight=None, size_average=None, reduce=None, reduction='mean')
   Creates a criterion that optimizes a multi-class classification
   hinge loss (margin-based loss) between input x (a 2D mini-batch
   Tensor) and output y (which is a 1D tensor of target class
   indices, 0 \leq y \leq \text{x.size}(1)-1):
   For each mini-batch sample, the loss in terms of the 1D input x and
   scalar output y is:
      \text{loss}(x, y) = \frac{\sum_i \max(0, \text{margin} - x[y] +
      x[i])^p}{\text{x.size}(0)}
   where i \in \left{0, \; \cdots , \; \text{x.size}(0) - 1\right}
   and i \neq y.
   Optionally, you can give non-equal weighting on the classes by
   passing a 1D "weight" tensor into the constructor.
   The loss function then becomes:
      \text{loss}(x, y) = \frac{\sum_i \max(0, w[y] * (\text{margin} -
      x[y] + x[i]))^p}{\text{x.size}(0)}
   Parameters:
      * p (int, optional) -- Has a default value of 1. 1 and


2 are the only supported values.
      * margin (float, optional) -- Has a default value of
        1.
      * weight (Tensor, optional) -- a manual rescaling
        weight given to each class. If given, it has to be a Tensor of
        size C. Otherwise, it is treated as if having all ones.
      * size_average (bool, optional) -- Deprecated (see
        "reduction"). By default, the losses are averaged over each
        loss element in the batch. Note that for some losses, there
        are multiple elements per sample. If the field "size_average"
        is set to "False", the losses are instead summed for each
        minibatch. Ignored when "reduce" is "False". Default: "True"
      * reduce (bool, optional) -- Deprecated (see
        "reduction"). By default, the losses are averaged or summed
        over observations for each minibatch depending on
        "size_average". When "reduce" is "False", returns a loss per


batch element instead and ignores "size_average". Default:
        "True"
      * reduction (str, optional) -- Specifies the reduction
        to apply to the output: "'none'" | "'mean'" | "'sum'".
        "'none'": no reduction will be applied, "'mean'": the sum of
        the output will be divided by the number of elements in the
        output, "'sum'": the output will be summed. Note:
        "size_average" and "reduce" are in the process of being
        deprecated, and in the meantime, specifying either of those
        two args will override "reduction". Default: "'mean'"
   Shape:
      * Input: (N, C) or (C), where N is the batch size and C is the
        number of classes.
      * Target: (N) or (), where each value is 0 \leq
        \text{targets}[i] \leq C-1.
      * Output: scalar. If "reduction" is "'none'", then same shape as
        the target.
   Examples:
      >>> loss = nn.MultiMarginLoss()
      >>> x = torch.tensor([[0.1, 0.2, 0.4, 0.8]])


x = torch.tensor([[0.1, 0.2, 0.4, 0.8]])
      >>> y = torch.tensor([3])
      >>> # 0.25 * ((1-(0.8-0.1)) + (1-(0.8-0.2)) + (1-(0.8-0.4)))
      >>> loss(x, y)
      tensor(0.32...)


torch.slice_scattertorch.slice_scatter(input, src, dim=0, start=None, end=None, step=1) -> Tensor
   Embeds the values of the "src" tensor into "input" at the given
   dimension. This function returns a tensor with fresh storage; it
   does not create a view.
   Parameters:
      * input (Tensor) -- the input tensor.
      * src (Tensor) -- The tensor to embed into "input"
      * dim (int) -- the dimension to insert the slice into
      * start (Optional[int]) -- the start index of where
        to insert the slice
      * end (Optional[int]) -- the end index of where to
        insert the slice
      * step (int) -- the how many elements to skip in
   Example:
      >>> a = torch.zeros(8, 8)
      >>> b = torch.ones(8)
      >>> a.slice_scatter(b, start=6)
      tensor([[0., 0., 0., 0., 0., 0., 0., 0.],
              [0., 0., 0., 0., 0., 0., 0., 0.],
              [0., 0., 0., 0., 0., 0., 0., 0.],


[0., 0., 0., 0., 0., 0., 0., 0.],
              [0., 0., 0., 0., 0., 0., 0., 0.],
              [0., 0., 0., 0., 0., 0., 0., 0.],
              [0., 0., 0., 0., 0., 0., 0., 0.],
              [1., 1., 1., 1., 1., 1., 1., 1.],
              [1., 1., 1., 1., 1., 1., 1., 1.]])
      >>> b = torch.ones(2)
      >>> a.slice_scatter(b, dim=1, start=2, end=6, step=2)
      tensor([[0., 0., 1., 0., 1., 0., 0., 0.],
              [0., 0., 1., 0., 1., 0., 0., 0.],
              [0., 0., 1., 0., 1., 0., 0., 0.],
              [0., 0., 1., 0., 1., 0., 0., 0.],
              [0., 0., 1., 0., 1., 0., 0., 0.],
              [0., 0., 1., 0., 1., 0., 0., 0.],
              [0., 0., 1., 0., 1., 0., 0., 0.],
              [0., 0., 1., 0., 1., 0., 0., 0.]])


torch.dettorch.det(input) -> Tensor
   Alias for "torch.linalg.det()"


torch.linalg.matrix_powertorch.linalg.matrix_power(A, n, , out=None) -> Tensor
   Computes the n-th power of a square matrix for an integer n.
   Supports input of float, double, cfloat and cdouble dtypes. Also
   supports batches of matrices, and if "A" is a batch of matrices
   then the output has the same batch dimensions.
   If "n"= 0, it returns the identity matrix (or batch) of the same
   shape as "A". If "n" is negative, it returns the inverse of each
   matrix (if invertible) raised to the power of abs(n).
   Note:
     Consider using "torch.linalg.solve()" if possible for multiplying
     a matrix on the left by a negative power as, if "n"> 0:
        matrix_power(torch.linalg.solve(A, B), n) == matrix_power(A, -n)  @ B
     It is always preferred to use "solve()" when possible, as it is
     faster and more numerically stable than computing A^{-n}
     explicitly.
   See also:
     "torch.linalg.solve()" computes "A".inverse() @ *"B" with a


numerically stable algorithm.
   Parameters:
      * A (Tensor) -- tensor of shape (, m, m) where *** is
        zero or more batch dimensions.
      * n (int) -- the exponent.
   Keyword Arguments:
      out (Tensor, optional) -- output tensor. Ignored if
      None. Default: None.
   Raises:
      RuntimeError -- if "n"< 0* and the matrix "A" or any matrix
      in the     batch of matrices "A" is not invertible.
   Examples:
      >>> A = torch.randn(3, 3)
      >>> torch.linalg.matrix_power(A, 0)
      tensor([[1., 0., 0.],
              [0., 1., 0.],
              [0., 0., 1.]])
      >>> torch.linalg.matrix_power(A, 3)
      tensor([[ 1.0756,  0.4980,  0.0100],
              [-1.6617,  1.4994, -1.9980],
              [-0.4509,  0.2731,  0.8001]])
      >>> torch.linalg.matrix_power(A.expand(2, -1, -1), -2)
      tensor([[[ 0.2640,  0.4571, -0.5511],
              [-1.0163,  0.3491, -1.5292],
              [-0.4899,  0.0822,  0.2773]],


[-0.4899,  0.0822,  0.2773]],
              [[ 0.2640,  0.4571, -0.5511],
              [-1.0163,  0.3491, -1.5292],
              [-0.4899,  0.0822,  0.2773]]])


Adadeltaclass torch.optim.Adadelta(params, lr=1.0, rho=0.9, eps=1e-06, weight_decay=0, foreach=None, *, maximize=False, differentiable=False)
   Implements Adadelta algorithm.
      \begin{aligned}      &\rule{110mm}{0.4pt}
      \      &\textbf{input}      : \gamma \text{ (lr)}, \: \theta_0
      \text{ (params)},          \: f(\theta) \text{ (objective)}, \:
      \rho \text{ (decay)},          \: \lambda \text{ (weight decay)}
      \      &\textbf{initialize} :  v_0  \leftarrow 0 \: \text{
      (square avg)},          \: u_0 \leftarrow 0 \: \text{
      (accumulate variables)}                     \[-1.ex]
      &\rule{110mm}{0.4pt}
      \      &\textbf{for} \: t=1 \: \textbf{to} \: \ldots \:
      \textbf{do}                         \      &\hspace{5mm}g_t
      \leftarrow   \nabla_{\theta} f_t (\theta_{t-1})           \
      &\hspace{5mm}if \: \lambda \neq 0
      \      &\hspace{10mm} g_t \leftarrow g_t + \lambda
      \theta_{t-1}                            \      &\hspace{5mm}


v_t      \leftarrow v_{t-1} \rho + g^2_t (1 - \rho)
      \      &\hspace{5mm}\Delta x_t    \leftarrow
      \frac{\sqrt{u_{t-1} +          \epsilon }}{ \sqrt{v_t +
      \epsilon}  }g_t \hspace{21mm}                           \
      &\hspace{5mm} u_t  \leftarrow   u_{t-1}  \rho +           \Delta
      x^2_t  (1 - \rho)
      \      &\hspace{5mm}\theta_t      \leftarrow   \theta_{t-1} -
      \gamma  \Delta x_t            \      &\rule{110mm}{0.4pt}
      \[-1.ex]      &\bf{return} \:  \theta_t
      \[-1.ex]      &\rule{110mm}{0.4pt}
      \[-1.ex] \end{aligned}
   For further details regarding the algorithm we refer to ADADELTA:
   An Adaptive Learning Rate Method.
   Parameters:
      * params (iterable) -- iterable of parameters to optimize
        or dicts defining parameter groups
      * rho (float, optional) -- coefficient used for
        computing a running average of squared gradients (default:
        0.9)
      * eps (float, optional) -- term added to the


denominator to improve numerical stability (default: 1e-6)
      * lr (float, optional) -- coefficient that scale delta
        before it is applied to the parameters (default: 1.0)
      * weight_decay (float, optional) -- weight decay (L2
        penalty) (default: 0)
      * foreach (bool, optional) -- whether foreach
        implementation of optimizer is used. If unspecified by the
        user (so foreach is None), we will try to use foreach over the
        for-loop implementation on CUDA, since it is usually
        significantly more performant. (default: None)
      * maximize (bool, optional) -- maximize the params
        based on the objective, instead of minimizing (default: False)
      * differentiable (bool, optional) -- whether autograd
        should occur through the optimizer step in training.
        Otherwise, the step() function runs in a torch.no_grad()
        context. Setting to True can impair performance, so leave it


False if you don't intend to run autograd through this
        instance (default: False)
   add_param_group(param_group)
      Add a param group to the "Optimizer" s param_groups.
      This can be useful when fine tuning a pre-trained network as
      frozen layers can be made trainable and added to the "Optimizer"
      as training progresses.
      Parameters:
         param_group (dict) -- Specifies what Tensors should be
         optimized along with group specific optimization options.
   load_state_dict(state_dict)
      Loads the optimizer state.
      Parameters:
         state_dict (dict) -- optimizer state. Should be an
         object returned from a call to "state_dict()".
   register_step_post_hook(hook)
      Register an optimizer step post hook which will be called after
      optimizer step. It should have the following signature:
         hook(optimizer, args, kwargs) -> None
      The "optimizer" argument is the optimizer instance being used.
      Parameters:


Parameters:
         hook (Callable) -- The user defined hook to be
         registered.
      Returns:
         a handle that can be used to remove the added hook by calling
         "handle.remove()"
      Return type:
         "torch.utils.hooks.RemoveableHandle"
   register_step_pre_hook(hook)
      Register an optimizer step pre hook which will be called before
      optimizer step. It should have the following signature:
         hook(optimizer, args, kwargs) -> None or modified args and kwargs
      The "optimizer" argument is the optimizer instance being used.
      If args and kwargs are modified by the pre-hook, then the
      transformed values are returned as a tuple containing the
      new_args and new_kwargs.
      Parameters:
         hook (Callable) -- The user defined hook to be
         registered.
      Returns:
         a handle that can be used to remove the added hook by calling
         "handle.remove()"
      Return type:
         "torch.utils.hooks.RemoveableHandle"


"torch.utils.hooks.RemoveableHandle"
   state_dict()
      Returns the state of the optimizer as a "dict".
      It contains two entries:
      * state - a dict holding current optimization state. Its content
           differs between optimizer classes.
      * param_groups - a list containing all parameter groups where
        each
           parameter group is a dict
   zero_grad(set_to_none=False)
      Sets the gradients of all optimized "torch.Tensor" s to zero.
      Parameters:
         set_to_none (bool) -- instead of setting to zero, set
         the grads to None. This will in general have lower memory
         footprint, and can modestly improve performance. However, it
         changes certain behaviors. For example: 1. When the user
         tries to access a gradient and perform manual ops on it, a
         None attribute or a Tensor full of 0s will behave
         differently. 2. If the user requests
         "zero_grad(set_to_none=True)" followed by a backward pass,


".grad"s are guaranteed to be None for params that did not
         receive a gradient. 3. "torch.optim" optimizers have a
         different behavior if the gradient is 0 or None (in one case
         it does the step with a gradient of 0 and in the other it
         skips the step altogether).


torch.autograd.function.FunctionCtx.mark_non_differentiableFunctionCtx.mark_non_differentiable(args)
   Marks outputs as non-differentiable.
   This should be called at most once, only from inside the
   "forward()" method, and all arguments should be tensor outputs.*
   This will mark outputs as not requiring gradients, increasing the
   efficiency of backward computation. You still need to accept a
   gradient for each output in "backward()", but it's always going to
   be a zero tensor with the same shape as the shape of a
   corresponding output.
   This is used e.g. for indices returned from a sort. See example::
      >>> class Func(Function):
      >>>     @staticmethod
      >>>     def forward(ctx, x):
      >>>         sorted, idx = x.sort()
      >>>         ctx.mark_non_differentiable(idx)
      >>>         ctx.save_for_backward(x, idx)
      >>>         return sorted, idx
      >>>
      >>>     @staticmethod


  >>>     @staticmethod
  >>>     @once_differentiable
  >>>     def backward(ctx, g1, g2):  # still need to accept g2
  >>>         x, idx = ctx.saved_tensors
  >>>         grad_input = torch.zeros_like(x)
  >>>         grad_input.index_add_(0, idx, g1)
  >>>         return grad_input


torch.sparse.mmtorch.sparse.mm()
      Performs a matrix multiplication of the sparse matrix "mat1" and
      the (sparse or strided) matrix "mat2". Similar to "torch.mm()",
      if "mat1" is a (n \times m) tensor, "mat2" is a (m \times p)
      tensor, out will be a (n \times p) tensor. When "mat1" is a COO
      tensor it must have sparse_dim = 2. When inputs are COO
      tensors, this function also supports backward for both inputs.
      Supports both CSR and COO storage formats.
   Note:
     This function doesn't support computing derivaties with respect
     to CSR matrices.
     Args:
        mat1 (Tensor): the first sparse matrix to be multiplied mat2
        (Tensor): the second matrix to be multiplied, which could be
        sparse or dense
     Shape:
        The format of the output tensor of this function follows: -
        sparse x sparse -> sparse - sparse x dense -> dense
     Example:
        >>> a = torch.randn(2, 3).to_sparse().requires_grad_(True)
        >>> a


a
        tensor(indices=tensor([[0, 0, 0, 1, 1, 1],
                               [0, 1, 2, 0, 1, 2]]),
               values=tensor([ 1.5901,  0.0183, -0.6146,  1.8061, -0.0112,  0.6302]),
               size=(2, 3), nnz=6, layout=torch.sparse_coo, requires_grad=True)
        >>> b = torch.randn(3, 2, requires_grad=True)
        >>> b
        tensor([[-0.6479,  0.7874],
                [-1.2056,  0.5641],
                [-1.1716, -0.9923]], requires_grad=True)
        >>> y = torch.sparse.mm(a, b)
        >>> y
        tensor([[-0.3323,  1.8723],
                [-1.8951,  0.7904]], grad_fn=)
        >>> y.sum().backward()
        >>> a.grad
        tensor(indices=tensor([[0, 0, 0, 1, 1, 1],
                               [0, 1, 2, 0, 1, 2]]),
               values=tensor([ 0.1394, -0.6415, -2.1639,  0.1394, -0.6415, -2.1639]),
               size=(2, 3), nnz=6, layout=torch.sparse_coo)


torch.Tensor.get_deviceTensor.get_device() -> Device ordinal (Integer)
   For CUDA tensors, this function returns the device ordinal of the
   GPU on which the tensor resides. For CPU tensors, this function
   returns -1.
   Example:
      >>> x = torch.randn(3, 4, 5, device='cuda:0')
      >>> x.get_device()
      0
      >>> x.cpu().get_device()
      -1


torch.cuda.set_per_process_memory_fractiontorch.cuda.set_per_process_memory_fraction(fraction, device=None)
   Set memory fraction for a process. The fraction is used to limit an
   caching allocator to allocated memory on a CUDA device. The allowed
   value equals the total visible memory multiplied fraction. If
   trying to allocate more than the allowed value in a process, will
   raise an out of memory error in allocator.
   Parameters:
      * fraction (float) -- Range: 0~1. Allowed memory equals
        total_memory * fraction.
      * device (torch.device or int, optional) --
        selected device. If it is "None" the default CUDA device is
        used.
   Note:
     In general, the total available free memory is less than the
     total capacity.


ConvBn2dclass torch.ao.nn.intrinsic.ConvBn2d(conv, bn)
   This is a sequential container which calls the Conv 2d and Batch
   Norm 2d modules. During quantization this will be replaced with the
   corresponding fused module.


torch.func.gradtorch.func.grad(func, argnums=0, has_aux=False)
   "grad" operator helps computing gradients of "func" with respect to
   the input(s) specified by "argnums". This operator can be nested to
   compute higher-order gradients.
   Parameters:
      * func (Callable) -- A Python function that takes one or
        more arguments. Must return a single-element Tensor. If
        specified "has_aux" equals "True", function can return a tuple
        of single-element Tensor and other auxiliary objects:
        "(output, aux)".
      * argnums (int or Tuple[int]) -- Specifies
        arguments to compute gradients with respect to. "argnums" can
        be single integer or tuple of integers. Default: 0.
      * has_aux (bool) -- Flag indicating that "func" returns a
        tensor and other auxiliary objects: "(output, aux)". Default:
        False.
   Returns:
      Function to compute gradients with respect to its inputs. By


default, the output of the function is the gradient tensor(s)
      with respect to the first argument. If specified "has_aux"
      equals "True", tuple of gradients and output auxiliary objects
      is returned. If "argnums" is a tuple of integers, a tuple of
      output gradients with respect to each "argnums" value is
      returned.
   Return type:
      Callable
   Example of using "grad":

from torch.func import grad
x = torch.randn([])
cos_x = grad(lambda x: torch.sin(x))(x)
assert torch.allclose(cos_x, x.cos())
Second-order gradients
neg_sin_x = grad(grad(lambda x: torch.sin(x)))(x)
assert torch.allclose(neg_sin_x, -x.sin())
   When composed with "vmap", "grad" can be used to compute per-
   sample-gradients:
from torch.func import grad, vmap
batch_size, feature_size = 3, 5
def model(weights, feature_vec):
    # Very simple linear model with activation
    assert feature_vec.dim() == 1


assert feature_vec.dim() == 1
return feature_vec.dot(weights).relu()

def compute_loss(weights, example, target):
    y = model(weights, example)
    return ((y - target) ** 2).mean()  # MSELoss
weights = torch.randn(feature_size, requires_grad=True)
examples = torch.randn(batch_size, feature_size)
targets = torch.randn(batch_size)
inputs = (weights, examples, targets)
grad_weight_per_example = vmap(grad(compute_loss), in_dims=(None, 0, 0))(*inputs)
   Example of using "grad" with "has_aux" and "argnums":
from torch.func import grad
def my_loss_func(y, y_pred):
   loss_per_sample = (0.5 * y_pred - y) ** 2
   loss = loss_per_sample.mean()
   return loss, (y_pred, loss_per_sample)
fn = grad(my_loss_func, argnums=(0, 1), has_aux=True)
y_true = torch.rand(4)
y_preds = torch.rand(4, requires_grad=True)
out = fn(y_true, y_preds)


out = fn(y_true, y_preds)
> output is ((grads w.r.t y_true, grads w.r.t y_preds), (y_pred, loss_per_sample))
Note:
     Using PyTorch "torch.no_grad" together with "grad".Case 1: Using
     "torch.no_grad" inside a function:
     >>> def f(x):
     >>>     with torch.no_grad():
     >>>         c = x ** 2
     >>>     return x - c
     In this case, "grad(f)(x)" will respect the inner
     "torch.no_grad".Case 2: Using "grad" inside "torch.no_grad"
     context manager:
     >>> with torch.no_grad():
     >>>     grad(f)(x)
     In this case, "grad" will respect the inner "torch.no_grad", but
     not the outer one. This is because "grad" is a "function
     transform": its result should not depend on the result of a
     context manager outside of "f".


torch.sparse_coo_tensortorch.sparse_coo_tensor(indices, values, size=None, , dtype=None, device=None, requires_grad=False, check_invariants=None) -> Tensor
   Constructs a sparse tensor in COO(rdinate) format with specified
   values at the given "indices".
   Note:
     This function returns an uncoalesced tensor.
   Note:
     If the "device" argument is not specified the device of the given
     "values" and indices tensor(s) must match. If, however, the
     argument is specified the input Tensors will be converted to the
     given device and in turn determine the device of the constructed
     sparse tensor.
   Parameters:
      * indices (array_like*) -- Initial data for the tensor. Can
        be a list, tuple, NumPy "ndarray", scalar, and other types.
        Will be cast to a "torch.LongTensor" internally. The indices
        are the coordinates of the non-zero values in the matrix, and
        thus should be two-dimensional where the first dimension is


the number of tensor dimensions and the second dimension is
        the number of non-zero values.
      * values (array_like) -- Initial values for the tensor.
        Can be a list, tuple, NumPy "ndarray", scalar, and other
        types.
      * size (list, tuple, or "torch.Size", optional) -- Size of
        the sparse tensor. If not provided the size will be inferred
        as the minimum size big enough to hold all non-zero elements.
   Keyword Arguments:
      * dtype ("torch.dtype", optional) -- the desired data type
        of returned tensor. Default: if None, infers data type from
        "values".
      * device ("torch.device", optional) -- the desired device of
        returned tensor. Default: if None, uses the current device for
        the default tensor type (see
        "torch.set_default_tensor_type()"). "device" will be the CPU
        for CPU tensor types and the current CUDA device for CUDA
        tensor types.


tensor types.
      * requires_grad (bool, optional) -- If autograd should
        record operations on the returned tensor. Default: "False".
      * check_invariants (bool, optional) -- If sparse
        tensor invariants are checked. Default: as returned by
        "torch.sparse.check_sparse_tensor_invariants.is_enabled()",
        initially False.
   Example:
      >>> i = torch.tensor([[0, 1, 1],
      ...                   [2, 0, 2]])
      >>> v = torch.tensor([3, 4, 5], dtype=torch.float32)
      >>> torch.sparse_coo_tensor(i, v, [2, 4])
      tensor(indices=tensor([[0, 1, 1],
                             [2, 0, 2]]),
             values=tensor([3., 4., 5.]),
             size=(2, 4), nnz=3, layout=torch.sparse_coo)
      >>> torch.sparse_coo_tensor(i, v)  # Shape inference
      tensor(indices=tensor([[0, 1, 1],
                             [2, 0, 2]]),
             values=tensor([3., 4., 5.]),
             size=(2, 3), nnz=3, layout=torch.sparse_coo)


torch.sparse_coo_tensor(i, v, [2, 4],
      ...                         dtype=torch.float64,
      ...                         device=torch.device('cuda:0'))
      tensor(indices=tensor([[0, 1, 1],
                             [2, 0, 2]]),
             values=tensor([3., 4., 5.]),
             device='cuda:0', size=(2, 4), nnz=3, dtype=torch.float64,
             layout=torch.sparse_coo)
      # Create an empty sparse tensor with the following invariants:
      #   1. sparse_dim + dense_dim = len(SparseTensor.shape)
      #   2. SparseTensor._indices().shape = (sparse_dim, nnz)
      #   3. SparseTensor._values().shape = (nnz, SparseTensor.shape[sparse_dim:])
      #
      # For instance, to create an empty sparse tensor with nnz = 0, dense_dim = 0 and
      # sparse_dim = 1 (hence indices is a 2D tensor of shape = (1, 0))
      >>> S = torch.sparse_coo_tensor(torch.empty([1, 0]), [], [1])
      tensor(indices=tensor([], size=(1, 0)),
             values=tensor([], size=(0,)),


values=tensor([], size=(0,)),
             size=(1,), nnz=0, layout=torch.sparse_coo)
      # and to create an empty sparse tensor with nnz = 0, dense_dim = 1 and
      # sparse_dim = 1
      >>> S = torch.sparse_coo_tensor(torch.empty([1, 0]), torch.empty([0, 2]), [1, 2])
      tensor(indices=tensor([], size=(1, 0)),
             values=tensor([], size=(0, 2)),
             size=(1, 2), nnz=0, layout=torch.sparse_coo)


torch._foreach_roundtorch._foreach_round(self: List[Tensor]) -> List[Tensor]
   Apply "torch.round()" to each Tensor of the input list.


LeakyReLUclass torch.nn.LeakyReLU(negative_slope=0.01, inplace=False)
   Applies the element-wise function:
      \text{LeakyReLU}(x) = \max(0, x) + \text{negative_slope} *
      \min(0, x)
   or
      \text{LeakyReLU}(x) = \begin{cases} x, & \text{ if } x \geq 0 \
      \text{negative_slope} \times x, & \text{ otherwise }
      \end{cases}
   Parameters:
      * negative_slope (float) -- Controls the angle of the
        negative slope. Default: 1e-2
      * inplace (bool) -- can optionally do the operation in-
        place. Default: "False"
   Shape:
      * Input: () where *** means, any number of additional
        dimensions
      * Output: (), same shape as the input
   [image]
   Examples:
      >>> m = nn.LeakyReLU(0.1)
      >>> input = torch.randn(2)
      >>> output = m(input)


torch.Tensor.isrealTensor.isreal() -> Tensor
   See "torch.isreal()"


torch.nn.functional.leaky_relutorch.nn.functional.leaky_relu(input, negative_slope=0.01, inplace=False) -> Tensor
   Applies element-wise, \text{LeakyReLU}(x) = \max(0, x) +
   \text{negative_slope} * \min(0, x)
   See "LeakyReLU" for more details.
   Return type:
      Tensor


torch.Tensor.fill_diagonal_Tensor.fill_diagonal_(fill_value, wrap=False) -> Tensor
   Fill the main diagonal of a tensor that has at least 2-dimensions.
   When dims>2, all dimensions of input must be of equal length. This
   function modifies the input tensor in-place, and returns the input
   tensor.
   Parameters:
      * fill_value (Scalar) -- the fill value
      * wrap (bool) -- the diagonal 'wrapped' after N columns
        for tall matrices.
   Example:
      >>> a = torch.zeros(3, 3)
      >>> a.fill_diagonal_(5)
      tensor([[5., 0., 0.],
              [0., 5., 0.],
              [0., 0., 5.]])
      >>> b = torch.zeros(7, 3)
      >>> b.fill_diagonal_(5)
      tensor([[5., 0., 0.],
              [0., 5., 0.],
              [0., 0., 5.],
              [0., 0., 0.],
              [0., 0., 0.],
              [0., 0., 0.],
              [0., 0., 0.]])
      >>> c = torch.zeros(7, 3)
      >>> c.fill_diagonal_(5, wrap=True)
      tensor([[5., 0., 0.],


tensor([[5., 0., 0.],
              [0., 5., 0.],
              [0., 0., 5.],
              [0., 0., 0.],
              [5., 0., 0.],
              [0., 5., 0.],
              [0., 0., 5.]])


torch.Tensor.acosTensor.acos() -> Tensor
   See "torch.acos()"


ConstantLRclass torch.optim.lr_scheduler.ConstantLR(optimizer, factor=0.3333333333333333, total_iters=5, last_epoch=- 1, verbose=False)
   Decays the learning rate of each parameter group by a small
   constant factor until the number of epoch reaches a pre-defined
   milestone: total_iters. Notice that such decay can happen
   simultaneously with other changes to the learning rate from outside
   this scheduler. When last_epoch=-1, sets initial lr as lr.
   Parameters:
      * optimizer (Optimizer) -- Wrapped optimizer.
      * factor (float) -- The number we multiply learning rate
        until the milestone. Default: 1./3.
      * total_iters (int) -- The number of steps that the
        scheduler decays the learning rate. Default: 5.
      * last_epoch (int) -- The index of the last epoch.
        Default: -1.
      * verbose (bool) -- If "True", prints a message to stdout
        for each update. Default: "False".
   -[ Example ]-


-[ Example ]-

Assuming optimizer uses lr = 0.05 for all groups
lr = 0.025   if epoch == 0
lr = 0.025   if epoch == 1
lr = 0.025   if epoch == 2
lr = 0.025   if epoch == 3
lr = 0.05    if epoch >= 4
scheduler = ConstantLR(self.opt, factor=0.5, total_iters=4)
for epoch in range(100):
    train(...)
    validate(...)
    scheduler.step()
   get_last_lr()
      Return last computed learning rate by current scheduler.
   load_state_dict(state_dict)
      Loads the schedulers state.
      Parameters:
         state_dict (dict) -- scheduler state. Should be an
         object returned from a call to "state_dict()".
   print_lr(is_verbose, group, lr, epoch=None)
      Display the current learning rate.
   state_dict()
      Returns the state of the scheduler as a "dict".
      It contains an entry for every variable in self.dict which
      is not the optimizer.


BNReLU2dclass torch.ao.nn.intrinsic.quantized.BNReLU2d(num_features, eps=1e-05, momentum=0.1, device=None, dtype=None)
   A BNReLU2d module is a fused module of BatchNorm2d and ReLU
   We adopt the same interface as "torch.ao.nn.quantized.BatchNorm2d".
   Variables:
      torch.ao.nn.quantized.BatchNorm2d (Same as) --


freeze_bn_statsclass torch.ao.nn.intrinsic.qat.freeze_bn_stats(mod)


ConvertCustomConfigclass torch.ao.quantization.fx.custom_config.ConvertCustomConfig
   Custom configuration for "convert_fx()".
   Example usage:
      convert_custom_config = ConvertCustomConfig()             .set_observed_to_quantized_mapping(ObservedCustomModule, QuantizedCustomModule)             .set_preserved_attributes(["attr1", "attr2"])
   classmethod from_dict(convert_custom_config_dict)
      Create a "ConvertCustomConfig" from a dictionary with the
      following items:
         "observed_to_quantized_custom_module_class": a nested
         dictionary mapping from quantization mode to an inner mapping
         from observed module classes to quantized module classes,
         e.g.:: { "static": {FloatCustomModule: ObservedCustomModule},
         "dynamic": {FloatCustomModule: ObservedCustomModule},
         "weight_only": {FloatCustomModule: ObservedCustomModule} }
         "preserved_attributes": a list of attributes that persist


even if they are not used in "forward"
      This function is primarily for backward compatibility and may be
      removed in the future.
      Return type:
         ConvertCustomConfig
   set_observed_to_quantized_mapping(observed_class, quantized_class, quant_type=QuantType.STATIC)
      Set the mapping from a custom observed module class to a custom
      quantized module class.
      The quantized module class must have a "from_observed" class
      method that converts the observed module class to the quantized
      module class.
      Return type:
         ConvertCustomConfig
   set_preserved_attributes(attributes)
      Set the names of the attributes that will persist in the graph
      module even if they are not used in the model's "forward"
      method.
      Return type:
         ConvertCustomConfig
   to_dict()
      Convert this "ConvertCustomConfig" to a dictionary with the
      items described in "from_dict()".
      Return type:
         Dict[str, Any]


torch.Tensor.angleTensor.angle() -> Tensor
   See "torch.angle()"


torch.set_default_tensor_typetorch.set_default_tensor_type(t)
   Sets the default "torch.Tensor" type to floating point tensor type
   "t". This type will also be used as default floating point type for
   type inference in "torch.tensor()".
   The default floating point tensor type is initially
   "torch.FloatTensor".
   Parameters:
      t (type or string) -- the floating point tensor type
      or its name
   Example:
      >>> torch.tensor([1.2, 3]).dtype    # initial default for floating point is torch.float32
      torch.float32
      >>> torch.set_default_tensor_type(torch.DoubleTensor)
      >>> torch.tensor([1.2, 3]).dtype    # a new floating point tensor
      torch.float64


PairwiseDistanceclass torch.nn.PairwiseDistance(p=2.0, eps=1e-06, keepdim=False)
   Computes the pairwise distance between input vectors, or between
   columns of input matrices.
   Distances are computed using "p"-norm, with constant "eps" added to
   avoid division by zero if "p" is negative, i.e.:
      \mathrm{dist}\left(x, y\right) = \left\Vert x-y + \epsilon e
      \right\Vert_p,
   where e is the vector of ones and the "p"-norm is given by.
      \Vert x \Vert p = \left( \sum^n  \vert x_i \vert ^ p
      \right) ^ {1/p}.
   Parameters:
      * p (real, optional) -- the norm degree. Can be
        negative. Default: 2
      * eps (float, optional) -- Small value to avoid
        division by zero. Default: 1e-6
      * keepdim (bool, optional) -- Determines whether or
        not to keep the vector dimension. Default: False
   Shape:
      * Input1: (N, D) or (D) where N = batch dimension and D =
        vector dimension


vector dimension*
      * Input2: (N, D) or (D), same shape as the Input1
      * Output: (N) or () based on input dimension. If "keepdim" is
        "True", then (N, 1) or (1) based on input dimension.
   Examples::
      >>> pdist = nn.PairwiseDistance(p=2)
      >>> input1 = torch.randn(100, 128)
      >>> input2 = torch.randn(100, 128)
      >>> output = pdist(input1, input2)


torch.fft.ifftntorch.fft.ifftn(input, s=None, dim=None, norm=None, , out=None) -> Tensor
   Computes the N dimensional inverse discrete Fourier transform of
   "input".
   Note:
     Supports torch.half and torch.chalf on CUDA with GPU Architecture
     SM53 or greater. However it only supports powers of 2 signal
     length in every transformed dimensions.
   Parameters:
      * input (Tensor) -- the input tensor
      * s (Tuple[int], optional*) -- Signal size in the
        transformed dimensions. If given, each dimension "dim[i]" will
        either be zero-padded or trimmed to the length "s[i]" before
        computing the IFFT. If a length "-1" is specified, no padding
        is done in that dimension. Default: "s = [input.size(d) for d
        in dim]"
      * dim (*Tuple[int], optional*) -- Dimensions to be
        transformed. Default: all dimensions, or the last "len(s)"
        dimensions if "s" is given.


dimensions if "s" is given.
      * norm (str, optional) --
        Normalization mode. For the backward transform ("ifftn()"),
        these correspond to:
        * ""forward"" - no normalization
        * ""backward"" - normalize by "1/n"
        * ""ortho"" - normalize by "1/sqrt(n)" (making the IFFT
          orthonormal)
        Where "n = prod(s)" is the logical IFFT size. Calling the
        forward transform ("fftn()") with the same normalization mode
        will apply an overall normalization of "1/n" between the two
        transforms. This is required to make "ifftn()" the exact
        inverse.
        Default is ""backward"" (normalize by "1/n").
   Keyword Arguments:
      out (Tensor, optional) -- the output tensor.
   -[ Example ]-

x = torch.rand(10, 10, dtype=torch.complex64)
ifftn = torch.fft.ifftn(x)
   The discrete Fourier transform is separable, so "ifftn()" here is
   equivalent to two one-dimensional "ifft()" calls:


two_iffts = torch.fft.ifft(torch.fft.ifft(x, dim=0), dim=1)
torch.testing.assert_close(ifftn, two_iffts, check_stride=False)


torch.Tensor.ldexpTensor.ldexp(other) -> Tensor
   See "torch.ldexp()"


torch.nn.functional.lp_pool1dtorch.nn.functional.lp_pool1d(input, norm_type, kernel_size, stride=None, ceil_mode=False)
   Applies a 1D power-average pooling over an input signal composed of
   several input planes. If the sum of all inputs to the power of p
   is zero, the gradient is set to zero as well.
   See "LPPool1d" for details.
   Return type:
      Tensor


torch.frexptorch.frexp(input, , out=None) -> (Tensor mantissa, Tensor exponent)
   Decomposes "input" into mantissa and exponent tensors such that
   \text{input} = \text{mantissa} \times 2^{\text{exponent}}.
   The range of mantissa is the open interval (-1, 1).
   Supports float inputs.
   Parameters:
      input (Tensor) -- the input tensor
   Keyword Arguments:
      out (tuple, optional*) -- the output tensors
   Example:
      >>> x = torch.arange(9.)
      >>> mantissa, exponent = torch.frexp(x)
      >>> mantissa
      tensor([0.0000, 0.5000, 0.5000, 0.7500, 0.5000, 0.6250, 0.7500, 0.8750, 0.5000])
      >>> exponent
      tensor([0, 1, 2, 2, 3, 3, 3, 3, 4], dtype=torch.int32)
      >>> torch.ldexp(mantissa, exponent)
      tensor([0., 1., 2., 3., 4., 5., 6., 7., 8.])


torch.vsplittorch.vsplit(input, indices_or_sections) -> List of Tensors
   Splits "input", a tensor with two or more dimensions, into multiple
   tensors vertically according to "indices_or_sections". Each split
   is a view of "input".
   This is equivalent to calling torch.tensor_split(input,
   indices_or_sections, dim=0) (the split dimension is 0), except that
   if "indices_or_sections" is an integer it must evenly divide the
   split dimension or a runtime error will be thrown.
   This function is based on NumPy's "numpy.vsplit()".
   Parameters:
      * input (Tensor) -- tensor to split.
      * indices_or_sections (int or list or tuple of
        ints) -- See argument in "torch.tensor_split()".
   Example::
      >>> t = torch.arange(16.0).reshape(4,4)
      >>> t
      tensor([[ 0.,  1.,  2.,  3.],
              [ 4.,  5.,  6.,  7.],
              [ 8.,  9., 10., 11.],
              [12., 13., 14., 15.]])
      >>> torch.vsplit(t, 2)
      (tensor([[0., 1., 2., 3.],


(tensor([[0., 1., 2., 3.],
               [4., 5., 6., 7.]]),
       tensor([[ 8.,  9., 10., 11.],
               [12., 13., 14., 15.]]))
      >>> torch.vsplit(t, [3, 6])
      (tensor([[ 0.,  1.,  2.,  3.],
               [ 4.,  5.,  6.,  7.],
               [ 8.,  9., 10., 11.]]),
       tensor([[12., 13., 14., 15.]]),
       tensor([], size=(0, 4)))


no_gradclass torch.no_grad
   Context-manager that disabled gradient calculation.
   Disabling gradient calculation is useful for inference, when you
   are sure that you will not call "Tensor.backward()". It will reduce
   memory consumption for computations that would otherwise have
   requires_grad=True.
   In this mode, the result of every computation will have
   requires_grad=False, even when the inputs have
   requires_grad=True.
   This context manager is thread local; it will not affect
   computation in other threads.
   Also functions as a decorator. (Make sure to instantiate with
   parenthesis.)
   Note:
     No-grad is one of several mechanisms that can enable or disable
     gradients locally see Locally disabling gradient computation for
     more information on how they compare.
   Note:
     This API does not apply to forward-mode AD. If you want to
     disable forward AD for a computation, you can unpack your dual
     tensors.
   Example::


tensors.
   Example::
      >>> x = torch.tensor([1.], requires_grad=True)
      >>> with torch.no_grad():
      ...     y = x * 2
      >>> y.requires_grad
      False
      >>> @torch.no_grad()
      ... def doubler(x):
      ...     return x * 2
      >>> z = doubler(x)
      >>> z.requires_grad
      False


ConvReLU1dclass torch.ao.nn.intrinsic.quantized.ConvReLU1d(in_channels, out_channels, kernel_size, stride=1, padding=0, dilation=1, groups=1, bias=True, padding_mode='zeros', device=None, dtype=None)
   A ConvReLU1d module is a fused module of Conv1d and ReLU
   We adopt the same interface as "torch.ao.nn.quantized.Conv1d".
   Variables:
      torch.ao.nn.quantized.Conv1d (Same as) --


torch.nn.functional.rrelutorch.nn.functional.rrelu(input, lower=1. / 8, upper=1. / 3, training=False, inplace=False) -> Tensor
   Randomized leaky ReLU.
   See "RReLU" for more details.
   Return type:
      Tensor


InstanceNorm1dclass torch.ao.nn.quantized.InstanceNorm1d(num_features, weight, bias, scale, zero_point, eps=1e-05, momentum=0.1, affine=False, track_running_stats=False, device=None, dtype=None)
   This is the quantized version of "InstanceNorm1d".
   Additional args:
      * scale - quantization scale of the output, type: double.
      * zero_point - quantization zero point of the output, type:
        long.


torch.cuda.max_memory_reservedtorch.cuda.max_memory_reserved(device=None)
   Returns the maximum GPU memory managed by the caching allocator in
   bytes for a given device.
   By default, this returns the peak cached memory since the beginning
   of this program. "reset_peak_memory_stats()" can be used to reset
   the starting point in tracking this metric. For example, these two
   functions can measure the peak cached memory amount of each
   iteration in a training loop.
   Parameters:
      device (torch.device or int, optional) -- selected
      device. Returns statistic for the current device, given by
      "current_device()", if "device" is "None" (default).
   Return type:
      int
   Note:
     See Memory management for more details about GPU memory
     management.


torch.nn.functional.affine_gridtorch.nn.functional.affine_grid(theta, size, align_corners=None)
   Generates a 2D or 3D flow field (sampling grid), given a batch of
   affine matrices "theta".
   Note:
     This function is often used in conjunction with "grid_sample()"
     to build Spatial Transformer Networks .
   Parameters:
      * theta (Tensor) -- input batch of affine matrices with
        shape (N \times 2 \times 3) for 2D or (N \times 3 \times 4)
        for 3D
      * size (torch.Size) -- the target output image size. (N
        \times C \times H \times W for 2D or N \times C \times D
        \times H \times W for 3D) Example: torch.Size((32, 3, 24, 24))
      * align_corners (bool, optional) -- if "True",
        consider "-1" and "1" to refer to the centers of the corner
        pixels rather than the image corners. Refer to "grid_sample()"
        for a more complete description. A grid generated by


"affine_grid()" should be passed to "grid_sample()" with the
        same setting for this option. Default: "False"
   Returns:
      output Tensor of size (N \times H \times W \times 2)
   Return type:
      output (Tensor)
   Warning:
     When "align_corners = True", the grid positions depend on the
     pixel size relative to the input image size, and so the locations
     sampled by "grid_sample()" will differ for the same input given
     at different resolutions (that is, after being upsampled or
     downsampled). The default behavior up to version 1.2.0 was
     "align_corners = True". Since then, the default behavior has been
     changed to "align_corners = False", in order to bring it in line
     with the default for "interpolate()".
   Warning:
     When "align_corners = True", 2D affine transforms on 1D data and
     3D affine transforms on 2D data (that is, when one of the spatial
     dimensions has unit size) are ill-defined, and not an intended


use case. This is not a problem when "align_corners = False". Up
     to version 1.2.0, all grid points along a unit dimension were
     considered arbitrarily to be at "-1". From version 1.3.0, under
     "align_corners = True" all grid points along a unit dimension are
     considered to be at "0" (the center of the input image).


torch.Tensor.to_sparse_cooTensor.to_sparse_coo()
   Convert a tensor to coordinate format.
   Examples:
      >>> dense = torch.randn(5, 5)
      >>> sparse = dense.to_sparse_coo()
      >>> sparse._nnz()
      25


torch.Tensor.negative_Tensor.negative_() -> Tensor
   In-place version of "negative()"


torch.Tensor.expand_asTensor.expand_as(other) -> Tensor
   Expand this tensor to the same size as "other".
   "self.expand_as(other)" is equivalent to
   "self.expand(other.size())".
   Please see "expand()" for more information about "expand".
   Parameters:
      other ("torch.Tensor") -- The result tensor has the same
      size as "other".


torch.erftorch.erf(input, *, out=None) -> Tensor
   Alias for "torch.special.erf()".


torch.cuda.get_rng_statetorch.cuda.get_rng_state(device='cuda')
   Returns the random number generator state of the specified GPU as a
   ByteTensor.
   Parameters:
      device (torch.device or int, optional) -- The
      device to return the RNG state of. Default: "'cuda'" (i.e.,
      "torch.device('cuda')", the current CUDA device).
   Return type:
      Tensor
   Warning:
     This function eagerly initializes CUDA.


torch.Tensor.diffTensor.diff(n=1, dim=- 1, prepend=None, append=None) -> Tensor
   See "torch.diff()"


torch.rangetorch.range(start=0, end, step=1, , out=None, dtype=None, layout=torch.strided, device=None, requires_grad=False) -> Tensor
   Returns a 1-D tensor of size \left\lfloor \frac{\text{end} -
   \text{start}}{\text{step}} \right\rfloor + 1 with values from
   "start" to "end" with step "step". Step is the gap between two
   values in the tensor.
      \text{out}_{i+1} = \text{out}_i + \text{step}.
   Warning:
     This function is deprecated and will be removed in a future
     release because its behavior is inconsistent with Python's range
     builtin. Instead, use "torch.arange()", which produces values in
     [start, end).
   Parameters:
      * start (float) -- the starting value for the set of
        points. Default: "0".
      * end (float) -- the ending value for the set of points
      * step (float) -- the gap between each pair of adjacent
        points. Default: "1".
   Keyword Arguments:
      * out (Tensor, optional*) -- the output tensor.


dtype ("torch.dtype", optional) -- the desired data type
        of returned tensor. Default: if "None", uses a global default
        (see "torch.set_default_tensor_type()"). If dtype is not
        given, infer the data type from the other input arguments. If
        any of start, end, or stop are floating-point, the
        dtype is inferred to be the default dtype, see
        "get_default_dtype()". Otherwise, the dtype is inferred to
        be torch.int64.
layout ("torch.layout", optional) -- the desired layout of
    returned Tensor. Default: "torch.strided".
device ("torch.device", optional) -- the desired device of
    returned tensor. Default: if "None", uses the current device
    for the default tensor type (see
    "torch.set_default_tensor_type()"). "device" will be the CPU
    for CPU tensor types and the current CUDA device for CUDA
    tensor types.
requires_grad (bool, optional) -- If autograd should


record operations on the returned tensor. Default: "False".
   Example:
      >>> torch.range(1, 4)
      tensor([ 1.,  2.,  3.,  4.])
      >>> torch.range(1, 4, 0.5)
      tensor([ 1.0000,  1.5000,  2.0000,  2.5000,  3.0000,  3.5000,  4.0000])


ConvBnReLU1dclass torch.ao.nn.intrinsic.qat.ConvBnReLU1d(in_channels, out_channels, kernel_size, stride=1, padding=0, dilation=1, groups=1, bias=None, padding_mode='zeros', eps=1e-05, momentum=0.1, freeze_bn=False, qconfig=None)
   A ConvBnReLU1d module is a module fused from Conv1d, BatchNorm1d
   and ReLU, attached with FakeQuantize modules for weight, used in
   quantization aware training.
   We combined the interface of "torch.nn.Conv1d" and
   "torch.nn.BatchNorm1d" and "torch.nn.ReLU".
   Similar to torch.nn.Conv1d, with FakeQuantize modules initialized
   to default.
   Variables:
      weight_fake_quant -- fake quant module for weight


torch.rand_liketorch.rand_like(input, , dtype=None, layout=None, device=None, requires_grad=False, memory_format=torch.preserve_format) -> Tensor
   Returns a tensor with the same size as "input" that is filled with
   random numbers from a uniform distribution on the interval [0, 1).
   "torch.rand_like(input)" is equivalent to "torch.rand(input.size(),
   dtype=input.dtype, layout=input.layout, device=input.device)".
   Parameters:
      input (Tensor) -- the size of "input" will determine size
      of the output tensor.
   Keyword Arguments:
      * dtype ("torch.dtype", optional) -- the desired data type
        of returned Tensor. Default: if "None", defaults to the dtype
        of "input".
      * layout ("torch.layout", optional) -- the desired layout of
        returned tensor. Default: if "None", defaults to the layout of
        "input".
      * device* ("torch.device", optional) -- the desired device of


returned tensor. Default: if "None", defaults to the device of
        "input".
      * requires_grad (bool, optional) -- If autograd should
        record operations on the returned tensor. Default: "False".
      * memory_format ("torch.memory_format", optional) -- the
        desired memory format of returned Tensor. Default:
        "torch.preserve_format".


torch.orgqrtorch.orgqr(input, tau) -> Tensor
   Alias for "torch.linalg.householder_product()".


torch.Tensor.detachTensor.detach()
   Returns a new Tensor, detached from the current graph.
   The result will never require gradient.
   This method also affects forward mode AD gradients and the result
   will never have forward mode AD gradients.
   Note:
     Returned Tensor shares the same storage with the original one.
     In-place modifications on either of them will be seen, and may
     trigger errors in correctness checks. IMPORTANT NOTE: Previously,
     in-place size / stride / storage changes (such as resize_ /
     resize_as_ / set_ / transpose_) to the returned tensor also
     update the original tensor. Now, these in-place changes will not
     update the original tensor anymore, and will instead trigger an
     error. For sparse tensors: In-place indices / values changes
     (such as zero_ / copy_ / add_) to the returned tensor will
     not update the original tensor anymore, and will instead trigger
     an error.


torch.cliptorch.clip(input, min=None, max=None, *, out=None) -> Tensor
   Alias for "torch.clamp()".


torch.Tensor.histogramTensor.histogram(input, bins, *, range=None, weight=None, density=False)
   See "torch.histogram()"


CUDAGraphclass torch.cuda.CUDAGraph
   Wrapper around a CUDA graph.
   Warning:
     This API is in beta and may change in future releases.
   capture_begin(pool=None)
      Begins capturing CUDA work on the current stream.
      Typically, you shouldn't call "capture_begin" yourself. Use
      "graph" or "make_graphed_callables()", which call
      "capture_begin" internally.
      Parameters:
         pool (optional) -- Token (returned by
         "graph_pool_handle()" or "other_Graph_instance.pool()") that
         hints this graph may share memory with the indicated pool.
         See Graph memory management.
   capture_end()
      Ends CUDA graph capture on the current stream. After
      "capture_end", "replay" may be called on this instance.
      Typically, you shouldn't call "capture_end" yourself. Use
      "graph" or "make_graphed_callables()", which call "capture_end"
      internally.
   debug_dump(debug_path)
      Parameters:


debug_dump(debug_path)
      Parameters:
         debug_path (required) -- Path to dump the graph to.
      Calls a debugging function to dump the graph if the debugging is
      enabled via CUDAGraph.enable_debug_mode()
   enable_debug_mode()
      Enables debugging mode for CUDAGraph.debug_dump.
   pool()
      Returns an opaque token representing the id of this graph's
      memory pool. This id can optionally be passed to another graph's
      "capture_begin", which hints the other graph may share the same
      memory pool.
   replay()
      Replays the CUDA work captured by this graph.
   reset()
      Deletes the graph currently held by this instance.


torch.are_deterministic_algorithms_enabledtorch.are_deterministic_algorithms_enabled()
   Returns True if the global deterministic flag is turned on. Refer
   to "torch.use_deterministic_algorithms()" documentation for more
   details.


torch.Tensor.squeezeTensor.squeeze(dim=None) -> Tensor
   See "torch.squeeze()"


Softmax2dclass torch.nn.Softmax2d
   Applies SoftMax over features to each spatial location.
   When given an image of "Channels x Height x Width", it will apply
   Softmax to each location (Channels, h_i, w_j)
   Shape:
      * Input: (N, C, H, W) or (C, H, W).
      * Output: (N, C, H, W) or (C, H, W) (same shape as input)
   Returns:
      a Tensor of the same dimension and shape as the input with
      values in the range [0, 1]
   Return type:
      None
   Examples:
      >>> m = nn.Softmax2d()
      >>> # you softmax over the 2nd dimension
      >>> input = torch.randn(2, 3, 12, 13)
      >>> output = m(input)


torch.chunktorch.chunk(input, chunks, dim=0) -> List of Tensors
   Attempts to split a tensor into the specified number of chunks.
   Each chunk is a view of the input tensor.
   Note:
     This function may return fewer than the specified number of
     chunks!
   See also:
     "torch.tensor_split()" a function that always returns exactly the
     specified number of chunks
   If the tensor size along the given dimension "dim" is divisible by
   "chunks", all returned chunks will be the same size. If the tensor
   size along the given dimension "dim" is not divisible by "chunks",
   all returned chunks will be the same size, except the last one. If
   such division is not possible, this function may return fewer than
   the specified number of chunks.
   Parameters:
      * input (Tensor) -- the tensor to split
      * chunks (int) -- number of chunks to return
      * dim (int) -- dimension along which to split the tensor
   -[ Example ]-

torch.arange(11).chunk(6)


-[ Example ]-

torch.arange(11).chunk(6)
   (tensor([0, 1]),
    tensor([2, 3]),
    tensor([4, 5]),
    tensor([6, 7]),
    tensor([8, 9]),
    tensor([10]))
torch.arange(12).chunk(6)
   (tensor([0, 1]),
    tensor([2, 3]),
    tensor([4, 5]),
    tensor([6, 7]),
    tensor([8, 9]),
    tensor([10, 11]))
torch.arange(13).chunk(6)
   (tensor([0, 1, 2]),
    tensor([3, 4, 5]),
    tensor([6, 7, 8]),
    tensor([ 9, 10, 11]),
    tensor([12]))


torch.nn.functional.group_normtorch.nn.functional.group_norm(input, num_groups, weight=None, bias=None, eps=1e-05)
   Applies Group Normalization for last certain number of dimensions.
   See "GroupNorm" for details.
   Return type:
      Tensor


SGDclass torch.optim.SGD(params, lr=, momentum=0, dampening=0, weight_decay=0, nesterov=False, *, maximize=False, foreach=None, differentiable=False)
   Implements stochastic gradient descent (optionally with momentum).
      \begin{aligned}      &\rule{110mm}{0.4pt}
      \      &\textbf{input}      : \gamma \text{ (lr)}, \: \theta_0
      \text{ (params)}, \: f(\theta)          \text{ (objective)}, \:
      \lambda \text{ (weight decay)},                          \
      &\hspace{13mm} \:\mu \text{ (momentum)}, \:\tau \text{
      (dampening)},      \:\textit{ nesterov,}\:\textit{ maximize}
      \[-1.ex]      &\rule{110mm}{0.4pt}
      \      &\textbf{for} \: t=1 \: \textbf{to} \: \ldots \:
      \textbf{do}                         \      &\hspace{5mm}g_t
      \leftarrow   \nabla_{\theta} f_t (\theta_{t-1})           \
      &\hspace{5mm}\textbf{if} \: \lambda \neq 0
      \      &\hspace{10mm} g_t \leftarrow g_t + \lambda
      \theta_{t-1}                            \


\theta_{t-1}                            \
      &\hspace{5mm}\textbf{if} \: \mu \neq 0
      \      &\hspace{10mm}\textbf{if} \: t > 1
      \      &\hspace{15mm} \textbf{b}t \leftarrow \mu
      \textbf{b} + (1-\tau) g_t           \
      &\hspace{10mm}\textbf{else}
      \      &\hspace{15mm} \textbf{b}t \leftarrow g_t
      \      &\hspace{10mm}\textbf{if} \: \textit{nesterov}
      \      &\hspace{15mm} g_t \leftarrow g + \mu \textbf{b}t
      \      &\hspace{10mm}\textbf{else}
      \[-1.ex]      &\hspace{15mm} g_t  \leftarrow  \textbf{b}_t
      \      &\hspace{5mm}\textbf{if} \: \textit{maximize}
      \      &\hspace{10mm}\theta_t \leftarrow \theta + \gamma
      g_t                   \[-1.ex]      &\hspace{5mm}\textbf{else}
      \[-1.ex]      &\hspace{10mm}\theta_t \leftarrow \theta_{t-1} -
      \gamma g_t                   \[-1.ex]      &\rule{110mm}{0.4pt}
      \[-1.ex]      &\bf{return} \:  \theta_t
      \[-1.ex]      &\rule{110mm}{0.4pt}


\[-1.ex]      &\rule{110mm}{0.4pt}
      \[-1.ex] \end{aligned}
   Nesterov momentum is based on the formula from On the importance of
   initialization and momentum in deep learning.
   Parameters:
      * params (iterable) -- iterable of parameters to optimize
        or dicts defining parameter groups
      * lr (float) -- learning rate
      * momentum (float, optional) -- momentum factor
        (default: 0)
      * weight_decay (float, optional) -- weight decay (L2
        penalty) (default: 0)
      * dampening (float, optional) -- dampening for
        momentum (default: 0)
      * nesterov (bool, optional) -- enables Nesterov
        momentum (default: False)
      * maximize (bool, optional) -- maximize the params
        based on the objective, instead of minimizing (default: False)
      * foreach (bool, optional) -- whether foreach
        implementation of optimizer is used (default: None)


differentiable (bool, optional) -- whether autograd
        should occur through the optimizer step in training.
        Otherwise, the step() function runs in a torch.no_grad()
        context. Setting to True can impair performance, so leave it
        False if you don't intend to run autograd through this
        instance (default: False)
   -[ Example ]-

optimizer = torch.optim.SGD(model.parameters(), lr=0.1, momentum=0.9)
optimizer.zero_grad()
loss_fn(model(input), target).backward()
optimizer.step()
   Note:
     The implementation of SGD with Momentum/Nesterov subtly differs
     from Sutskever et. al. and implementations in some other
     frameworks.Considering the specific case of Momentum, the update
     can be written as
        \begin{aligned}     v_{t+1} & = \mu * v_{t} + g_{t+1}, \
        p_{t+1} & = p_{t} - \text{lr} * v_{t+1}, \end{aligned}
     where p, g, v and \mu denote the parameters, gradient, velocity,


and momentum respectively.This is in contrast to Sutskever et.
     al. and other frameworks which employ an update of the form
        \begin{aligned}     v_{t+1} & = \mu * v_{t} + \text{lr} *
        g_{t+1}, \     p_{t+1} & = p_{t} - v_{t+1}. \end{aligned}
     The Nesterov version is analogously modified.Moreover, the
     initial value of the momentum buffer is set to the gradient value
     at the first step. This is in contrast to some other frameworks
     that initialize it to all zeros.
   add_param_group(param_group)
      Add a param group to the "Optimizer" s param_groups.
      This can be useful when fine tuning a pre-trained network as
      frozen layers can be made trainable and added to the "Optimizer"
      as training progresses.
      Parameters:
         param_group (dict) -- Specifies what Tensors should be
         optimized along with group specific optimization options.
   load_state_dict(state_dict)
      Loads the optimizer state.
      Parameters:


Loads the optimizer state.
      Parameters:
         state_dict (dict) -- optimizer state. Should be an
         object returned from a call to "state_dict()".
   register_step_post_hook(hook)
      Register an optimizer step post hook which will be called after
      optimizer step. It should have the following signature:
         hook(optimizer, args, kwargs) -> None
      The "optimizer" argument is the optimizer instance being used.
      Parameters:
         hook (Callable) -- The user defined hook to be
         registered.
      Returns:
         a handle that can be used to remove the added hook by calling
         "handle.remove()"
      Return type:
         "torch.utils.hooks.RemoveableHandle"
   register_step_pre_hook(hook)
      Register an optimizer step pre hook which will be called before
      optimizer step. It should have the following signature:
         hook(optimizer, args, kwargs) -> None or modified args and kwargs


The "optimizer" argument is the optimizer instance being used.
      If args and kwargs are modified by the pre-hook, then the
      transformed values are returned as a tuple containing the
      new_args and new_kwargs.
      Parameters:
         hook (Callable) -- The user defined hook to be
         registered.
      Returns:
         a handle that can be used to remove the added hook by calling
         "handle.remove()"
      Return type:
         "torch.utils.hooks.RemoveableHandle"
   state_dict()
      Returns the state of the optimizer as a "dict".
      It contains two entries:
      * state - a dict holding current optimization state. Its content
           differs between optimizer classes.
      * param_groups - a list containing all parameter groups where
        each
           parameter group is a dict
   zero_grad(set_to_none=False)
      Sets the gradients of all optimized "torch.Tensor" s to zero.
      Parameters:


Parameters:
         set_to_none (bool) -- instead of setting to zero, set
         the grads to None. This will in general have lower memory
         footprint, and can modestly improve performance. However, it
         changes certain behaviors. For example: 1. When the user
         tries to access a gradient and perform manual ops on it, a
         None attribute or a Tensor full of 0s will behave
         differently. 2. If the user requests
         "zero_grad(set_to_none=True)" followed by a backward pass,
         ".grad"s are guaranteed to be None for params that did not
         receive a gradient. 3. "torch.optim" optimizers have a
         different behavior if the gradient is 0 or None (in one case
         it does the step with a gradient of 0 and in the other it
         skips the step altogether).


torch.stdtorch.std(input, dim=None, , correction=1, keepdim=False, out=None) -> Tensor
   Calculates the standard deviation over the dimensions specified by
   "dim". "dim" can be a single dimension, list of dimensions, or
   "None" to reduce over all dimensions.
   The standard deviation (\sigma) is calculated as
      \sigma = \sqrt{\frac{1}{N - \delta
      N}\sum_{i=0}^{N-1}(x_i-\bar{x})^2}
   where x is the sample set of elements, \bar{x} is the sample mean,
   N is the number of samples and \delta N is the "correction".
   If "keepdim" is "True", the output tensor is of the same size as
   "input" except in the dimension(s) "dim" where it is of size 1.
   Otherwise, "dim" is squeezed (see "torch.squeeze()"), resulting in
   the output tensor having 1 (or "len(dim)") fewer dimension(s).
   Parameters:
      * input (Tensor) -- the input tensor.
      * dim (int or tuple of ints*) -- the dimension or
        dimensions to reduce.
   Keyword Arguments:


Keyword Arguments:
      * correction (int) --
        difference between the sample size and sample degrees of
        freedom. Defaults to Bessel's correction, "correction=1".
        Changed in version 2.0: Previously this argument was called
        "unbiased" and was a boolean with "True" corresponding to
        "correction=1" and "False" being "correction=0".
      * keepdim (bool) -- whether the output tensor has "dim"
        retained or not.
      * out (Tensor, optional) -- the output tensor.
   -[ Example ]-

a = torch.tensor(
   ...     [[ 0.2035,  1.2959,  1.8101, -0.4644],
   ...      [ 1.5027, -0.3270,  0.5905,  0.6538],
   ...      [-1.5745,  1.3330, -0.5596, -0.6548],
   ...      [ 0.1264, -0.5080,  1.6420,  0.1992]])
torch.std(a, dim=1, keepdim=True)
   tensor([[1.0311],
           [0.7477],
           [1.2204],
           [0.9087]])


torch.linalg.invtorch.linalg.inv(A, , out=None) -> Tensor
   Computes the inverse of a square matrix if it exists. Throws a
   RuntimeError if the matrix is not invertible.
   Letting \mathbb{K} be \mathbb{R} or \mathbb{C}, for a matrix A \in
   \mathbb{K}^{n \times n}, its inverse matrix A^{-1} \in
   \mathbb{K}^{n \times n} (if it exists) is defined as
      A^{-1}A = AA^{-1} = \mathrm{I}_n
   where \mathrm{I}_n is the n*-dimensional identity matrix.
   The inverse matrix exists if and only if A is invertible. In this
   case, the inverse is unique.
   Supports input of float, double, cfloat and cdouble dtypes. Also
   supports batches of matrices, and if "A" is a batch of matrices
   then the output has the same batch dimensions.
   Note:
     When inputs are on a CUDA device, this function synchronizes that
     device with the CPU.
   Note:
     Consider using "torch.linalg.solve()" if possible for multiplying
     a matrix on the left by the inverse, as: