torch.jit.annotatetorch.jit.annotate(the_type, the_value)
   This method is a pass-through function that returns the_value,
   used to hint TorchScript compiler the type of the_value. It is a
   no-op when running outside of TorchScript.
   Though TorchScript can infer correct type for most Python
   expressions, there are some cases where type inference can be
   wrong, including:
   * Empty containers like [] and {}, which TorchScript assumes to
     be container of Tensor
   * Optional types like Optional[T] but assigned a valid value of
     type T, TorchScript would assume it is type T rather than
     Optional[T]
   Note that annotate() does not help in init method of
   torch.nn.Module subclasses because it is executed in eager mode.
   To annotate types of torch.nn.Module attributes, use "Annotate()"
   instead.
   Example:
      import torch
      from typing import Dict
      @torch.jit.script
      def fn():


@torch.jit.script
      def fn():
          # Telling TorchScript that this empty dictionary is a (str -> int) dictionary
          # instead of default dictionary type of (str -> Tensor).
          d = torch.jit.annotate(Dict[str, int], {})
          # Without torch.jit.annotate above, following statement would fail because of
          # type mismatch.
          d["name"] = 20
   Parameters:
      * the_type -- Python type that should be passed to
        TorchScript compiler as type hint for the_value
      * the_value -- Value or expression to hint type for.
   Returns:
      the_value is passed back as return value.


torch.isfinitetorch.isfinite(input) -> Tensor
   Returns a new tensor with boolean elements representing if each
   element is finite or not.
   Real values are finite when they are not NaN, negative infinity, or
   infinity. Complex values are finite when both their real and
   imaginary parts are finite.
   Parameters:
      input (Tensor) -- the input tensor.
   Returns:
      A boolean tensor that is True where "input" is finite and False
      elsewhere
   Example:
      >>> torch.isfinite(torch.tensor([1, float('inf'), 2, float('-inf'), float('nan')]))
      tensor([True,  False,  True,  False,  False])


torch.set_rng_statetorch.set_rng_state(new_state)
   Sets the random number generator state.
   Parameters:
      new_state (torch.ByteTensor) -- The desired state


FixedQParamsFakeQuantizeclass torch.quantization.fake_quantize.FixedQParamsFakeQuantize(observer)
   Simulate quantize and dequantize with fixed quantization parameters
   in training time. Only per tensor quantization is supported.


torch.greatertorch.greater(input, other, *, out=None) -> Tensor
   Alias for "torch.gt()".


torch.Tensor.greater_equalTensor.greater_equal(other) -> Tensor
   See "torch.greater_equal()".


torch.Tensor.sortTensor.sort(dim=- 1, descending=False)
   See "torch.sort()"


torch.linspacetorch.linspace(start, end, steps, , out=None, dtype=None, layout=torch.strided, device=None, requires_grad=False) -> Tensor
   Creates a one-dimensional tensor of size "steps" whose values are
   evenly spaced from "start" to "end", inclusive. That is, the value
   are:
      (\text{start}, \text{start} + \frac{\text{end} -
      \text{start}}{\text{steps} - 1}, \ldots, \text{start} +
      (\text{steps} - 2) * \frac{\text{end} -
      \text{start}}{\text{steps} - 1}, \text{end})
   From PyTorch 1.11 linspace requires the steps argument. Use
   steps=100 to restore the previous behavior.
   Parameters:
      * start (float) -- the starting value for the set of
        points
      * end (float) -- the ending value for the set of points
      * steps (int) -- size of the constructed tensor
   Keyword Arguments:
      * out (Tensor, optional) -- the output tensor.
      * dtype (torch.dtype, optional*) -- the data type to


perform the computation in. Default: if None, uses the global
        default dtype (see torch.get_default_dtype()) when both
        "start" and "end" are real, and corresponding complex dtype
        when either is complex.
      * layout ("torch.layout", optional) -- the desired layout of
        returned Tensor. Default: "torch.strided".
      * device ("torch.device", optional) -- the desired device of
        returned tensor. Default: if "None", uses the current device
        for the default tensor type (see
        "torch.set_default_tensor_type()"). "device" will be the CPU
        for CPU tensor types and the current CUDA device for CUDA
        tensor types.
      * requires_grad (bool, optional) -- If autograd should
        record operations on the returned tensor. Default: "False".
   Example:
      >>> torch.linspace(3, 10, steps=5)
      tensor([  3.0000,   4.7500,   6.5000,   8.2500,  10.0000])
      >>> torch.linspace(-10, 10, steps=5)


torch.linspace(-10, 10, steps=5)
      tensor([-10.,  -5.,   0.,   5.,  10.])
      >>> torch.linspace(start=-10, end=10, steps=5)
      tensor([-10.,  -5.,   0.,   5.,  10.])
      >>> torch.linspace(start=-10, end=10, steps=1)
      tensor([-10.])


eluclass torch.ao.nn.quantized.functional.elu(input, scale, zero_point, alpha=1.0)
   This is the quantized version of "elu()".
   Parameters:
      * input (Tensor) -- quantized input
      * scale (float) -- quantization scale of the output tensor
      * zero_point (int) -- quantization zero point of the
        output tensor
      * alpha (float) -- the alpha constant
   Return type:
      Tensor


torch.nn.functional.pairwise_distancetorch.nn.functional.pairwise_distance(x1, x2, p=2.0, eps=1e-6, keepdim=False) -> Tensor
   See "torch.nn.PairwiseDistance" for details


torch.nn.functional.multi_margin_losstorch.nn.functional.multi_margin_loss(input, target, p=1, margin=1, weight=None, size_average=None, reduce=None, reduction='mean') -> Tensor
   See "MultiMarginLoss" for details.
   Return type:
      Tensor


PolynomialLRclass torch.optim.lr_scheduler.PolynomialLR(optimizer, total_iters=5, power=1.0, last_epoch=- 1, verbose=False)
   Decays the learning rate of each parameter group using a polynomial
   function in the given total_iters. When last_epoch=-1, sets initial
   lr as lr.
   Parameters:
      * optimizer (Optimizer) -- Wrapped optimizer.
      * total_iters (int) -- The number of steps that the
        scheduler decays the learning rate. Default: 5.
      * power (int) -- The power of the polynomial. Default:
        1.0.
      * verbose (bool) -- If "True", prints a message to stdout
        for each update. Default: "False".
   -[ Example ]-

Assuming optimizer uses lr = 0.001 for all groups
lr = 0.001     if epoch == 0
lr = 0.00075   if epoch == 1
lr = 0.00050   if epoch == 2
lr = 0.00025   if epoch == 3
lr = 0.0       if epoch >= 4
scheduler = PolynomialLR(self.opt, total_iters=4, power=1.0)


for epoch in range(100):
    train(...)
    validate(...)
    scheduler.step()
   get_last_lr()
      Return last computed learning rate by current scheduler.
   load_state_dict(state_dict)
      Loads the schedulers state.
      Parameters:
         state_dict (dict) -- scheduler state. Should be an
         object returned from a call to "state_dict()".
   print_lr(is_verbose, group, lr, epoch=None)
      Display the current learning rate.
   state_dict()
      Returns the state of the scheduler as a "dict".
      It contains an entry for every variable in self.dict which
      is not the optimizer.


torch.Tensor.flipTensor.flip(dims) -> Tensor
   See "torch.flip()"


ReflectionPad2dclass torch.nn.ReflectionPad2d(padding)
   Pads the input tensor using the reflection of the input boundary.
   For N-dimensional padding, use "torch.nn.functional.pad()".
   Parameters:
      padding (int, tuple) -- the size of the padding. If is
      int, uses the same padding in all boundaries. If a 4-tuple,
      uses (\text{padding_left}, \text{padding_right},
      \text{padding_top}, \text{padding_bottom})
   Shape:
      * Input: (N, C, H_{in}, W_{in}) or (C, H_{in}, W_{in}).
      * Output: (N, C, H_{out}, W_{out}) or (C, H_{out}, W_{out})
        where
        H_{out} = H_{in} + \text{padding_top} +
        \text{padding_bottom}
        W_{out} = W_{in} + \text{padding_left} +
        \text{padding_right}
   Examples:
      >>> m = nn.ReflectionPad2d(2)
      >>> input = torch.arange(9, dtype=torch.float).reshape(1, 1, 3, 3)
      >>> input
      tensor([[[[0., 1., 2.],
                [3., 4., 5.],
                [6., 7., 8.]]]])


[6., 7., 8.]]]])
      >>> m(input)
      tensor([[[[8., 7., 6., 7., 8., 7., 6.],
                [5., 4., 3., 4., 5., 4., 3.],
                [2., 1., 0., 1., 2., 1., 0.],
                [5., 4., 3., 4., 5., 4., 3.],
                [8., 7., 6., 7., 8., 7., 6.],
                [5., 4., 3., 4., 5., 4., 3.],
                [2., 1., 0., 1., 2., 1., 0.]]]])
      >>> # using different paddings for different sides
      >>> m = nn.ReflectionPad2d((1, 1, 2, 0))
      >>> m(input)
      tensor([[[[7., 6., 7., 8., 7.],
                [4., 3., 4., 5., 4.],
                [1., 0., 1., 2., 1.],
                [4., 3., 4., 5., 4.],
                [7., 6., 7., 8., 7.]]]])


torch.Tensor.takeTensor.take(indices) -> Tensor
   See "torch.take()"


torch.matmultorch.matmul(input, other, *, out=None) -> Tensor
   Matrix product of two tensors.
   The behavior depends on the dimensionality of the tensors as
   follows:
   * If both tensors are 1-dimensional, the dot product (scalar) is
     returned.
   * If both arguments are 2-dimensional, the matrix-matrix product is
     returned.
   * If the first argument is 1-dimensional and the second argument is
     2-dimensional, a 1 is prepended to its dimension for the purpose
     of the matrix multiply. After the matrix multiply, the prepended
     dimension is removed.
   * If the first argument is 2-dimensional and the second argument is
     1-dimensional, the matrix-vector product is returned.
   * If both arguments are at least 1-dimensional and at least one
     argument is N-dimensional (where N > 2), then a batched matrix
     multiply is returned.  If the first argument is 1-dimensional, a
     1 is prepended to its dimension for the purpose of the batched


matrix multiply and removed after.  If the second argument is
     1-dimensional, a 1 is appended to its dimension for the purpose
     of the batched matrix multiple and removed after. The non-matrix
     (i.e. batch) dimensions are broadcasted (and thus must be
     broadcastable).  For example, if "input" is a (j \times 1 \times
     n \times n) tensor and "other" is a (k \times n \times n) tensor,
     "out" will be a (j \times k \times n \times n) tensor.
     Note that the broadcasting logic only looks at the batch
     dimensions when determining if the inputs are broadcastable, and
     not the matrix dimensions. For example, if "input" is a (j \times
     1 \times n \times m) tensor and "other" is a (k \times m \times
     p) tensor, these inputs are valid for broadcasting even though
     the final two dimensions (i.e. the matrix dimensions) are
     different. "out" will be a (j \times k \times n \times p) tensor.
   This operation has support for arguments with sparse layouts. In


particular the matrix-matrix (both arguments 2-dimensional)
   supports sparse arguments with the same restrictions as
   "torch.mm()"
   Warning:
     Sparse support is a beta feature and some layout(s)/dtype/device
     combinations may not be supported, or may not have autograd
     support. If you notice missing functionality please open a
     feature request.
   This operator supports TensorFloat32.
   On certain ROCm devices, when using float16 inputs this module will
   use different precision for backward.
   Note:
     The 1-dimensional dot product version of this function does not
     support an "out" parameter.
   Parameters:
      * input (Tensor) -- the first tensor to be multiplied
      * other (Tensor) -- the second tensor to be multiplied
   Keyword Arguments:
      out (Tensor, optional) -- the output tensor.
   Example:
      >>> # vector x vector
      >>> tensor1 = torch.randn(3)
      >>> tensor2 = torch.randn(3)
      >>> torch.matmul(tensor1, tensor2).size()


torch.matmul(tensor1, tensor2).size()
      torch.Size([])
      >>> # matrix x vector
      >>> tensor1 = torch.randn(3, 4)
      >>> tensor2 = torch.randn(4)
      >>> torch.matmul(tensor1, tensor2).size()
      torch.Size([3])
      >>> # batched matrix x broadcasted vector
      >>> tensor1 = torch.randn(10, 3, 4)
      >>> tensor2 = torch.randn(4)
      >>> torch.matmul(tensor1, tensor2).size()
      torch.Size([10, 3])
      >>> # batched matrix x batched matrix
      >>> tensor1 = torch.randn(10, 3, 4)
      >>> tensor2 = torch.randn(10, 4, 5)
      >>> torch.matmul(tensor1, tensor2).size()
      torch.Size([10, 3, 5])
      >>> # batched matrix x broadcasted matrix
      >>> tensor1 = torch.randn(10, 3, 4)
      >>> tensor2 = torch.randn(4, 5)
      >>> torch.matmul(tensor1, tensor2).size()
      torch.Size([10, 3, 5])


default_eval_fnclass torch.quantization.default_eval_fn(model, calib_data)
   Default evaluation function takes a torch.utils.data.Dataset or a
   list of input Tensors and run the model on the dataset


Linearclass torch.ao.nn.quantized.Linear(in_features, out_features, bias_=True, dtype=torch.qint8)
   A quantized linear module with quantized tensor as inputs and
   outputs. We adopt the same interface as torch.nn.Linear, please
   see https://pytorch.org/docs/stable/nn.html#torch.nn.Linear for
   documentation.
   Similar to "Linear", attributes will be randomly initialized at
   module creation time and will be overwritten later
   Variables:
      * weight (Tensor) -- the non-learnable quantized weights
        of the module of shape (\text{out_features},
        \text{in_features}).
      * bias (Tensor) -- the non-learnable bias of the module of
        shape (\text{out_features}). If "bias" is "True", the values
        are initialized to zero.
      * scale -- scale parameter of output Quantized Tensor,
        type: double
      * zero_point -- zero_point parameter for output Quantized
        Tensor, type: long
   Examples:


Tensor, type: long
   Examples:
      >>> m = nn.quantized.Linear(20, 30)
      >>> input = torch.randn(128, 20)
      >>> input = torch.quantize_per_tensor(input, 1.0, 0, torch.quint8)
      >>> output = m(input)
      >>> print(output.size())
      torch.Size([128, 30])
   classmethod from_float(mod)
      Create a quantized module from an observed float module
      Parameters:
         mod (Module) -- a float module, either produced by
         torch.ao.quantization utilities or provided by the user
   classmethod from_reference(ref_qlinear, output_scale, output_zero_point)
      Create a (fbgemm/qnnpack) quantized module from a reference
      quantized module
      Parameters:
         * ref_qlinear (Module) -- a reference quantized linear
           module, either produced by torch.ao.quantization utilities
           or provided by the user
         * output_scale (float) -- scale for output Tensor
         * output_zero_point (int) -- zero point for output


Tensor


torch.lerptorch.lerp(input, end, weight, , out=None)
   Does a linear interpolation of two tensors "start" (given by
   "input") and "end" based on a scalar or tensor "weight" and returns
   the resulting "out" tensor.
      \text{out}_i = \text{start}_i + \text{weight}_i \times
      (\text{end}_i - \text{start}_i)
   The shapes of "start" and "end" must be broadcastable. If "weight"
   is a tensor, then the shapes of "weight", "start", and "end" must
   be broadcastable.
   Parameters:
      * input (Tensor) -- the tensor with the starting points
      * end (Tensor) -- the tensor with the ending points
      * weight (float or tensor) -- the weight for the
        interpolation formula
   Keyword Arguments:
      out (Tensor, optional*) -- the output tensor.
   Example:
      >>> start = torch.arange(1., 5.)
      >>> end = torch.empty(4).fill_(10)
      >>> start
      tensor([ 1.,  2.,  3.,  4.])
      >>> end
      tensor([ 10.,  10.,  10.,  10.])


tensor([ 10.,  10.,  10.,  10.])
      >>> torch.lerp(start, end, 0.5)
      tensor([ 5.5000,  6.0000,  6.5000,  7.0000])
      >>> torch.lerp(start, end, torch.full_like(start, 0.5))
      tensor([ 5.5000,  6.0000,  6.5000,  7.0000])


torch.Tensor.cfloatTensor.cfloat(memory_format=torch.preserve_format) -> Tensor
   "self.cfloat()" is equivalent to "self.to(torch.complex64)". See
   "to()".
   Parameters:
      memory_format ("torch.memory_format", optional) -- the
      desired memory format of returned Tensor. Default:
      "torch.preserve_format".


torch.Tensor.atanhTensor.atanh() -> Tensor
   See "torch.atanh()"


torch.nn.functional.softmaxtorch.nn.functional.softmax(input, dim=None, stacklevel=3, dtype=None)
   Applies a softmax function.
   Softmax is defined as:
   \text{Softmax}(x) = \frac{\exp(x_i)}{\sum_j \exp(x_j)}
   It is applied to all slices along dim, and will re-scale them so
   that the elements lie in the range [0, 1] and sum to 1.
   See "Softmax" for more details.
   Parameters:
      * input (Tensor) -- input
      * dim (int) -- A dimension along which softmax will be
        computed.
      * dtype ("torch.dtype", optional) -- the desired data type
        of returned tensor. If specified, the input tensor is casted
        to "dtype" before the operation is performed. This is useful
        for preventing data type overflows. Default: None.
   Return type:
      Tensor
   Note:
     This function doesn't work directly with NLLLoss, which expects
     the Log to be computed between the Softmax and itself. Use


log_softmax instead (it's faster and has better numerical
     properties).


torch.sym_floattorch.sym_float(a)
   SymInt-aware utility for float casting.
   Parameters:
      a (SymInt, SymFloat, or object) -- Object to cast


torch.addrtorch.addr(input, vec1, vec2, , beta=1, alpha=1, out=None) -> Tensor
   Performs the outer-product of vectors "vec1" and "vec2" and adds it
   to the matrix "input".
   Optional values "beta" and "alpha" are scaling factors on the outer
   product between "vec1" and "vec2" and the added matrix "input"
   respectively.
      \text{out} = \beta\ \text{input} + \alpha\ (\text{vec1} \otimes
      \text{vec2})
   If "beta" is 0, then "input" will be ignored, and nan and inf
   in it will not be propagated.
   If "vec1" is a vector of size n and "vec2" is a vector of size
   m, then "input" must be broadcastable with a matrix of size (n
   \times m) and "out" will be a matrix of size (n \times m).
   Parameters:
      * input (Tensor) -- matrix to be added
      * vec1 (Tensor) -- the first vector of the outer product
      * vec2 (Tensor*) -- the second vector of the outer product
   Keyword Arguments:


Keyword Arguments:
      * beta (Number, optional) -- multiplier for "input"
        (\beta)
      * alpha (Number, optional) -- multiplier for
        \text{vec1} \otimes \text{vec2} (\alpha)
      * out (Tensor, optional) -- the output tensor.
   Example:
      >>> vec1 = torch.arange(1., 4.)
      >>> vec2 = torch.arange(1., 3.)
      >>> M = torch.zeros(3, 2)
      >>> torch.addr(M, vec1, vec2)
      tensor([[ 1.,  2.],
              [ 2.,  4.],
              [ 3.,  6.]])


torch.Tensor.index_selectTensor.index_select(dim, index) -> Tensor
   See "torch.index_select()"


torch.linalg.pinvtorch.linalg.pinv(A, , atol=None, rtol=None, hermitian=False, out=None) -> Tensor
   Computes the pseudoinverse (Moore-Penrose inverse) of a matrix.
   The pseudoinverse may be defined algebraically but it is more
   computationally convenient to understand it through the SVD
   Supports input of float, double, cfloat and cdouble dtypes. Also
   supports batches of matrices, and if "A" is a batch of matrices
   then the output has the same batch dimensions.
   If "hermitian"= True, "A" is assumed to be Hermitian if complex
   or symmetric if real, but this is not checked internally. Instead,
   just the lower triangular part of the matrix is used in the
   computations.
   The singular values (or the norm of the eigenvalues when
   "hermitian"= True*) that are below \max(\text{atol}, \sigma_1
   \cdot \text{rtol}) threshold are treated as zero and discarded in
   the computation, where \sigma_1 is the largest singular value (or
   eigenvalue).


eigenvalue).
   If "rtol" is not specified and "A" is a matrix of dimensions (m,
   n), the relative tolerance is set to be \text{rtol} = \max(m, n)
   \varepsilon and \varepsilon is the epsilon value for the dtype of
   "A" (see "finfo"). If "rtol" is not specified and "atol" is
   specified to be larger than zero then "rtol" is set to zero.
   If "atol" or "rtol" is a "torch.Tensor", its shape must be
   broadcastable to that of the singular values of "A" as returned by
   "torch.linalg.svd()".
   Note:
     This function uses "torch.linalg.svd()" if "hermitian"= False
     and "torch.linalg.eigh()" if "hermitian"= True. For CUDA
     inputs, this function synchronizes that device with the CPU.
   Note:
     Consider using "torch.linalg.lstsq()" if possible for multiplying
     a matrix on the left by the pseudoinverse, as:
        torch.linalg.lstsq(A, B).solution == A.pinv() @ B
     It is always preferred to use "lstsq()" when possible, as it is


faster and more numerically stable than computing the
     pseudoinverse explicitly.
   Note:
     This function has NumPy compatible variant linalg.pinv(A, rcond,
     hermitian=False). However, use of the positional argument
     "rcond" is deprecated in favor of "rtol".
   Warning:
     This function uses internally "torch.linalg.svd()" (or
     "torch.linalg.eigh()" when "hermitian"= True), so its
     derivative has the same problems as those of these functions. See
     the warnings in "torch.linalg.svd()" and "torch.linalg.eigh()"
     for more details.
   See also:
     "torch.linalg.inv()" computes the inverse of a square matrix.
     "torch.linalg.lstsq()" computes "A".pinv() @ "B" with a
     numerically stable algorithm.
   Parameters:
      * A (Tensor) -- tensor of shape (, m, n) where *** is
        zero or more batch dimensions.
      * rcond (float, Tensor, optional) -- [NumPy
        Compat]. Alias for "rtol". Default: None*.
   Keyword Arguments:


Keyword Arguments:
      * atol (float, Tensor, optional) -- the absolute
        tolerance value. When None it's considered to be zero.
        Default: None.
      * rtol (float, Tensor, optional) -- the relative
        tolerance value. See above for the value it takes when None.
        Default: None.
      * hermitian (bool, optional) -- indicates whether "A"
        is Hermitian if complex or symmetric if real. Default:
        False.
      * out (Tensor, optional) -- output tensor. Ignored if
        None. Default: None.
   Examples:
      >>> A = torch.randn(3, 5)
      >>> A
      tensor([[ 0.5495,  0.0979, -1.4092, -0.1128,  0.4132],
              [-1.1143, -0.3662,  0.3042,  1.6374, -0.9294],
              [-0.3269, -0.5745, -0.0382, -0.5922, -0.6759]])
      >>> torch.linalg.pinv(A)
      tensor([[ 0.0600, -0.1933, -0.2090],
              [-0.0903, -0.0817, -0.4752],
              [-0.7124, -0.1631, -0.2272],


[-0.7124, -0.1631, -0.2272],
              [ 0.1356,  0.3933, -0.5023],
              [-0.0308, -0.1725, -0.5216]])
      >>> A = torch.randn(2, 6, 3)
      >>> Apinv = torch.linalg.pinv(A)
      >>> torch.dist(Apinv @ A, torch.eye(3))
      tensor(8.5633e-07)
      >>> A = torch.randn(3, 3, dtype=torch.complex64)
      >>> A = A + A.T.conj()  # creates a Hermitian matrix
      >>> Apinv = torch.linalg.pinv(A, hermitian=True)
      >>> torch.dist(Apinv @ A, torch.eye(3))
      tensor(1.0830e-06)


HingeEmbeddingLossclass torch.nn.HingeEmbeddingLoss(margin=1.0, size_average=None, reduce=None, reduction='mean')
   Measures the loss given an input tensor x and a labels tensor y
   (containing 1 or -1). This is usually used for measuring whether
   two inputs are similar or dissimilar, e.g. using the L1 pairwise
   distance as x, and is typically used for learning nonlinear
   embeddings or semi-supervised learning.
   The loss function for n-th sample in the mini-batch is
      l_n = \begin{cases}     x_n, & \text{if}\; y_n = 1,\     \max
      {0, \Delta - x_n}, & \text{if}\; y_n = -1, \end{cases}
   and the total loss functions is
      \ell(x, y) = \begin{cases}     \operatorname{mean}(L), &
      \text{if reduction} = \text{mean';}\\
      \operatorname{sum}(L),  & \text{if reduction} = \text{sum'.}
      \end{cases}
   where L = {l_1,\dots,l_N}^\top.
   Parameters:
      * margin (float, optional) -- Has a default value of
        1.


1.
      * size_average (bool, optional) -- Deprecated (see
        "reduction"). By default, the losses are averaged over each
        loss element in the batch. Note that for some losses, there
        are multiple elements per sample. If the field "size_average"
        is set to "False", the losses are instead summed for each
        minibatch. Ignored when "reduce" is "False". Default: "True"
      * reduce (bool, optional) -- Deprecated (see
        "reduction"). By default, the losses are averaged or summed
        over observations for each minibatch depending on
        "size_average". When "reduce" is "False", returns a loss per
        batch element instead and ignores "size_average". Default:
        "True"
      * reduction (str, optional) -- Specifies the reduction
        to apply to the output: "'none'" | "'mean'" | "'sum'".
        "'none'": no reduction will be applied, "'mean'": the sum of


the output will be divided by the number of elements in the
        output, "'sum'": the output will be summed. Note:
        "size_average" and "reduce" are in the process of being
        deprecated, and in the meantime, specifying either of those
        two args will override "reduction". Default: "'mean'"
   Shape:
      * Input: () where * means, any number of dimensions. The sum
        operation operates over all the elements.
      * Target: (), same shape as the input
      * Output: scalar. If "reduction" is "'none'", then same shape as
        the input


torch.set_default_devicetorch.set_default_device(device)
   Sets the default "torch.Tensor" to be allocated on "device".  This
   does not affect factory function calls which are called with an
   explicit "device" argument.  Factory calls will be performed as if
   they were passed "device" as an argument.
   To only temporarily change the default device instead of setting it
   globally, use "with torch.device(device):" instead.
   The default device is initially "cpu".  If you set the default
   tensor device to another device (e.g., "cuda") without a device
   index, tensors will be allocated on whatever the current device for
   the device type, even after "torch.cuda.set_device()" is called.
   Warning:
     This function imposes a slight performance cost on every Python
     call to the torch API (not just factory functions).  If this is
     causing problems for you, please comment on
     https://github.com/pytorch/pytorch/issues/92701
   Parameters:


Parameters:
      device (device or string) -- the device to set as
      default
   Example:
      >>> torch.tensor([1.2, 3]).device
      device(type='cpu')
      >>> torch.set_default_device('cuda')  # current device is 0
      >>> torch.tensor([1.2, 3]).device
      device(type='cuda', index=0)
      >>> torch.set_default_device('cuda:1')
      >>> torch.tensor([1.2, 3]).device
      device(type='cuda', index=1)


torch.nn.functional.max_pool1dtorch.nn.functional.max_pool1d(input, kernel_size, stride=None, padding=0, dilation=1, ceil_mode=False, return_indices=False)
   Applies a 1D max pooling over an input signal composed of several
   input planes.
   Note:
     The order of "ceil_mode" and "return_indices" is different from
     what seen in "MaxPool1d", and will change in a future release.
   See "MaxPool1d" for details.
   Parameters:
      * input -- input tensor of shape (\text{minibatch} ,
        \text{in_channels} , iW), minibatch dim optional.
      * kernel_size -- the size of the window. Can be a single
        number or a tuple (kW,)
      * stride -- the stride of the window. Can be a single number
        or a tuple (sW,). Default: "kernel_size"
      * padding -- Implicit negative infinity padding to be added
        on both sides, must be >= 0 and <= kernel_size / 2.
      * dilation -- The stride between elements within a sliding


window, must be > 0.
      * ceil_mode -- If "True", will use ceil instead of floor
        to compute the output shape. This ensures that every element
        in the input tensor is covered by a sliding window.
      * return_indices -- If "True", will return the argmax along
        with the max values. Useful for
        "torch.nn.functional.max_unpool1d" later


torch.Tensor.to_sparse_cscTensor.to_sparse_csc() -> Tensor
   Convert a tensor to compressed column storage (CSC) format.  Except
   for strided tensors, only works with 2D tensors.  If the "self" is
   strided, then the number of dense dimensions could be specified,
   and a hybrid CSC tensor will be created, with dense_dim dense
   dimensions and self.dim() - 2 - dense_dim batch dimension.
   Parameters:
      dense_dim (int, optional) -- Number of dense
      dimensions of the resulting CSC tensor.  This argument should be
      used only if "self" is a strided tensor, and must be a value
      between 0 and dimension of "self" tensor minus two.
   Example:
      >>> dense = torch.randn(5, 5)
      >>> sparse = dense.to_sparse_csc()
      >>> sparse._nnz()
      25
      >>> dense = torch.zeros(3, 3, 1, 1)
      >>> dense[0, 0] = dense[1, 2] = dense[2, 1] = 1
      >>> dense.to_sparse_csc(dense_dim=2)
      tensor(ccol_indices=tensor([0, 1, 2, 3]),


tensor(ccol_indices=tensor([0, 1, 2, 3]),
             row_indices=tensor([0, 2, 1]),
             values=tensor([[[1.]],
                            [[1.]],
                            [[1.]]]), size=(3, 3, 1, 1), nnz=3,
             layout=torch.sparse_csc)


torch.savetorch.save(obj, f, pickle_module=pickle, pickle_protocol=DEFAULT_PROTOCOL, _use_new_zipfile_serialization=True)
   Saves an object to a disk file.
   See also: Saving and loading tensors
   Parameters:
      * obj (object) -- saved object
      * f (Union[str, PathLike, BinaryIO,
        IO[bytes]]) -- a file-like object (has to implement
        write and flush) or a string or os.PathLike object containing
        a file name
      * pickle_module (Any) -- module used for pickling metadata
        and objects
      * pickle_protocol (int) -- can be specified to override
        the default protocol
   Note:
     A common PyTorch convention is to save tensors using .pt file
     extension.
   Note:
     PyTorch preserves storage sharing across serialization. See
     Saving and loading tensors preserves views for more details.
   Note:
     The 1.6 release of PyTorch switched "torch.save" to use a new


zipfile-based file format. "torch.load" still retains the ability
     to load files in the old format. If for any reason you want
     "torch.save" to use the old format, pass the kwarg
     "_use_new_zipfile_serialization=False".
   -[ Example ]-

Save to file
x = torch.tensor([0, 1, 2, 3, 4])
torch.save(x, 'tensor.pt')
Save to io.BytesIO buffer
buffer = io.BytesIO()
torch.save(x, buffer)


torch.triutorch.triu(input, diagonal=0, , out=None) -> Tensor
   Returns the upper triangular part of a matrix (2-D tensor) or batch
   of matrices "input", the other elements of the result tensor "out"
   are set to 0.
   The upper triangular part of the matrix is defined as the elements
   on and above the diagonal.
   The argument "diagonal" controls which diagonal to consider. If
   "diagonal" = 0, all elements on and above the main diagonal are
   retained. A positive value excludes just as many diagonals above
   the main diagonal, and similarly a negative value includes just as
   many diagonals below the main diagonal. The main diagonal are the
   set of indices \lbrace (i, i) \rbrace for i \in [0, \min{d_{1},
   d_{2}} - 1] where d_{1}, d_{2} are the dimensions of the matrix.
   Parameters:
      * input (Tensor) -- the input tensor.
      * diagonal (int, optional*) -- the diagonal to consider
   Keyword Arguments:


Keyword Arguments:
      out (Tensor, optional) -- the output tensor.
   Example:
      >>> a = torch.randn(3, 3)
      >>> a
      tensor([[ 0.2309,  0.5207,  2.0049],
              [ 0.2072, -1.0680,  0.6602],
              [ 0.3480, -0.5211, -0.4573]])
      >>> torch.triu(a)
      tensor([[ 0.2309,  0.5207,  2.0049],
              [ 0.0000, -1.0680,  0.6602],
              [ 0.0000,  0.0000, -0.4573]])
      >>> torch.triu(a, diagonal=1)
      tensor([[ 0.0000,  0.5207,  2.0049],
              [ 0.0000,  0.0000,  0.6602],
              [ 0.0000,  0.0000,  0.0000]])
      >>> torch.triu(a, diagonal=-1)
      tensor([[ 0.2309,  0.5207,  2.0049],
              [ 0.2072, -1.0680,  0.6602],
              [ 0.0000, -0.5211, -0.4573]])
      >>> b = torch.randn(4, 6)
      >>> b
      tensor([[ 0.5876, -0.0794, -1.8373,  0.6654,  0.2604,  1.5235],
              [-0.2447,  0.9556, -1.2919,  1.3378, -0.1768, -1.0857],
              [ 0.4333,  0.3146,  0.6576, -1.0432,  0.9348, -0.4410],


[-0.9888,  1.0679, -1.3337, -1.6556,  0.4798,  0.2830]])
      >>> torch.triu(b, diagonal=1)
      tensor([[ 0.0000, -0.0794, -1.8373,  0.6654,  0.2604,  1.5235],
              [ 0.0000,  0.0000, -1.2919,  1.3378, -0.1768, -1.0857],
              [ 0.0000,  0.0000,  0.0000, -1.0432,  0.9348, -0.4410],
              [ 0.0000,  0.0000,  0.0000,  0.0000,  0.4798,  0.2830]])
      >>> torch.triu(b, diagonal=-1)
      tensor([[ 0.5876, -0.0794, -1.8373,  0.6654,  0.2604,  1.5235],
              [-0.2447,  0.9556, -1.2919,  1.3378, -0.1768, -1.0857],
              [ 0.0000,  0.3146,  0.6576, -1.0432,  0.9348, -0.4410],
              [ 0.0000,  0.0000, -1.3337, -1.6556,  0.4798,  0.2830]])


torch.Tensor.select_scatterTensor.select_scatter(src, dim, index) -> Tensor
   See "torch.select_scatter()"


torch.linalg.svdvalstorch.linalg.svdvals(A, , driver=None, out=None) -> Tensor
   Computes the singular values of a matrix.
   Supports input of float, double, cfloat and cdouble dtypes. Also
   supports batches of matrices, and if "A" is a batch of matrices
   then the output has the same batch dimensions.
   The singular values are returned in descending order.
   Note:
     This function is equivalent to NumPy's linalg.svd(A,
     compute_uv=False).
   Note:
     When inputs are on a CUDA device, this function synchronizes that
     device with the CPU.
   See also:
     "torch.linalg.svd()" computes the full singular value
     decomposition.
   Parameters:
      A (Tensor) -- tensor of shape (, m, n) where *** is
      zero or more batch dimensions.
   Keyword Arguments:
      * driver (str, optional) -- name of the cuSOLVER
        method to be used. This keyword argument only works on CUDA
        inputs. Available options are: None, gesvd, gesvdj, and


gesvda. Check "torch.linalg.svd()" for details. Default:
        None.
      * out (Tensor, optional) -- output tensor. Ignored if
        None. Default: None.
   Returns:
      A real-valued tensor, even when "A" is complex.
   Examples:
      >>> A = torch.randn(5, 3)
      >>> S = torch.linalg.svdvals(A)
      >>> S
      tensor([2.5139, 2.1087, 1.1066])
      >>> torch.dist(S, torch.linalg.svd(A, full_matrices=False).S)
      tensor(2.4576e-07)


AdamWclass torch.optim.AdamW(params, lr=0.001, betas=(0.9, 0.999), eps=1e-08, weight_decay=0.01, amsgrad=False, *, maximize=False, foreach=None, capturable=False, differentiable=False)
   Implements AdamW algorithm.
      \begin{aligned}      &\rule{110mm}{0.4pt}
      \      &\textbf{input}      : \gamma \text{(lr)}, \: \beta_1,
      \beta_2          \text{(betas)}, \: \theta_0 \text{(params)}, \:
      f(\theta) \text{(objective)},          \: \epsilon \text{
      (epsilon)}                                                    \
      &\hspace{13mm}      \lambda \text{(weight decay)},  \:
      \textit{amsgrad},          \: \textit{maximize}
      \      &\textbf{initialize} : m_0 \leftarrow 0 \text{ (first
      moment)}, v_0 \leftarrow 0          \text{ ( second moment)}, \:
      \widehat{v_0}^{max}\leftarrow 0              \[-1.ex]
      &\rule{110mm}{0.4pt}
      \      &\textbf{for} \: t=1 \: \textbf{to} \: \ldots \:
      \textbf{do}                         \


\textbf{do}                         \
      &\hspace{5mm}\textbf{if} \: \textit{maximize}:
      \      &\hspace{10mm}g_t           \leftarrow
      -\nabla_{\theta} f_t (\theta_{t-1})          \
      &\hspace{5mm}\textbf{else}
      \      &\hspace{10mm}g_t           \leftarrow   \nabla_{\theta}
      f_t (\theta_{t-1})           \      &\hspace{5mm} \theta_t
      \leftarrow \theta_{t-1} - \gamma \lambda \theta_{t-1}         \
      &\hspace{5mm}m_t           \leftarrow   \beta_1 m_{t-1} + (1 -
      \beta_1) g_t          \      &\hspace{5mm}v_t
      \leftarrow   \beta_2 v_{t-1} + (1-\beta_2) g^2_t          \
      &\hspace{5mm}\widehat{m_t} \leftarrow   m_t/\big(1-\beta_1^t
      \big)                   \      &\hspace{5mm}\widehat{v_t}
      \leftarrow   v_t/\big(1-\beta_2^t \big)                   \
      &\hspace{5mm}\textbf{if} \: amsgrad
      \      &\hspace{10mm}\widehat{v_t}^{max} \leftarrow
      \mathrm{max}(\widehat{v_t}^{max},          \widehat{v_t})


\      &\hspace{10mm}\theta_t \leftarrow \theta_t - \gamma
      \widehat{m_t}/          \big(\sqrt{\widehat{v_t}^{max}} +
      \epsilon \big)                                 \
      &\hspace{5mm}\textbf{else}
      \      &\hspace{10mm}\theta_t \leftarrow \theta_t - \gamma
      \widehat{m_t}/          \big(\sqrt{\widehat{v_t}} + \epsilon
      \big)                                       \
      &\rule{110mm}{0.4pt}
      \[-1.ex]      &\bf{return} \:  \theta_t
      \[-1.ex]      &\rule{110mm}{0.4pt}
      \[-1.ex] \end{aligned}
   For further details regarding the algorithm we refer to Decoupled
   Weight Decay Regularization.
   Parameters:
      * params (iterable) -- iterable of parameters to optimize
        or dicts defining parameter groups
      * lr (float, optional) -- learning rate (default:
        1e-3)
      * betas (Tuple[float, float], optional) --
        coefficients used for computing running averages of gradient


and its square (default: (0.9, 0.999))
      * eps (float, optional) -- term added to the
        denominator to improve numerical stability (default: 1e-8)
      * weight_decay (float, optional) -- weight decay
        coefficient (default: 1e-2)
      * amsgrad (bool, optional) -- whether to use the
        AMSGrad variant of this algorithm from the paper On the
        Convergence of Adam and Beyond (default: False)
      * maximize (bool, optional) -- maximize the params
        based on the objective, instead of minimizing (default: False)
      * foreach (bool, optional) -- whether foreach
        implementation of optimizer is used. If unspecified by the
        user (so foreach is None), we will try to use foreach over the
        for-loop implementation on CUDA, since it is usually
        significantly more performant. (default: None)
      * capturable (bool, optional) -- whether this instance


is safe to capture in a CUDA graph. Passing True can impair
        ungraphed performance, so if you don't intend to graph capture
        this instance, leave it False (default: False)
      * differentiable (bool, optional) -- whether autograd
        should occur through the optimizer step in training.
        Otherwise, the step() function runs in a torch.no_grad()
        context. Setting to True can impair performance, so leave it
        False if you don't intend to run autograd through this
        instance (default: False)
   add_param_group(param_group)
      Add a param group to the "Optimizer" s param_groups.
      This can be useful when fine tuning a pre-trained network as
      frozen layers can be made trainable and added to the "Optimizer"
      as training progresses.
      Parameters:
         param_group (dict) -- Specifies what Tensors should be
         optimized along with group specific optimization options.
   load_state_dict(state_dict)


load_state_dict(state_dict)
      Loads the optimizer state.
      Parameters:
         state_dict (dict) -- optimizer state. Should be an
         object returned from a call to "state_dict()".
   register_step_post_hook(hook)
      Register an optimizer step post hook which will be called after
      optimizer step. It should have the following signature:
         hook(optimizer, args, kwargs) -> None
      The "optimizer" argument is the optimizer instance being used.
      Parameters:
         hook (Callable) -- The user defined hook to be
         registered.
      Returns:
         a handle that can be used to remove the added hook by calling
         "handle.remove()"
      Return type:
         "torch.utils.hooks.RemoveableHandle"
   register_step_pre_hook(hook)
      Register an optimizer step pre hook which will be called before
      optimizer step. It should have the following signature:
         hook(optimizer, args, kwargs) -> None or modified args and kwargs


The "optimizer" argument is the optimizer instance being used.
      If args and kwargs are modified by the pre-hook, then the
      transformed values are returned as a tuple containing the
      new_args and new_kwargs.
      Parameters:
         hook (Callable) -- The user defined hook to be
         registered.
      Returns:
         a handle that can be used to remove the added hook by calling
         "handle.remove()"
      Return type:
         "torch.utils.hooks.RemoveableHandle"
   state_dict()
      Returns the state of the optimizer as a "dict".
      It contains two entries:
      * state - a dict holding current optimization state. Its content
           differs between optimizer classes.
      * param_groups - a list containing all parameter groups where
        each
           parameter group is a dict
   zero_grad(set_to_none=False)
      Sets the gradients of all optimized "torch.Tensor" s to zero.
      Parameters:


Parameters:
         set_to_none (bool) -- instead of setting to zero, set
         the grads to None. This will in general have lower memory
         footprint, and can modestly improve performance. However, it
         changes certain behaviors. For example: 1. When the user
         tries to access a gradient and perform manual ops on it, a
         None attribute or a Tensor full of 0s will behave
         differently. 2. If the user requests
         "zero_grad(set_to_none=True)" followed by a backward pass,
         ".grad"s are guaranteed to be None for params that did not
         receive a gradient. 3. "torch.optim" optimizers have a
         different behavior if the gradient is 0 or None (in one case
         it does the step with a gradient of 0 and in the other it
         skips the step altogether).


torch.Tensor.cumminTensor.cummin(dim)
   See "torch.cummin()"


FuseCustomConfigclass torch.ao.quantization.fx.custom_config.FuseCustomConfig
   Custom configuration for "fuse_fx()".
   Example usage:
      fuse_custom_config = FuseCustomConfig().set_preserved_attributes(["attr1", "attr2"])
   classmethod from_dict(fuse_custom_config_dict)
      Create a "ConvertCustomConfig" from a dictionary with the
      following items:
         "preserved_attributes": a list of attributes that persist
         even if they are not used in "forward"
      This function is primarily for backward compatibility and may be
      removed in the future.
      Return type:
         FuseCustomConfig
   set_preserved_attributes(attributes)
      Set the names of the attributes that will persist in the graph
      module even if they are not used in the model's "forward"
      method.
      Return type:
         FuseCustomConfig
   to_dict()
      Convert this "FuseCustomConfig" to a dictionary with the items
      described in "from_dict()".
      Return type:


Return type:
         Dict[str, Any]


Adamclass torch.optim.Adam(params, lr=0.001, betas=(0.9, 0.999), eps=1e-08, weight_decay=0, amsgrad=False, *, foreach=None, maximize=False, capturable=False, differentiable=False, fused=None)
   Implements Adam algorithm.
      \begin{aligned}      &\rule{110mm}{0.4pt}
      \      &\textbf{input}      : \gamma \text{ (lr)}, \beta_1,
      \beta_2          \text{ (betas)},\theta_0 \text{
      (params)},f(\theta) \text{ (objective)}          \
      &\hspace{13mm}      \lambda \text{ (weight decay)},  \:
      \textit{amsgrad},          \:\textit{maximize}
      \      &\textbf{initialize} :  m_0 \leftarrow 0 \text{ ( first
      moment)},          v_0\leftarrow 0 \text{ (second moment)},\:
      \widehat{v_0}^{max}\leftarrow 0\[-1.ex]
      &\rule{110mm}{0.4pt}
      \      &\textbf{for} \: t=1 \: \textbf{to} \: \ldots \:
      \textbf{do}                         \
      &\hspace{5mm}\textbf{if} \: \textit{maximize}:
      \      &\hspace{10mm}g_t           \leftarrow


-\nabla_{\theta} f_t (\theta_{t-1})         \
      &\hspace{5mm}\textbf{else}
      \      &\hspace{10mm}g_t           \leftarrow   \nabla_{\theta}
      f_t (\theta_{t-1})          \      &\hspace{5mm}\textbf{if} \:
      \lambda \neq 0                                           \
      &\hspace{10mm} g_t \leftarrow g_t + \lambda  \theta_{t-1}
      \      &\hspace{5mm}m_t           \leftarrow   \beta_1 m_{t-1}
      + (1 - \beta_1) g_t          \      &\hspace{5mm}v_t
      \leftarrow   \beta_2 v_{t-1} + (1-\beta_2) g^2_t          \
      &\hspace{5mm}\widehat{m_t} \leftarrow   m_t/\big(1-\beta_1^t
      \big)                   \      &\hspace{5mm}\widehat{v_t}
      \leftarrow   v_t/\big(1-\beta_2^t \big)                   \
      &\hspace{5mm}\textbf{if} \: amsgrad
      \      &\hspace{10mm}\widehat{v_t}^{max} \leftarrow
      \mathrm{max}(\widehat{v_t}^{max},          \widehat{v_t})
      \      &\hspace{10mm}\theta_t \leftarrow \theta_{t-1} - \gamma


\widehat{m_t}/          \big(\sqrt{\widehat{v_t}^{max}} +
      \epsilon \big)                                 \
      &\hspace{5mm}\textbf{else}
      \      &\hspace{10mm}\theta_t \leftarrow \theta_{t-1} - \gamma
      \widehat{m_t}/          \big(\sqrt{\widehat{v_t}} + \epsilon
      \big)                                       \
      &\rule{110mm}{0.4pt}
      \[-1.ex]      &\bf{return} \:  \theta_t
      \[-1.ex]      &\rule{110mm}{0.4pt}
      \[-1.ex] \end{aligned}
   For further details regarding the algorithm we refer to Adam: A
   Method for Stochastic Optimization.
   Parameters:
      * params (iterable) -- iterable of parameters to optimize
        or dicts defining parameter groups
      * lr (float, optional) -- learning rate (default:
        1e-3)
      * betas (Tuple[float, float], optional) --
        coefficients used for computing running averages of gradient
        and its square (default: (0.9, 0.999))


and its square (default: (0.9, 0.999))
      * eps (float, optional) -- term added to the
        denominator to improve numerical stability (default: 1e-8)
      * weight_decay (float, optional) -- weight decay (L2
        penalty) (default: 0)
      * amsgrad (bool, optional) -- whether to use the
        AMSGrad variant of this algorithm from the paper On the
        Convergence of Adam and Beyond (default: False)
      * foreach (bool, optional) -- whether foreach
        implementation of optimizer is used (default: None)
      * maximize (bool, optional) -- maximize the params
        based on the objective, instead of minimizing (default: False)
      * capturable (bool, optional) -- whether this instance
        is safe to capture in a CUDA graph. Passing True can impair
        ungraphed performance, so if you don't intend to graph capture
        this instance, leave it False (default: False)


differentiable (bool, optional) -- whether autograd
        should occur through the optimizer step in training.
        Otherwise, the step() function runs in a torch.no_grad()
        context. Setting to True can impair performance, so leave it
        False if you don't intend to run autograd through this
        instance (default: False)
fused (bool, optional) -- whether the fused
    implementation (CUDA only) is used. Currently,
    torch.float64, torch.float32, torch.float16, and
    torch.bfloat16 are supported. Since the fused implementation
    is usually significantly faster than the for-loop
    implementation, we try to use it whenever possible (all
    parameters are on CUDA and are of a supported type). Else, we
    continue with the for-loop implementation. (default: None)
   add_param_group(param_group)
  Add a param group to the "Optimizer" s param_groups.


This can be useful when fine tuning a pre-trained network as
      frozen layers can be made trainable and added to the "Optimizer"
      as training progresses.
      Parameters:
         param_group (dict) -- Specifies what Tensors should be
         optimized along with group specific optimization options.
   load_state_dict(state_dict)
      Loads the optimizer state.
      Parameters:
         state_dict (dict) -- optimizer state. Should be an
         object returned from a call to "state_dict()".
   register_step_post_hook(hook)
      Register an optimizer step post hook which will be called after
      optimizer step. It should have the following signature:
         hook(optimizer, args, kwargs) -> None
      The "optimizer" argument is the optimizer instance being used.
      Parameters:
         hook (Callable) -- The user defined hook to be
         registered.
      Returns:
         a handle that can be used to remove the added hook by calling
         "handle.remove()"


"handle.remove()"
      Return type:
         "torch.utils.hooks.RemoveableHandle"
   register_step_pre_hook(hook)
      Register an optimizer step pre hook which will be called before
      optimizer step. It should have the following signature:
         hook(optimizer, args, kwargs) -> None or modified args and kwargs
      The "optimizer" argument is the optimizer instance being used.
      If args and kwargs are modified by the pre-hook, then the
      transformed values are returned as a tuple containing the
      new_args and new_kwargs.
      Parameters:
         hook (Callable) -- The user defined hook to be
         registered.
      Returns:
         a handle that can be used to remove the added hook by calling
         "handle.remove()"
      Return type:
         "torch.utils.hooks.RemoveableHandle"
   state_dict()
      Returns the state of the optimizer as a "dict".
      It contains two entries:
      * state - a dict holding current optimization state. Its content


differs between optimizer classes.
      * param_groups - a list containing all parameter groups where
        each
           parameter group is a dict
   zero_grad(set_to_none=False)
      Sets the gradients of all optimized "torch.Tensor" s to zero.
      Parameters:
         set_to_none (bool) -- instead of setting to zero, set
         the grads to None. This will in general have lower memory
         footprint, and can modestly improve performance. However, it
         changes certain behaviors. For example: 1. When the user
         tries to access a gradient and perform manual ops on it, a
         None attribute or a Tensor full of 0s will behave
         differently. 2. If the user requests
         "zero_grad(set_to_none=True)" followed by a backward pass,
         ".grad"s are guaranteed to be None for params that did not
         receive a gradient. 3. "torch.optim" optimizers have a
         different behavior if the gradient is 0 or None (in one case


it does the step with a gradient of 0 and in the other it
         skips the step altogether).


torch.nn.utils.skip_inittorch.nn.utils.skip_init(module_cls, args, kwargs)
   Given a module class object and args / kwargs, instantiates the
   module without initializing parameters / buffers.  This can be
   useful if initialization is slow or if custom initialization will
   be performed, making the default initialization unnecessary. There
   are some caveats to this, due to the way this function is
   implemented:
   1. The module must accept a device arg in its constructor that is
   passed to any parameters or buffers created during construction.
   2. The module must not perform any computation on parameters in its
   constructor except initialization (i.e. functions from
   "torch.nn.init").
   If these conditions are satisfied, the module can be instantiated
   with parameter / buffer values uninitialized, as if having been
   created using "torch.empty()".
   Parameters:
      * module_cls* -- Class object; should be a subclass of
        "torch.nn.Module"


"torch.nn.Module"
      * args -- args to pass to the module's constructor
      * kwargs -- kwargs to pass to the module's constructor
   Returns:
      Instantiated module with uninitialized parameters / buffers
   Example:
      >>> import torch
      >>> m = torch.nn.utils.skip_init(torch.nn.Linear, 5, 1)
      >>> m.weight
      Parameter containing:
      tensor([[0.0000e+00, 1.5846e+29, 7.8307e+00, 2.5250e-29, 1.1210e-44]],
             requires_grad=True)
      >>> m2 = torch.nn.utils.skip_init(torch.nn.Linear, in_features=6, out_features=1)
      >>> m2.weight
      Parameter containing:
      tensor([[-1.4677e+24,  4.5915e-41,  1.4013e-45,  0.0000e+00, -1.4677e+24,
                4.5915e-41]], requires_grad=True)


QConfigclass torch.quantization.qconfig.QConfig(activation, weight)
   Describes how to quantize a layer or a part of the network by
   providing settings (observer classes) for activations and weights
   respectively.
   Note that QConfig needs to contain observer classes (like
   MinMaxObserver) or a callable that returns instances on invocation,
   not the concrete observer instances themselves. Quantization
   preparation function will instantiate observers multiple times for
   each of the layers.
   Observer classes have usually reasonable default arguments, but
   they can be overwritten with with_args method (that behaves like
   functools.partial):
      my_qconfig = QConfig(
          activation=MinMaxObserver.with_args(dtype=torch.qint8),
          weight=default_observer.with_args(dtype=torch.qint8))


torch.nn.functional.gelutorch.nn.functional.gelu(input, approximate='none') -> Tensor
   When the approximate argument is 'none', it applies element-wise
   the function \text{GELU}(x) = x * \Phi(x)
   where \Phi(x) is the Cumulative Distribution Function for Gaussian
   Distribution.
   When the approximate argument is 'tanh', Gelu is estimated with
      \text{GELU}(x) = 0.5 * x * (1 + \text{Tanh}(\sqrt(2 / \pi) * (x
      + 0.044715 * x^3)))
   See Gaussian Error Linear Units (GELUs).


torch.linalg.ldl_factortorch.linalg.ldl_factor(A, , hermitian=False, out=None)
   Computes a compact representation of the LDL factorization of a
   Hermitian or symmetric (possibly indefinite) matrix.
   When "A" is complex valued it can be Hermitian ("hermitian"=
   True) or symmetric ("hermitian"= False).
   The factorization is of the form the form A = L D L^T. If
   "hermitian" is True* then transpose operation is the conjugate
   transpose.
   L (or U) and D are stored in compact form in "LD". They follow the
   format specified by LAPACK's sytrf function. These tensors may be
   used in "torch.linalg.ldl_solve()" to solve linear systems.
   Supports input of float, double, cfloat and cdouble dtypes. Also
   supports batches of matrices, and if "A" is a batch of matrices
   then the output has the same batch dimensions.
   Note:
     When inputs are on a CUDA device, this function synchronizes that
     device with the CPU. For a version of this function that does not


synchronize, see "torch.linalg.ldl_factor_ex()".
   Parameters:
      A (Tensor) -- tensor of shape (, n, n) where * is zero or
      more batch dimensions consisting of symmetric or Hermitian
      matrices. (*, n, n) where *** is one or more batch dimensions.
   Keyword Arguments:
      * hermitian (bool, optional) -- whether to consider
        the input to be Hermitian or symmetric. For real-valued
        matrices, this switch has no effect. Default: False.
      * out (tuple, optional) -- tuple of two tensors to
        write the output to. Ignored if None. Default: None.
   Returns:
      A named tuple (LD, pivots)*.
   Examples:
      >>> A = torch.randn(3, 3)
      >>> A = A @ A.mT # make symmetric
      >>> A
      tensor([[7.2079, 4.2414, 1.9428],
              [4.2414, 3.4554, 0.3264],
              [1.9428, 0.3264, 1.3823]])
      >>> LD, pivots = torch.linalg.ldl_factor(A)
      >>> LD
      tensor([[ 7.2079,  0.0000,  0.0000],


tensor([[ 7.2079,  0.0000,  0.0000],
              [ 0.5884,  0.9595,  0.0000],
              [ 0.2695, -0.8513,  0.1633]])
      >>> pivots
      tensor([1, 2, 3], dtype=torch.int32)


torch.cuda.change_current_allocatortorch.cuda.change_current_allocator(allocator)
   Changes the currently used memory allocator to be the one provided.
   If the current allocator has already been used/initialized, this
   function will error.
   Parameters:
      allocator (torch.cuda.memory._CUDAAllocator) -- allocator
      to be set as the active one.
   Note:
     See Memory management for details on creating and using a custom
     allocator


torch.bitwise_nottorch.bitwise_not(input, , out=None) -> Tensor
   Computes the bitwise NOT of the given input tensor. The input
   tensor must be of integral or Boolean types. For bool tensors, it
   computes the logical NOT.
   Parameters:
      input (Tensor) -- the input tensor.
   Keyword Arguments:
      out (Tensor, optional*) -- the output tensor.
   Example:
      >>> torch.bitwise_not(torch.tensor([-1, -2, 3], dtype=torch.int8))
      tensor([ 0,  1, -4], dtype=torch.int8)


torch.nn.functional.hardshrinktorch.nn.functional.hardshrink(input, lambd=0.5) -> Tensor
   Applies the hard shrinkage function element-wise
   See "Hardshrink" for more details.


torch.atleast_2dtorch.atleast_2d(tensors)
   Returns a 2-dimensional view of each input tensor with zero
   dimensions. Input tensors with two or more dimensions are returned
   as-is.
   Parameters:
      input (Tensor or list of Tensors*) --
   Returns:
      output (Tensor or tuple of Tensors)
   Example:
      >>> x = torch.tensor(1.)
      >>> x
      tensor(1.)
      >>> torch.atleast_2d(x)
      tensor([[1.]])
      >>> x = torch.arange(4).view(2, 2)
      >>> x
      tensor([[0, 1],
              [2, 3]])
      >>> torch.atleast_2d(x)
      tensor([[0, 1],
              [2, 3]])
      >>> x = torch.tensor(0.5)
      >>> y = torch.tensor(1.)
      >>> torch.atleast_2d((x, y))
      (tensor([[0.5000]]), tensor([[1.]]))


torch.nn.functional.dropout1dtorch.nn.functional.dropout1d(input, p=0.5, training=True, inplace=False)
   Randomly zero out entire channels (a channel is a 1D feature map,
   e.g., the j-th channel of the i-th sample in the batched input is a
   1D tensor \text{input}[i, j]) of the input tensor). Each channel
   will be zeroed out independently on every forward call with
   probability "p" using samples from a Bernoulli distribution.
   See "Dropout1d" for details.
   Parameters:
      * p (float) -- probability of a channel to be zeroed.
        Default: 0.5
      * training (bool) -- apply dropout if is "True". Default:
        "True"
      * inplace (bool) -- If set to "True", will do this
        operation in-place. Default: "False"
   Return type:
      Tensor


torch.signal.windows.exponentialtorch.signal.windows.exponential(M, , center=None, tau=1.0, sym=True, dtype=None, layout=torch.strided, device=None, requires_grad=False)
   Computes a window with an exponential waveform. Also known as
   Poisson window.
   The exponential window is defined as follows:
      w_n = \exp{\left(-\frac{|n - c|}{\tau}\right)}
   where c is the "center" of the window.
   The window is normalized to 1 (maximum value is 1). However, the 1
   doesn't appear if "M" is even and "sym" is True.
   Parameters:
      M (int) -- the length of the window. In other words, the
      number of points of the returned window.
   Keyword Arguments:
      * center (float, optional) -- where the center of the
        window will be located. Default: M / 2 if sym is False,
        else (M - 1) / 2.
      * tau (float, optional*) -- the decay value. Tau is
        generally associated with a percentage, that means, that the


value should vary within the interval (0, 100]. If tau is 100,
        it is considered the uniform window. Default: 1.0.
      * sym (bool, optional) -- If False, returns a
        periodic window suitable for use in spectral analysis. If
        True, returns a symmetric window suitable for use in filter
        design. Default: True.
      * dtype ("torch.dtype", optional) -- the desired data type
        of returned tensor. Default: if "None", uses a global default
        (see "torch.set_default_tensor_type()").
      * layout ("torch.layout", optional) -- the desired layout of
        returned Tensor. Default: "torch.strided".
      * device ("torch.device", optional) -- the desired device of
        returned tensor. Default: if "None", uses the current device
        for the default tensor type (see
        "torch.set_default_tensor_type()"). "device" will be the CPU
        for CPU tensor types and the current CUDA device for CUDA
        tensor types.


tensor types.
      * requires_grad (bool, optional) -- If autograd should
        record operations on the returned tensor. Default: "False".
   Return type:
      Tensor
   Examples:
      >>> # Generates a symmetric exponential window of size 10 and with a decay value of 1.0.
      >>> # The center will be at (M - 1) / 2, where M is 10.
      >>> torch.signal.windows.exponential(10)
      tensor([0.0111, 0.0302, 0.0821, 0.2231, 0.6065, 0.6065, 0.2231, 0.0821, 0.0302, 0.0111])
      >>> # Generates a periodic exponential window and decay factor equal to .5
      >>> torch.signal.windows.exponential(10, sym=False,tau=.5)
      tensor([4.5400e-05, 3.3546e-04, 2.4788e-03, 1.8316e-02, 1.3534e-01, 1.0000e+00, 1.3534e-01, 1.8316e-02, 2.4788e-03, 3.3546e-04])


torch.netorch.ne(input, other, , out=None) -> Tensor
   Computes \text{input} \neq \text{other} element-wise.
   The second argument can be a number or a tensor whose shape is
   broadcastable with the first argument.
   Parameters:
      * input (Tensor) -- the tensor to compare
      * other (Tensor or float) -- the tensor or value to
        compare
   Keyword Arguments:
      out (Tensor, optional*) -- the output tensor.
   Returns:
      A boolean tensor that is True where "input" is not equal to
      "other" and False elsewhere
   Example:
      >>> torch.ne(torch.tensor([[1, 2], [3, 4]]), torch.tensor([[1, 1], [4, 4]]))
      tensor([[False, True], [True, False]])


torch.Tensor.logcumsumexpTensor.logcumsumexp(dim) -> Tensor
   See "torch.logcumsumexp()"


default_activation_only_qconfigtorch.quantization.qconfig.default_activation_only_qconfig
   alias of QConfig(activation=functools.partial(,
   observer=,
   quant_min=0, quant_max=255, dtype=torch.quint8,
   qscheme=torch.per_tensor_affine, reduce_range=True){},
   weight=)


torch.set_float32_matmul_precisiontorch.set_float32_matmul_precision(precision)
   Sets the internal precision of float32 matrix multiplications.
   Running float32 matrix multiplications in lower precision may
   significantly increase performance, and in some programs the loss
   of precision has a negligible impact.
   Supports three settings:
      * "highest", float32 matrix multiplications use the float32
        datatype for internal computations.
      * "high", float32 matrix multiplications use the TensorFloat32
        or bfloat16_3x datatypes for internal computations, if fast
        matrix multiplication algorithms using those datatypes
        internally are available. Otherwise float32 matrix
        multiplications are computed as if the precision is "highest".
      * "medium", float32 matrix multiplications use the bfloat16
        datatype for internal computations, if a fast matrix
        multiplication algorithm using that datatype internally is


available. Otherwise float32 matrix multiplications are
        computed as if the precision is "high".
   Note:
     This does not change the output dtype of float32 matrix
     multiplications, it controls how the internal computation of the
     matrix multiplication is performed.
   Note:
     This does not change the precision of convolution operations.
     Other flags, like torch.backends.cudnn.allow_tf32, may control
     the precision of convolution operations.
   Note:
     This flag currently only affects one native device type: CUDA. If
     "high" or "medium" are set then the TensorFloat32 datatype will
     be used when computing float32 matrix multiplications, equivalent
     to setting torch.backends.cuda.matmul.allow_tf32 = True. When
     "highest" (the default) is set then the float32 datatype is used
     for internal computations, equivalent to setting
     torch.backends.cuda.matmul.allow_tf32 = False.
   Parameters:


Parameters:
      precision (str) -- can be set to "highest" (default),
      "high", or "medium" (see above).


Linearclass torch.ao.nn.qat.Linear(in_features, out_features, bias=True, qconfig=None, device=None, dtype=None)
   A linear module attached with FakeQuantize modules for weight, used
   for quantization aware training.
   We adopt the same interface as torch.nn.Linear, please see
   https://pytorch.org/docs/stable/nn.html#torch.nn.Linear for
   documentation.
   Similar to torch.nn.Linear, with FakeQuantize modules initialized
   to default.
   Variables:
      weight (torch.Tensor) -- fake quant module for weight
   classmethod from_float(mod)
      Create a qat module from a float module or qparams_dict Args:
      mod a float module, either produced by torch.ao.quantization
      utilities or directly from user


LazyModuleMixinclass torch.nn.modules.lazy.LazyModuleMixin(args, *kwargs)
   A mixin for modules that lazily initialize parameters, also known
   as "lazy modules."
   Modules that lazily initialize parameters, or "lazy modules",
   derive the shapes of their parameters from the first input(s) to
   their forward method. Until that first forward they contain
   "torch.nn.UninitializedParameter" s that should not be accessed or
   used, and afterward they contain regular "torch.nn.Parameter" s.
   Lazy modules are convenient since they don't require computing some
   module arguments, like the "in_features" argument of a typical
   "torch.nn.Linear".
   After construction, networks with lazy modules should first be
   converted to the desired dtype and placed on the expected device.
   This is because lazy modules only perform shape inference so the
   usual dtype and device placement behavior applies. The lazy modules
   should then perform "dry runs" to initialize all the components in


the module. These "dry runs" send inputs of the correct size,
   dtype, and device through the network and to each one of its lazy
   modules. After this the network can be used as usual.

class LazyMLP(torch.nn.Module):
   ...    def init(self):
   ...        super().init()
   ...        self.fc1 = torch.nn.LazyLinear(10)
   ...        self.relu1 = torch.nn.ReLU()
   ...        self.fc2 = torch.nn.LazyLinear(1)
   ...        self.relu2 = torch.nn.ReLU()
   ...
   ...    def forward(self, input):
   ...        x = self.relu1(self.fc1(input))
   ...        y = self.relu2(self.fc2(x))
   ...        return y
constructs a network with lazy modules
lazy_mlp = LazyMLP()
transforms the network's device and dtype
NOTE: these transforms can and should be applied after construction and before any 'dry runs'
lazy_mlp = lazy_mlp.cuda().double()
lazy_mlp
   LazyMLP( (fc1): LazyLinear(in_features=0, out_features=10, bias=True)
     (relu1): ReLU()


(relu1): ReLU()
     (fc2): LazyLinear(in_features=0, out_features=1, bias=True)
     (relu2): ReLU()
   )

performs a dry run to initialize the network's lazy modules
lazy_mlp(torch.ones(10,10).cuda())
after initialization, LazyLinear modules become regular Linear modules
lazy_mlp
   LazyMLP(
     (fc1): Linear(in_features=10, out_features=10, bias=True)
     (relu1): ReLU()
     (fc2): Linear(in_features=10, out_features=1, bias=True)
     (relu2): ReLU()
   )
attaches an optimizer, since parameters can now be used as usual
optim = torch.optim.SGD(mlp.parameters(), lr=0.01)
   A final caveat when using lazy modules is that the order of
   initialization of a network's parameters may change, since the lazy
   modules are always initialized after other modules. For example, if
   the LazyMLP class defined above had a "torch.nn.LazyLinear" module
   first and then a regular "torch.nn.Linear" second, the second


module would be initialized on construction and the first module
   would be initialized during the first dry run. This can cause the
   parameters of a network using lazy modules to be initialized
   differently than the parameters of a network without lazy modules
   as the order of parameter initializations, which often depends on a
   stateful random number generator, is different. Check
   Reproducibility for more details.
   Lazy modules can be serialized with a state dict like other
   modules. For example:

lazy_mlp = LazyMLP()
The state dict shows the uninitialized parameters
lazy_mlp.state_dict()
   OrderedDict([('fc1.weight', Uninitialized parameter),
                ('fc1.bias',
                 tensor([-1.8832e+25,  4.5636e-41, -1.8832e+25,  4.5636e-41, -6.1598e-30,
                          4.5637e-41, -1.8788e+22,  4.5636e-41, -2.0042e-31,  4.5637e-41])),
                ('fc2.weight', Uninitialized parameter),
                ('fc2.bias', tensor([0.0019]))])


('fc2.bias', tensor([0.0019]))])
   Lazy modules can load regular "torch.nn.Parameter" s (i.e. you can
   serialize/deserialize initialized LazyModules and they will remain
   initialized)

full_mlp = LazyMLP()
Dry run to initialize another module
full_mlp.forward(torch.ones(10, 1))
Load an initialized state into a lazy module
lazy_mlp.load_state_dict(full_mlp.state_dict())
The state dict now holds valid values
lazy_mlp.state_dict()
   OrderedDict([('fc1.weight',
                 tensor([[-0.3837],
                         [ 0.0907],
                         [ 0.6708],
                         [-0.5223],
                         [-0.9028],
                         [ 0.2851],
                         [-0.4537],
                         [ 0.6813],
                         [ 0.5766],
                         [-0.8678]])),
                ('fc1.bias',
                 tensor([-1.8832e+25,  4.5636e-41, -1.8832e+25,  4.5636e-41, -6.1598e-30,


4.5637e-41, -1.8788e+22,  4.5636e-41, -2.0042e-31,  4.5637e-41])),
                ('fc2.weight',
                 tensor([[ 0.1320,  0.2938,  0.0679,  0.2793,  0.1088, -0.1795, -0.2301,  0.2807,
                           0.2479,  0.1091]])),
                ('fc2.bias', tensor([0.0019]))])
   Note, however, that the loaded parameters will not be replaced when
   doing a "dry run" if they are initialized when the state is loaded.
   This prevents using initialized modules in different contexts.
   has_uninitialized_params()
      Check if a module has parameters that are not initialized
   initialize_parameters(args, *kwargs)
      Initialize parameters according to the input batch properties.
      This adds an interface to isolate parameter initialization from
      the forward pass when doing parameter shape inference.


torch.fft.rffttorch.fft.rfft(input, n=None, dim=- 1, norm=None, , out=None) -> Tensor
   Computes the one dimensional Fourier transform of real-valued
   "input".
   The FFT of a real signal is Hermitian-symmetric, "X[i] =
   conj(X[-i])" so the output contains only the positive frequencies
   below the Nyquist frequency. To compute the full output, use
   "fft()"
   Note:
     Supports torch.half on CUDA with GPU Architecture SM53 or
     greater. However it only supports powers of 2 signal length in
     every transformed dimension.
   Parameters:
      * input (Tensor) -- the real input tensor
      * n (int, optional) -- Signal length. If given, the
        input will either be zero-padded or trimmed to this length
        before computing the real FFT.
      * dim (int, optional) -- The dimension along which to
        take the one dimensional real FFT.
      * norm (str, optional*) --
        Normalization mode. For the forward transform ("rfft()"),


these correspond to:
        * ""forward"" - normalize by "1/n"
        * ""backward"" - no normalization
        * ""ortho"" - normalize by "1/sqrt(n)" (making the FFT
          orthonormal)
        Calling the backward transform ("irfft()") with the same
        normalization mode will apply an overall normalization of
        "1/n" between the two transforms. This is required to make
        "irfft()" the exact inverse.
        Default is ""backward"" (no normalization).
   Keyword Arguments:
      out (Tensor, optional) -- the output tensor.
   -[ Example ]-

t = torch.arange(4)
t
   tensor([0, 1, 2, 3])
torch.fft.rfft(t)
   tensor([ 6.+0.j, -2.+2.j, -2.+0.j])
   Compare against the full output from "fft()":
torch.fft.fft(t)
   tensor([ 6.+0.j, -2.+2.j, -2.+0.j, -2.-2.j])
   Notice that the symmetric element "T[-1] == T[1].conj()" is
   omitted. At the Nyquist frequency "T[-2] == T[2]" is it's own
   symmetric pair, and therefore must always be real-valued.


torch.nanmeantorch.nanmean(input, dim=None, keepdim=False, , dtype=None, out=None) -> Tensor
   Computes the mean of all non-NaN elements along the specified
   dimensions.
   This function is identical to "torch.mean()" when there are no
   NaN values in the "input" tensor. In the presence of NaN,
   "torch.mean()" will propagate the NaN to the output whereas
   "torch.nanmean()" will ignore the NaN values (torch.nanmean(a)
   is equivalent to torch.mean(a[~a.isnan()])).
   If "keepdim" is "True", the output tensor is of the same size as
   "input" except in the dimension(s) "dim" where it is of size 1.
   Otherwise, "dim" is squeezed (see "torch.squeeze()"), resulting in
   the output tensor having 1 (or "len(dim)") fewer dimension(s).
   Parameters:
      * input (Tensor) -- the input tensor.
      * dim (int or tuple of ints, optional*) -- the
        dimension or dimensions to reduce. If "None", all dimensions
        are reduced.


are reduced.
      * keepdim (bool) -- whether the output tensor has "dim"
        retained or not.
   Keyword Arguments:
      * dtype ("torch.dtype", optional) -- the desired data type
        of returned tensor. If specified, the input tensor is casted
        to "dtype" before the operation is performed. This is useful
        for preventing data type overflows. Default: None.
      * out (Tensor, optional) -- the output tensor.
   See also:
     "torch.mean()" computes the mean value, propagating NaN.
   Example:
      >>> x = torch.tensor([[torch.nan, 1, 2], [1, 2, 3]])
      >>> x.mean()
      tensor(nan)
      >>> x.nanmean()
      tensor(1.8000)
      >>> x.mean(dim=0)
      tensor([   nan, 1.5000, 2.5000])
      >>> x.nanmean(dim=0)
      tensor([1.0000, 1.5000, 2.5000])
      # If all elements in the reduced dimensions are NaN then the result is NaN
      >>> torch.tensor([torch.nan]).nanmean()
      tensor(nan)


Identityclass torch.nn.utils.prune.Identity
   Utility pruning method that does not prune any units but generates
   the pruning parametrization with a mask of ones.
   classmethod apply(module, name)
      Adds the forward pre-hook that enables pruning on the fly and
      the reparametrization of a tensor in terms of the original
      tensor and the pruning mask.
      Parameters:
         * module (nn.Module) -- module containing the tensor to
           prune
         * name (str) -- parameter name within "module" on which
           pruning will act.
   apply_mask(module)
      Simply handles the multiplication between the parameter being
      pruned and the generated mask. Fetches the mask and the original
      tensor from the module and returns the pruned version of the
      tensor.
      Parameters:
         module (nn.Module) -- module containing the tensor to
         prune
      Returns:
         pruned version of the input tensor
      Return type:


Return type:
         pruned_tensor (torch.Tensor)
   prune(t, default_mask=None, importance_scores=None)
      Computes and returns a pruned version of input tensor "t"
      according to the pruning rule specified in "compute_mask()".
      Parameters:
         * t (torch.Tensor) -- tensor to prune (of same
           dimensions as "default_mask").
         * importance_scores (torch.Tensor) -- tensor of
           importance scores (of same shape as "t") used to compute
           mask for pruning "t". The values in this tensor indicate
           the importance of the corresponding elements in the "t"
           that is being pruned. If unspecified or None, the tensor
           "t" will be used in its place.
         * default_mask (torch.Tensor, optional) -- mask
           from previous pruning iteration, if any. To be considered
           when determining what portion of the tensor that pruning
           should act on. If None, default to a mask of ones.
      Returns:


Returns:
         pruned version of tensor "t".
   remove(module)
      Removes the pruning reparameterization from a module. The pruned
      parameter named "name" remains permanently pruned, and the
      parameter named "name+'_orig'" is removed from the parameter
      list. Similarly, the buffer named "name+'_mask'" is removed from
      the buffers.
      Note:
        Pruning itself is NOT undone or reversed!


torch.kthvaluetorch.kthvalue(input, k, dim=None, keepdim=False, , out=None)
   Returns a namedtuple "(values, indices)" where "values" is the "k"
   th smallest element of each row of the "input" tensor in the given
   dimension "dim". And "indices" is the index location of each
   element found.
   If "dim" is not given, the last dimension of the input is chosen.
   If "keepdim" is "True", both the "values" and "indices" tensors are
   the same size as "input", except in the dimension "dim" where they
   are of size 1. Otherwise, "dim" is squeezed (see
   "torch.squeeze()"), resulting in both the "values" and "indices"
   tensors having 1 fewer dimension than the "input" tensor.
   Note:
     When "input" is a CUDA tensor and there are multiple valid "k" th
     values, this function may nondeterministically return "indices"
     for any of them.
   Parameters:
      * input (Tensor) -- the input tensor.
      * k (int*) -- k for the k-th smallest element


dim (int, optional) -- the dimension to find the kth
        value along
keepdim (bool) -- whether the output tensor has "dim"
    retained or not.
   Keyword Arguments:
  out (tuple, optional) -- the output tuple of (Tensor,
  LongTensor) can be optionally given to be used as output buffers
   Example:

x = torch.arange(1., 6.)
x
  tensor([ 1.,  2.,  3.,  4.,  5.])
torch.kthvalue(x, 4)
  torch.return_types.kthvalue(values=tensor(4.), indices=tensor(3))
x=torch.arange(1.,7.).resize_(2,3)
x
  tensor([[ 1.,  2.,  3.],
          [ 4.,  5.,  6.]])
torch.kthvalue(x, 2, 0, True)
  torch.return_types.kthvalue(values=tensor([[4., 5., 6.]]), indices=tensor([[1, 1, 1]]))


torch.foreach_sinh_torch._foreach_sinh(self: List[Tensor]) -> None
   Apply "torch.sinh()" to each Tensor of the input list.


torch.Tensor.nanmedianTensor.nanmedian(dim=None, keepdim=False)
   See "torch.nanmedian()"


torch.Tensor.fix_Tensor.fix_() -> Tensor
   In-place version of "fix()"


torch.Tensor.nonzeroTensor.nonzero() -> LongTensor
   See "torch.nonzero()"


interpolateclass torch.ao.nn.quantized.functional.interpolate(input, size=None, scale_factor=None, mode='nearest', align_corners=None)
   Down/up samples the input to either the given "size" or the given
   "scale_factor"
   See "torch.nn.functional.interpolate()" for implementation details.
   The input dimensions are interpreted in the form: mini-batch x
   channels x [optional depth] x [optional height] x width.
   Note:
     The input quantization parameters propagate to the output.
   Note:
     Only 2D/3D input is supported for quantized inputs
   Note:
     Only the following modes are supported for the quantized inputs:
     * bilinear
     * nearest
   Parameters:
      * input (Tensor) -- the input tensor
      * size (int or Tuple[int] or Tuple[int,
        int] or Tuple[int, int, int]) -- output
        spatial size.
      * scale_factor (float or Tuple[float]) --


multiplier for spatial size. Has to match input size if it is
        a tuple.
      * mode (str) -- algorithm used for upsampling: "'nearest'"
        | "'bilinear'"
      * align_corners (bool, optional) -- Geometrically, we
        consider the pixels of the input and output as squares rather
        than points. If set to "True", the input and output tensors
        are aligned by the center points of their corner pixels,
        preserving the values at the corner pixels. If set to "False",
        the input and output tensors are aligned by the corner points
        of their corner pixels, and the interpolation uses edge value
        padding for out-of-boundary values, making this operation
        independent of input size when "scale_factor" is kept the
        same. This only has an effect when "mode" is "'bilinear'".
        Default: "False"


torch.multorch.mul(input, other, , out=None) -> Tensor
   Multiplies "input" by "other".
      \text{out}_i = \text{input}_i \times \text{other}_i
   Supports broadcasting to a common shape, type promotion, and
   integer, float, and complex inputs.
   Parameters:
      * input (Tensor) -- the input tensor.
      * other (Tensor or Number) --
   Keyword Arguments:
      out (Tensor, optional*) -- the output tensor.
   Examples:
      >>> a = torch.randn(3)
      >>> a
      tensor([ 0.2015, -0.4255,  2.6087])
      >>> torch.mul(a, 100)
      tensor([  20.1494,  -42.5491,  260.8663])
      >>> b = torch.randn(4, 1)
      >>> b
      tensor([[ 1.1207],
              [-0.3137],
              [ 0.0700],
              [ 0.8378]])
      >>> c = torch.randn(1, 4)
      >>> c
      tensor([[ 0.5146,  0.1216, -0.5244,  2.2382]])
      >>> torch.mul(b, c)
      tensor([[ 0.5767,  0.1363, -0.5877,  2.5083],
              [-0.1614, -0.0382,  0.1645, -0.7021],


[ 0.0360,  0.0085, -0.0367,  0.1567],
              [ 0.4312,  0.1019, -0.4394,  1.8753]])


torch.nn.functional.adaptive_avg_pool3dtorch.nn.functional.adaptive_avg_pool3d(input, output_size)
   Applies a 3D adaptive average pooling over an input signal composed
   of several input planes.
   See "AdaptiveAvgPool3d" for details and output shape.
   Parameters:
      output_size (None) -- the target output size (single
      integer or triple-integer tuple)
   Return type:
      Tensor


torch.use_deterministic_algorithmstorch.use_deterministic_algorithms(mode, *, warn_only=False)
   Sets whether PyTorch operations must use "deterministic"
   algorithms. That is, algorithms which, given the same input, and
   when run on the same software and hardware, always produce the same
   output. When enabled, operations will use deterministic algorithms
   when available, and if only nondeterministic algorithms are
   available they will throw a "RuntimeError" when called.
   Note:
     This setting alone is not always enough to make an application
     reproducible. Refer to Reproducibility for more information.
   Note:
     "torch.set_deterministic_debug_mode()" offers an alternative
     interface for this feature.
   The following normally-nondeterministic operations will act
   deterministically when "mode=True":
      * "torch.nn.Conv1d" when called on CUDA tensor
      * "torch.nn.Conv2d" when called on CUDA tensor


"torch.nn.Conv3d" when called on CUDA tensor
"torch.nn.ConvTranspose1d" when called on CUDA tensor
"torch.nn.ConvTranspose2d" when called on CUDA tensor
"torch.nn.ConvTranspose3d" when called on CUDA tensor
"torch.bmm()" when called on sparse-dense CUDA tensors
"torch.Tensor.getitem()" when attempting to differentiate
    a CPU tensor and the index is a list of tensors
"torch.Tensor.index_put()" with "accumulate=False"
"torch.Tensor.index_put()" with "accumulate=True" when called
    on a CPU tensor
"torch.Tensor.put_()" with "accumulate=True" when called on a
    CPU tensor
"torch.Tensor.scatter_add_()" when called on a CUDA tensor
"torch.gather()" when called on a CUDA tensor that requires
    grad
"torch.index_add()" when called on CUDA tensor
"torch.index_select()" when attempting to differentiate a CUDA
    tensor


tensor
      * "torch.repeat_interleave()" when attempting to differentiate a
        CUDA tensor
      * "torch.Tensor.index_copy()" when called on a CPU or CUDA
        tensor
   The following normally-nondeterministic operations will throw a
   "RuntimeError" when "mode=True":
      * "torch.nn.AvgPool3d" when attempting to differentiate a CUDA
        tensor
      * "torch.nn.AdaptiveAvgPool2d" when attempting to differentiate
        a CUDA tensor
      * "torch.nn.AdaptiveAvgPool3d" when attempting to differentiate
        a CUDA tensor
      * "torch.nn.MaxPool3d" when attempting to differentiate a CUDA
        tensor
      * "torch.nn.AdaptiveMaxPool2d" when attempting to differentiate
        a CUDA tensor
      * "torch.nn.FractionalMaxPool2d" when attempting to
        differentiate a CUDA tensor
      * "torch.nn.FractionalMaxPool3d" when attempting to
        differentiate a CUDA tensor
      * "torch.nn.MaxUnpool1d"
      * "torch.nn.MaxUnpool2d"
      * "torch.nn.MaxUnpool3d"


"torch.nn.MaxUnpool3d"
"torch.nn.functional.interpolate()" when attempting to
    differentiate a CUDA tensor and one of the following modes is
    used:
"linear"
"bilinear"
"bicubic"
"trilinear"

"torch.nn.ReflectionPad1d" when attempting to differentiate a
    CUDA tensor
"torch.nn.ReflectionPad2d" when attempting to differentiate a
    CUDA tensor
"torch.nn.ReflectionPad3d" when attempting to differentiate a
    CUDA tensor
"torch.nn.ReplicationPad1d" when attempting to differentiate a
    CUDA tensor
"torch.nn.ReplicationPad2d" when attempting to differentiate a
    CUDA tensor
"torch.nn.ReplicationPad3d" when attempting to differentiate a
    CUDA tensor
"torch.nn.NLLLoss" when called on a CUDA tensor
"torch.nn.CTCLoss" when attempting to differentiate a CUDA
    tensor
"torch.nn.EmbeddingBag" when attempting to differentiate a


CUDA tensor when "mode='max'"
      * "torch.Tensor.put_()" when "accumulate=False"
      * "torch.Tensor.put_()" when "accumulate=True" and called on a
        CUDA tensor
      * "torch.histc()" when called on a CUDA tensor
      * "torch.bincount()" when called on a CUDA tensor
      * "torch.kthvalue()" with called on a CUDA tensor
      * "torch.median()" with indices output when called on a CUDA
        tensor
      * "torch.nn.functional.grid_sample()" when attempting to
        differentiate a CUDA tensor
      * "torch.cumsum()" when called on a CUDA tensor when dtype is
        floating point or complex
   A handful of CUDA operations are nondeterministic if the CUDA
   version is 10.2 or greater, unless the environment variable
   "CUBLAS_WORKSPACE_CONFIG=:4096:8" or
   "CUBLAS_WORKSPACE_CONFIG=:16:8" is set. See the CUDA documentation
   for more details: https://docs.nvidia.com/cuda/cublas/index.html#c
   ublasApi_reproducibility If one of these environment variable


configurations is not set, a "RuntimeError" will be raised from
   these operations when called with CUDA tensors:
      * "torch.mm()"
      * "torch.mv()"
      * "torch.bmm()"
   Note that deterministic operations tend to have worse performance
   than nondeterministic operations.
   Note:
     This flag does not detect or prevent nondeterministic behavior
     caused by calling an inplace operation on a tensor with an
     internal memory overlap or by giving such a tensor as the "out"
     argument for an operation. In these cases, multiple writes of
     different data may target a single memory location, and the order
     of writes is not guaranteed.
   Parameters:
      mode ("bool") -- If True, makes potentially nondeterministic
      operations switch to a deterministic algorithm or throw a
      runtime error. If False, allows nondeterministic operations.
   Keyword Arguments:
      warn_only ("bool", optional) -- If True, operations that do


not have a deterministic implementation will throw a warning
      instead of an error. Default: "False"
   Example:
      >>> torch.use_deterministic_algorithms(True)
      # Forward mode nondeterministic error
      >>> torch.randn(10, device='cuda').kthvalue(0)
      ...
      RuntimeError: kthvalue CUDA does not have a deterministic implementation...
      # Backward mode nondeterministic error
      >>> torch.nn.AvgPool3d(1)(torch.randn(3, 4, 5, 6, requires_grad=True).cuda()).sum().backward()
      ...
      RuntimeError: avg_pool3d_backward_cuda does not have a deterministic implementation...


torch.as_stridedtorch.as_strided(input, size, stride, storage_offset=None) -> Tensor
   Create a view of an existing torch.Tensor "input" with specified
   "size", "stride" and "storage_offset".
   Warning:
     Prefer using other view functions, like "torch.Tensor.expand()",
     to setting a view's strides manually with as_strided, as this
     function's behavior depends on the implementation of a tensor's
     storage. The constructed view of the storage must only refer to
     elements within the storage or a runtime error will be thrown,
     and if the view is "overlapped" (with multiple indices referring
     to the same element in memory) its behavior is undefined.
   Parameters:
      * input (Tensor) -- the input tensor.
      * size (tuple or ints) -- the shape of the output
        tensor
      * stride (tuple or ints) -- the stride of the output
        tensor
      * storage_offset (int, optional) -- the offset in the


underlying storage of the output tensor. If "None", the
        storage_offset of the output tensor will match the input
        tensor.
   Example:
      >>> x = torch.randn(3, 3)
      >>> x
      tensor([[ 0.9039,  0.6291,  1.0795],
              [ 0.1586,  2.1939, -0.4900],
              [-0.1909, -0.7503,  1.9355]])
      >>> t = torch.as_strided(x, (2, 2), (1, 2))
      >>> t
      tensor([[0.9039, 1.0795],
              [0.6291, 0.1586]])
      >>> t = torch.as_strided(x, (2, 2), (1, 2), 1)
      tensor([[0.6291, 0.1586],
              [1.0795, 2.1939]])


torch.einsumtorch.einsum(equation, operands) -> Tensor
   Sums the product of the elements of the input "operands" along
   dimensions specified using a notation based on the Einstein
   summation convention.
   Einsum allows computing many common multi-dimensional linear
   algebraic array operations by representing them in a short-hand
   format based on the Einstein summation convention, given by
   "equation". The details of this format are described below, but the
   general idea is to label every dimension of the input "operands"
   with some subscript and define which subscripts are part of the
   output. The output is then computed by summing the product of the
   elements of the "operands" along the dimensions whose subscripts
   are not part of the output. For example, matrix multiplication can
   be computed using einsum as torch.einsum("ij,jk->ik", A, B)*.
   Here, j is the summation subscript and i and k the output
   subscripts (see section below for more details on why).


Equation:
      The "equation" string specifies the subscripts (letters in
      [a-zA-Z]) for each dimension of the input "operands" in the
      same order as the dimensions, separating subscripts for each
      operand by a comma (','), e.g. 'ij,jk' specify subscripts for
      two 2D operands. The dimensions labeled with the same subscript
      must be broadcastable, that is, their size must either match or
      be 1. The exception is if a subscript is repeated for the same
      input operand, in which case the dimensions labeled with this
      subscript for this operand must match in size and the operand
      will be replaced by its diagonal along these dimensions. The
      subscripts that appear exactly once in the "equation" will be
      part of the output, sorted in increasing alphabetical order. The
      output is computed by multiplying the input "operands" element-
      wise, with their dimensions aligned based on the subscripts, and


then summing out the dimensions whose subscripts are not part of
      the output.
      Optionally, the output subscripts can be explicitly defined by
      adding an arrow ('->') at the end of the equation followed by
      the subscripts for the output. For instance, the following
      equation computes the transpose of a matrix multiplication:
      'ij,jk->ki'. The output subscripts must appear at least once for
      some input operand and at most once for the output.
      Ellipsis ('...') can be used in place of subscripts to broadcast
      the dimensions covered by the ellipsis. Each input operand may
      contain at most one ellipsis which will cover the dimensions not
      covered by subscripts, e.g. for an input operand with 5
      dimensions, the ellipsis in the equation 'ab...c' cover the
      third and fourth dimensions. The ellipsis does not need to cover
      the same number of dimensions across the "operands" but the


'shape' of the ellipsis (the size of the dimensions covered by
      them) must broadcast together. If the output is not explicitly
      defined with the arrow ('->') notation, the ellipsis will come
      first in the output (left-most dimensions), before the subscript
      labels that appear exactly once for the input operands. e.g. the
      following equation implements batch matrix multiplication
      '...ij,...jk'.
      A few final notes: the equation may contain whitespaces between
      the different elements (subscripts, ellipsis, arrow and comma)
      but something like '. . .' is not valid. An empty string ''
      is valid for scalar operands.
   Note:
     "torch.einsum" handles ellipsis ('...') differently from NumPy in
     that it allows dimensions covered by the ellipsis to be summed
     over, that is, ellipsis are not required to be part of the
     output.
   Note:
     This function uses opt_einsum (https://optimized-


einsum.readthedocs.io/en/stable/) to speed up computation or to
     consume less memory by optimizing contraction order. This
     optimization occurs when there are at least three inputs, since
     the order does not matter otherwise. Note that finding the
     optimal path is an NP-hard problem, thus, opt_einsum relies on
     different heuristics to achieve near-optimal results. If
     opt_einsum is not available, the default order is to contract
     from left to right.To bypass this default behavior, add the
     following line to disable the usage of opt_einsum and skip path
     calculation: torch.backends.opt_einsum.enabled = FalseTo
     specify which strategy you'd like for opt_einsum to compute the
     contraction path, add the following line:
     torch.backends.opt_einsum.strategy = 'auto'. The default
     strategy is 'auto', and we also support 'greedy' and 'optimal'.
     Disclaimer that the runtime of 'optimal' is factorial in the


number of inputs! See more details in the opt_einsum
     documentation (https://optimized-
     einsum.readthedocs.io/en/stable/path_finding.html).
   Note:
     As of PyTorch 1.10 "torch.einsum()" also supports the sublist
     format (see examples below). In this format, subscripts for each
     operand are specified by sublists, list of integers in the range
     [0, 52). These sublists follow their operands, and an extra
     sublist can appear at the end of the input to specify the
     output's subscripts., e.g. torch.einsum(op1, sublist1, op2,
     sublist2, ..., [subslist_out]). Python's Ellipsis object may
     be provided in a sublist to enable broadcasting as described in
     the Equation section above.
   Parameters:
      * equation (str) -- The subscripts for the Einstein
        summation.
      * operands (List[Tensor]) -- The tensors to compute
        the Einstein summation of.
   Return type:
      Tensor
   Examples:
      >>> # trace


Tensor
   Examples:
      >>> # trace
      >>> torch.einsum('ii', torch.randn(4, 4))
      tensor(-1.2104)
      >>> # diagonal
      >>> torch.einsum('ii->i', torch.randn(4, 4))
      tensor([-0.1034,  0.7952, -0.2433,  0.4545])
      >>> # outer product
      >>> x = torch.randn(5)
      >>> y = torch.randn(4)
      >>> torch.einsum('i,j->ij', x, y)
      tensor([[ 0.1156, -0.2897, -0.3918,  0.4963],
              [-0.3744,  0.9381,  1.2685, -1.6070],
              [ 0.7208, -1.8058, -2.4419,  3.0936],
              [ 0.1713, -0.4291, -0.5802,  0.7350],
              [ 0.5704, -1.4290, -1.9323,  2.4480]])
      >>> # batch matrix multiplication
      >>> As = torch.randn(3, 2, 5)
      >>> Bs = torch.randn(3, 5, 4)
      >>> torch.einsum('bij,bjk->bik', As, Bs)
      tensor([[[-1.0564, -1.5904,  3.2023,  3.1271],
              [-1.6706, -0.8097, -0.8025, -2.1183]],
              [[ 4.2239,  0.3107, -0.5756, -0.2354],
              [-1.4558, -0.3460,  1.5087, -0.8530]],


[[ 2.8153,  1.8787, -4.3839, -1.2112],
              [ 0.3728, -2.1131,  0.0921,  0.8305]]])
      >>> # with sublist format and ellipsis
      >>> torch.einsum(As, [..., 0, 1], Bs, [..., 1, 2], [..., 0, 2])
      tensor([[[-1.0564, -1.5904,  3.2023,  3.1271],
              [-1.6706, -0.8097, -0.8025, -2.1183]],
              [[ 4.2239,  0.3107, -0.5756, -0.2354],
              [-1.4558, -0.3460,  1.5087, -0.8530]],
              [[ 2.8153,  1.8787, -4.3839, -1.2112],
              [ 0.3728, -2.1131,  0.0921,  0.8305]]])
      >>> # batch permute
      >>> A = torch.randn(2, 3, 4, 5)
      >>> torch.einsum('...ij->...ji', A).shape
      torch.Size([2, 3, 5, 4])
      >>> # equivalent to torch.nn.functional.bilinear
      >>> A = torch.randn(3, 5, 4)
      >>> l = torch.randn(2, 5)
      >>> r = torch.randn(2, 4)
      >>> torch.einsum('bn,anm,bm->ba', l, A, r)
      tensor([[-0.3430, -5.2405,  0.4494],
              [ 0.3311,  5.5201, -3.0356]])


torch.less_equaltorch.less_equal(input, other, *, out=None) -> Tensor
   Alias for "torch.le()".


torch.nn.functional.margin_ranking_losstorch.nn.functional.margin_ranking_loss(input1, input2, target, margin=0, size_average=None, reduce=None, reduction='mean') -> Tensor
   See "MarginRankingLoss" for details.
   Return type:
      Tensor