torch.histogramddtorch.histogramdd(input, bins, *, range=None, weight=None, density=False, out=None) -> (Tensor, Tensor[])
   Computes a multi-dimensional histogram of the values in a tensor.
   Interprets the elements of an input tensor whose innermost
   dimension has size N as a collection of N-dimensional points. Maps
   each of the points into a set of N-dimensional bins and returns the
   number of points (or total weight) in each bin.
   "input" must be a tensor with at least 2 dimensions. If input has
   shape (M, N), each of its M rows defines a point in N-dimensional
   space. If input has three or more dimensions, all but the last
   dimension are flattened.
   Each dimension is independently associated with its own strictly
   increasing sequence of bin edges. Bin edges may be specified
   explicitly by passing a sequence of 1D tensors. Alternatively, bin
   edges may be constructed automatically by passing a sequence of


integers specifying the number of equal-width bins in each
   dimension.
   For each N-dimensional point in input:
      * Each of its coordinates is binned independently among the bin
        edges
           corresponding to its dimension
      * Binning results are combined to identify the N-dimensional bin
        (if any)
           into which the point falls
      * If the point falls into a bin, the bin's count (or total
        weight) is incremented
      * Points which do not fall into any bin do not contribute to the
        output
   "bins" can be a sequence of N 1D tensors, a sequence of N ints, or
   a single int.
   If "bins" is a sequence of N 1D tensors, it explicitly specifies
   the N sequences of bin edges. Each 1D tensor should contain a
   strictly increasing sequence with at least one element. A sequence
   of K bin edges defines K-1 bins, explicitly specifying the left and
   right edges of all bins. Every bin is exclusive of its left edge.


Only the rightmost bin is inclusive of its right edge.
   If "bins" is a sequence of N ints, it specifies the number of
   equal-width bins in each dimension. By default, the leftmost and
   rightmost bin edges in each dimension are determined by the minimum
   and maximum elements of the input tensor in the corresponding
   dimension. The "range" argument can be provided to manually specify
   the leftmost and rightmost bin edges in each dimension.
   If "bins" is an int, it specifies the number of equal-width bins
   for all dimensions.
   Note:
     See also "torch.histogram()", which specifically computes 1D
     histograms. While "torch.histogramdd()" infers the dimensionality
     of its bins and binned values from the shape of "input",
     "torch.histogram()" accepts and flattens "input" of any shape.
   Parameters:
      * input (Tensor) -- the input tensor.
      * bins -- Tensor[], int[], or int. If Tensor[], defines the


sequences of bin edges. If int[], defines the number of equal-
        width bins in each dimension. If int, defines the number of
        equal-width bins for all dimensions.
   Keyword Arguments:
      * range (sequence of python:float) -- Defines the leftmost
        and rightmost bin edges in each dimension.
      * weight (Tensor) -- By default, each value in the input
        has weight 1. If a weight tensor is passed, each N-dimensional
        coordinate in input contributes its associated weight towards
        its bin's result. The weight tensor should have the same shape
        as the "input" tensor excluding its innermost dimension N.
      * density (bool) -- If False (default), the result will
        contain the count (or total weight) in each bin. If True, each
        count (weight) is divided by the total count (total weight),
        then divided by the volume of its associated bin.
   Returns:
      N-dimensional Tensor containing the values of the histogram.


bin_edges(Tensor[]): sequence of N 1D Tensors containing the bin
      edges.
   Return type:
      hist (Tensor)
   Example::
      >>> torch.histogramdd(torch.tensor([[0., 1.], [1., 0.], [2., 0.], [2., 2.]]), bins=[3, 3],
      ...                   weight=torch.tensor([1., 2., 4., 8.]))
          torch.return_types.histogramdd(
              hist=tensor([[0., 1., 0.],
                           [2., 0., 0.],
                           [4., 0., 8.]]),
              bin_edges=(tensor([0.0000, 0.6667, 1.3333, 2.0000]),
                         tensor([0.0000, 0.6667, 1.3333, 2.0000])))
      >>> torch.histogramdd(torch.tensor([[0., 0.], [1., 1.], [2., 2.]]), bins=[2, 2],
      ...                   range=[0., 1., 0., 1.], density=True)
          torch.return_types.histogramdd(
             hist=tensor([[2., 0.],
                          [0., 2.]]),
             bin_edges=(tensor([0.0000, 0.5000, 1.0000]),
                        tensor([0.0000, 0.5000, 1.0000])))


torch.Tensor.logaddexpTensor.logaddexp(other) -> Tensor
   See "torch.logaddexp()"


torch.row_stacktorch.row_stack(tensors, *, out=None) -> Tensor
   Alias of "torch.vstack()".


torch.Tensor.is_conjTensor.is_conj() -> bool
   Returns True if the conjugate bit of "self" is set to true.


torch.emptytorch.empty(size, , out=None, dtype=None, layout=torch.strided, device=None, requires_grad=False, pin_memory=False, memory_format=torch.contiguous_format) -> Tensor
   Returns a tensor filled with uninitialized data. The shape of the
   tensor is defined by the variable argument "size".
   Parameters:
      size (int...) -- a sequence of integers defining the
      shape of the output tensor. Can be a variable number of
      arguments or a collection like a list or tuple.
   Keyword Arguments:
      * out (Tensor, optional) -- the output tensor.
      * dtype ("torch.dtype", optional) -- the desired data type
        of returned tensor. Default: if "None", uses a global default
        (see "torch.set_default_tensor_type()").
      * layout ("torch.layout", optional) -- the desired layout of
        returned Tensor. Default: "torch.strided".
      * device ("torch.device", optional) -- the desired device of


returned tensor. Default: if "None", uses the current device
        for the default tensor type (see
        "torch.set_default_tensor_type()"). "device" will be the CPU
        for CPU tensor types and the current CUDA device for CUDA
        tensor types.
      * requires_grad (bool, optional) -- If autograd should
        record operations on the returned tensor. Default: "False".
      * pin_memory (bool, optional) -- If set, returned
        tensor would be allocated in the pinned memory. Works only for
        CPU tensors. Default: "False".
      * memory_format ("torch.memory_format", optional) -- the
        desired memory format of returned Tensor. Default:
        "torch.contiguous_format".
   Example:
      >>> torch.empty((2,3), dtype=torch.int64)
      tensor([[ 9.4064e+13,  2.8000e+01,  9.3493e+13],
              [ 7.5751e+18,  7.1428e+18,  7.5955e+18]])


torch.nn.functional.elutorch.nn.functional.elu(input, alpha=1.0, inplace=False)
   Applies the Exponential Linear Unit (ELU) function element-wise.
   See "ELU" for more details.
   Return type:
      Tensor


ConvTranspose2dclass torch.ao.nn.quantized.ConvTranspose2d(in_channels, out_channels, kernel_size, stride=1, padding=0, output_padding=0, groups=1, bias=True, dilation=1, padding_mode='zeros', device=None, dtype=None)
   Applies a 2D transposed convolution operator over an input image
   composed of several input planes. For details on input arguments,
   parameters, and implementation see "ConvTranspose2d".
   For special notes, please, see "Conv2d"
   Variables:
      * weight (Tensor) -- packed tensor derived from the
        learnable weight parameter.
      * scale (Tensor) -- scalar for the output scale
      * zero_point (Tensor) -- scalar for the output zero point
   See "ConvTranspose2d" for other attributes.
   Examples:
      >>> # QNNPACK or FBGEMM as backend
      >>> torch.backends.quantized.engine = 'qnnpack'
      >>> # With square kernels and equal stride
      >>> import torch.nn.quantized as nnq
      >>> m = nnq.ConvTranspose2d(16, 33, 3, stride=2)


non-square kernels and unequal stride and with padding
  >>> m = nnq.ConvTranspose2d(16, 33, (3, 5), stride=(2, 1), padding=(4, 2))
  >>> input = torch.randn(20, 16, 50, 100)
  >>> q_input = torch.quantize_per_tensor(input, scale=1.0, zero_point=0, dtype=torch.quint8)
  >>> output = m(q_input)
  >>> # exact output size can be also specified as an argument
  >>> input = torch.randn(1, 16, 12, 12)
  >>> q_input = torch.quantize_per_tensor(input, scale=1.0, zero_point=0, dtype=torch.quint8)
  >>> downsample = nnq.Conv2d(16, 16, 3, stride=2, padding=1)
  >>> upsample = nnq.ConvTranspose2d(16, 16, 3, stride=2, padding=1)
  >>> h = downsample(q_input)
  >>> h.size()
  torch.Size([1, 16, 6, 6])
  >>> output = upsample(h, output_size=input.size())
  >>> output.size()
  torch.Size([1, 16, 12, 12])


torch.vdottorch.vdot(input, other, , out=None) -> Tensor
   Computes the dot product of two 1D vectors along a dimension.
   In symbols, this function computes
      \sum_{i=1}^n \overline{x_i}y_i.
   where \overline{x_i} denotes the conjugate for complex vectors, and
   it is the identity for real vectors.
   Note:
     Unlike NumPy's vdot, torch.vdot intentionally only supports
     computing the dot product of two 1D tensors with the same number
     of elements.
   See also:
     "torch.linalg.vecdot()" computes the dot product of two batches
     of vectors along a dimension.
   Parameters:
      * input (Tensor) -- first tensor in the dot product, must
        be 1D. Its conjugate is used if it's complex.
      * other (Tensor*) -- second tensor in the dot product, must
        be 1D.
   Keyword args:
   Note:
     out (Tensor, optional): the output tensor.
   Example:
      >>> torch.vdot(torch.tensor([2, 3]), torch.tensor([2, 1]))
      tensor(7)


tensor(7)
      >>> a = torch.tensor((1 +2j, 3 - 1j))
      >>> b = torch.tensor((2 +1j, 4 - 0j))
      >>> torch.vdot(a, b)
      tensor([16.+1.j])
      >>> torch.vdot(b, a)
      tensor([16.-1.j])


torch.nn.utils.vector_to_parameterstorch.nn.utils.vector_to_parameters(vec, parameters)
   Convert one vector to the parameters
   Parameters:
      * vec (Tensor) -- a single vector represents the
        parameters of a model.
      * parameters (Iterable[Tensor]) -- an iterator of
        Tensors that are the parameters of a model.


torch.svdtorch.svd(input, some=True, compute_uv=True, , out=None)
   Computes the singular value decomposition of either a matrix or
   batch of matrices "input". The singular value decomposition is
   represented as a namedtuple (U, S, V), such that "input" = U
   \text{diag}(S) V^{\text{H}}. where V^{\text{H}} is the transpose of
   V for real inputs, and the conjugate transpose of V for complex
   inputs. If "input" is a batch of matrices, then U, S, and V
   are also batched with the same batch dimensions as "input".
   If "some" is True (default), the method returns the reduced
   singular value decomposition. In this case, if the last two
   dimensions of "input" are m and n, then the returned U and
   V matrices will contain only min(n, m) orthonormal columns.
   If "compute_uv" is False, the returned U and V will be zero-
   filled matrices of shape (m, m) and (n, n)* respectively, and
   the same device as "input". The argument "some" has no effect when


"compute_uv" is False.
   Supports "input" of float, double, cfloat and cdouble data types.
   The dtypes of U and V are the same as "input"'s. S will
   always be real-valued, even if "input" is complex.
   Warning:
     "torch.svd()" is deprecated in favor of "torch.linalg.svd()" and
     will be removed in a future PyTorch release."U, S, V =
     torch.svd(A, some=some, compute_uv=True)" (default) should be
     replaced with
        U, S, Vh = torch.linalg.svd(A, full_matrices=not some)
        V = Vh.mH
     "_, S, _ = torch.svd(A, some=some, compute_uv=False)" should be
     replaced with
        S = torch.linalg.svdvals(A)
   Note:
     Differences with "torch.linalg.svd()":
     * "some" is the opposite of "torch.linalg.svd()"'s
       "full_matrices". Note that default value for both is True, so
       the default behavior is effectively the opposite.
     * "torch.svd()" returns V, whereas "torch.linalg.svd()" returns
       Vh, that is, V^{\text{H}}.


Vh, that is, V^{\text{H}}.
     * If "compute_uv" is False, "torch.svd()" returns zero-filled
       tensors for U and Vh, whereas "torch.linalg.svd()" returns
       empty tensors.
   Note:
     The singular values are returned in descending order. If "input"
     is a batch of matrices, then the singular values of each matrix
     in the batch are returned in descending order.
   Note:
     The S tensor can only be used to compute gradients if
     "compute_uv" is True.
   Note:
     When "some" is False, the gradients on U[..., :, min(m, n):]
     and V[..., :, min(m, n):] will be ignored in the backward pass,
     as those vectors can be arbitrary bases of the corresponding
     subspaces.
   Note:
     The implementation of "torch.linalg.svd()" on CPU uses LAPACK's
     routine ?gesdd (a divide-and-conquer algorithm) instead of
     ?gesvd for speed. Analogously, on GPU, it uses cuSOLVER's
     routines gesvdj and gesvdjBatched on CUDA 10.1.243 and later,


and MAGMA's routine gesdd on earlier versions of CUDA.
   Note:
     The returned U will not be contiguous. The matrix (or batch of
     matrices) will be represented as a column-major matrix (i.e.
     Fortran-contiguous).
   Warning:
     The gradients with respect to U and V will only be finite
     when the input does not have zero nor repeated singular values.
   Warning:
     If the distance between any two singular values is close to zero,
     the gradients with respect to U and V will be numerically
     unstable, as they depends on \frac{1}{\min_{i \neq j} \sigma_i^2
     - \sigma_j^2}. The same happens when the matrix has small
     singular values, as these gradients also depend on Sâ»Â¹.
   Warning:
     For complex-valued "input" the singular value decomposition is
     not unique, as U and V may be multiplied by an arbitrary
     phase factor e^{i \phi} on every column. The same happens when
     "input" has repeated singular values, where one may multiply the


columns of the spanning subspace in U and V by a rotation
     matrix and the resulting vectors will span the same subspace.
     Different platforms, like NumPy, or inputs on different device
     types, may produce different U and V tensors.
   Parameters:
      * input (Tensor) -- the input tensor of size (, m, n)
        where *** is zero or more batch dimensions consisting of (m,
        n) matrices.
      * some (bool, optional) -- controls whether to compute
        the reduced or full decomposition, and consequently, the shape
        of returned U and V. Default: True.
      * compute_uv (bool, optional) -- controls whether to
        compute U and V. Default: True.
   Keyword Arguments:
      out (tuple, optional*) -- the output tuple of tensors
   Example:
      >>> a = torch.randn(5, 3)
      >>> a
      tensor([[ 0.2364, -0.7752,  0.6372],
              [ 1.7201,  0.7394, -0.0504],
              [-0.3371, -1.0584,  0.5296],


[-0.3371, -1.0584,  0.5296],
              [ 0.3550, -0.4022,  1.5569],
              [ 0.2445, -0.0158,  1.1414]])
      >>> u, s, v = torch.svd(a)
      >>> u
      tensor([[ 0.4027,  0.0287,  0.5434],
              [-0.1946,  0.8833,  0.3679],
              [ 0.4296, -0.2890,  0.5261],
              [ 0.6604,  0.2717, -0.2618],
              [ 0.4234,  0.2481, -0.4733]])
      >>> s
      tensor([2.3289, 2.0315, 0.7806])
      >>> v
      tensor([[-0.0199,  0.8766,  0.4809],
              [-0.5080,  0.4054, -0.7600],
              [ 0.8611,  0.2594, -0.4373]])
      >>> torch.dist(a, torch.mm(torch.mm(u, torch.diag(s)), v.t()))
      tensor(8.6531e-07)
      >>> a_big = torch.randn(7, 5, 3)
      >>> u, s, v = torch.svd(a_big)
      >>> torch.dist(a_big, torch.matmul(torch.matmul(u, torch.diag_embed(s)), v.mT))
      tensor(2.6503e-06)


RNNBaseclass torch.nn.RNNBase(mode, input_size, hidden_size, num_layers=1, bias=True, batch_first=False, dropout=0.0, bidirectional=False, proj_size=0, device=None, dtype=None)
   flatten_parameters()
      Resets parameter data pointer so that they can use faster code
      paths.
      Right now, this works only if the module is on the GPU and cuDNN
      is enabled. Otherwise, it's a no-op.


torch.tril_indicestorch.tril_indices(row, col, offset=0, *, dtype=torch.long, device='cpu', layout=torch.strided) -> Tensor
   Returns the indices of the lower triangular part of a "row"-by-
   "col" matrix in a 2-by-N Tensor, where the first row contains row
   coordinates of all indices and the second row contains column
   coordinates. Indices are ordered based on rows and then columns.
   The lower triangular part of the matrix is defined as the elements
   on and below the diagonal.
   The argument "offset" controls which diagonal to consider. If
   "offset" = 0, all elements on and below the main diagonal are
   retained. A positive value includes just as many diagonals above
   the main diagonal, and similarly a negative value excludes just as
   many diagonals below the main diagonal. The main diagonal are the
   set of indices \lbrace (i, i) \rbrace for i \in [0, \min{d_{1},
   d_{2}} - 1] where d_{1}, d_{2} are the dimensions of the matrix.
   Note:


Note:
     When running on CUDA, "row * col" must be less than 2^{59} to
     prevent overflow during calculation.
   Parameters:
      * row ("int") -- number of rows in the 2-D matrix.
      * col ("int") -- number of columns in the 2-D matrix.
      * offset ("int") -- diagonal offset from the main diagonal.
        Default: if not provided, 0.
   Keyword Arguments:
      * dtype ("torch.dtype", optional) -- the desired data type
        of returned tensor. Default: if "None", "torch.long".
      * device ("torch.device", optional) -- the desired device of
        returned tensor. Default: if "None", uses the current device
        for the default tensor type (see
        "torch.set_default_tensor_type()"). "device" will be the CPU
        for CPU tensor types and the current CUDA device for CUDA
        tensor types.
      * layout ("torch.layout", optional) -- currently only
        support "torch.strided".
   Example:
      >>> a = torch.tril_indices(3, 3)
      >>> a


a = torch.tril_indices(3, 3)
      >>> a
      tensor([[0, 1, 1, 2, 2, 2],
              [0, 0, 1, 0, 1, 2]])
      >>> a = torch.tril_indices(4, 3, -1)
      >>> a
      tensor([[1, 2, 2, 3, 3, 3],
              [0, 0, 1, 0, 1, 2]])
      >>> a = torch.tril_indices(4, 3, 1)
      >>> a
      tensor([[0, 0, 1, 1, 1, 2, 2, 2, 3, 3, 3],
              [0, 1, 0, 1, 2, 0, 1, 2, 0, 1, 2]])


torch.Tensor.copy_Tensor.copy_(src, non_blocking=False) -> Tensor
   Copies the elements from "src" into "self" tensor and returns
   "self".
   The "src" tensor must be broadcastable with the "self" tensor. It
   may be of a different data type or reside on a different device.
   Parameters:
      * src (Tensor) -- the source tensor to copy from
      * non_blocking (bool) -- if "True" and this copy is
        between CPU and GPU, the copy may occur asynchronously with
        respect to the host. For other cases, this argument has no
        effect.


torch.Tensor.chalfTensor.chalf(memory_format=torch.preserve_format) -> Tensor
   "self.chalf()" is equivalent to "self.to(torch.complex32)". See
   "to()".
   Parameters:
      memory_format ("torch.memory_format", optional) -- the
      desired memory format of returned Tensor. Default:
      "torch.preserve_format".


torch.nn.functional.fractional_max_pool3dtorch.nn.functional.fractional_max_pool3d(args, kwargs)
   Applies 3D fractional max pooling over an input signal composed of
   several input planes.
   Fractional MaxPooling is described in detail in the paper
   Fractional MaxPooling by Ben Graham
   The max-pooling operation is applied in kT \times kH \times kW
   regions by a stochastic step size determined by the target output
   size. The number of output features is equal to the number of input
   planes.
   Parameters:
      * kernel_size -- the size of the window to take a max over.
        Can be a single number k (for a square kernel of k \times k
        \times k) or a tuple (kT, kH, kW)
      * output_size -- the target output size of the form oT
        \times oH \times oW. Can be a tuple (oT, oH, oW) or a single
        number oH for a cubic output oH \times oH \times oH
      * output_ratio* -- If one wants to have an output size as a


ratio of the input size, this option can be given. This has to
        be a number or tuple in the range (0, 1)
      * return_indices -- if "True", will return the indices along
        with the outputs. Useful to pass to "max_unpool3d()".
   Shape:
      * Input: (N, C, T_{in}, H_{in}, W_{in}) or (C, T_{in}, H_{in},
        W_{in}).
      * Output: (N, C, T_{out}, H_{out}, W_{out}) or (C, T_{out},
        H_{out}, W_{out}), where (T_{out}, H_{out},
        W_{out})=\text{output_size} or (T_{out}, H_{out},
        W_{out})=\text{output_ratio} \times (T_{in}, H_{in}, W_{in})
   Examples::
      >>> input = torch.randn(20, 16, 50, 32, 16)
      >>> # pool of cubic window of size=3, and target output size 13x12x11
      >>> F.fractional_max_pool3d(input, 3, output_size=(13, 12, 11))
      >>> # pool of cubic window and target output size being half of input size
      >>> F.fractional_max_pool3d(input, 3, output_ratio=(0.5, 0.5, 0.5))


ConvBnReLU2dclass torch.ao.nn.intrinsic.qat.ConvBnReLU2d(in_channels, out_channels, kernel_size, stride=1, padding=0, dilation=1, groups=1, bias=None, padding_mode='zeros', eps=1e-05, momentum=0.1, freeze_bn=False, qconfig=None)
   A ConvBnReLU2d module is a module fused from Conv2d, BatchNorm2d
   and ReLU, attached with FakeQuantize modules for weight, used in
   quantization aware training.
   We combined the interface of "torch.nn.Conv2d" and
   "torch.nn.BatchNorm2d" and "torch.nn.ReLU".
   Similar to torch.nn.Conv2d, with FakeQuantize modules initialized
   to default.
   Variables:
      weight_fake_quant -- fake quant module for weight


ParametrizationListclass torch.nn.utils.parametrize.ParametrizationList(modules, original, unsafe=False)
   A sequential container that holds and manages the "original" or
   "original0", "original1", ... parameters or buffers of a
   parametrized "torch.nn.Module".
   It is the type of "module.parametrizations[tensor_name]" when
   "module[tensor_name]" has been parametrized with
   "register_parametrization()".
   If the first registered parametrization has a "right_inverse" that
   returns one tensor or does not have a "right_inverse" (in which
   case we assume that "right_inverse" is the identity), it will hold
   the tensor under the name "original". If it has a "right_inverse"
   that returns more than one tensor, these will be registered as
   "original0", "original1", ...
   Warning:
     This class is used internally by "register_parametrization()". It
     is documented here for completeness. It shall not be instantiated
     by the user.
   Parameters:


by the user.
   Parameters:
      * modules (sequence) -- sequence of modules representing
        the parametrizations
      * original (Parameter or Tensor) -- parameter or
        buffer that is parametrized
      * unsafe (bool) -- a boolean flag that denotes whether the
        parametrization may change the dtype and shape of the tensor.
        Default: False Warning: the parametrization is not checked
        for consistency upon registration. Enable this flag at your
        own risk.
   right_inverse(value)
      Calls the methods "right_inverse" (see
      "register_parametrization()") of the parametrizations in the
      inverse order they were registered in. Then, it stores the
      result in "self.original" if "right_inverse" outputs one tensor
      or in "self.original0", "self.original1", ... if it outputs
      several.
      Parameters:
         value (Tensor) -- Value to which initialize the module


torch.nn.functional.cosine_embedding_losstorch.nn.functional.cosine_embedding_loss(input1, input2, target, margin=0, size_average=None, reduce=None, reduction='mean') -> Tensor
   See "CosineEmbeddingLoss" for details.
   Return type:
      Tensor


torch._foreach_fractorch._foreach_frac(self: List[Tensor]) -> List[Tensor]
   Apply "torch.frac()" to each Tensor of the input list.


torch.stfttorch.stft(input, n_fft, hop_length=None, win_length=None, window=None, center=True, pad_mode='reflect', normalized=False, onesided=None, return_complex=None)
   Short-time Fourier transform (STFT).
   Warning:
     From version 1.8.0, "return_complex" must always be given
     explicitly for real inputs and return_complex=False has been
     deprecated. Strongly prefer return_complex=True as in a future
     pytorch release, this function will only return complex
     tensors.Note that "torch.view_as_real()" can be used to recover a
     real tensor with an extra last dimension for real and imaginary
     components.
   The STFT computes the Fourier transform of short overlapping
   windows of the input. This giving frequency components of the
   signal as they change over time. The interface of this function is
   modeled after (but not a drop-in replacement for) librosa stft
   function.
   Ignoring the optional batch dimension, this method computes the
   following expression:


following expression:
      X[\omega, m] = \sum_{k = 0}^{\text{win_length-1}}%
      \text{window}[k]\ \text{input}[m \times \text{hop_length} + k]\
      %                     \exp\left(- j \frac{2 \pi \cdot \omega
      k}{\text{win_length}}\right),
   where m is the index of the sliding window, and \omega is the
   frequency 0 \leq \omega < \text{n_fft} for "onesided=False", or 0
   \leq \omega < \lfloor \text{n_fft} / 2 \rfloor + 1 for
   "onesided=True".
   * "input" must be either a 1-D time sequence or a 2-D batch of time
     sequences.
   * If "hop_length" is "None" (default), it is treated as equal to
     "floor(n_fft / 4)".
   * If "win_length" is "None" (default), it is treated as equal to
     "n_fft".
   * "window" can be a 1-D tensor of size "win_length", e.g., from
     "torch.hann_window()". If "window" is "None" (default), it is
     treated as if having 1 everywhere in the window. If
     \text{win_length} < \text{n_fft}, "window" will be padded on


both sides to length "n_fft" before being applied.
   * If "center" is "True" (default), "input" will be padded on both
     sides so that the t-th frame is centered at time t \times
     \text{hop_length}. Otherwise, the t-th frame begins at time  t
     \times \text{hop_length}.
   * "pad_mode" determines the padding method used on "input" when
     "center" is "True". See "torch.nn.functional.pad()" for all
     available options. Default is ""reflect"".
   * If "onesided" is "True" (default for real input), only values for
     \omega in \left[0, 1, 2, \dots, \left\lfloor
     \frac{\text{n_fft}}{2} \right\rfloor + 1\right] are returned
     because the real-to-complex Fourier transform satisfies the
     conjugate symmetry, i.e., X[m, \omega] = X[m, \text{n_fft} -
     \omega]^*. Note if the input or window tensors are complex, then
     "onesided" output is not possible.
   * If "normalized" is "True" (default is "False"), the function


returns the normalized STFT results, i.e., multiplied by
     (\text{frame_length})^{-0.5}.
   * If "return_complex" is "True" (default if input is complex), the
     return is a "input.dim() + 1" dimensional complex tensor. If
     "False", the output is a "input.dim() + 2" dimensional real
     tensor where the last dimension represents the real and imaginary
     components.
   Returns either a complex tensor of size ( \times N \times T) if
   "return_complex" is true, or a real tensor of size ( \times N
   \times T \times 2). Where * is the optional batch size of "input",
   N is the number of frequencies where STFT is applied and T is the
   total number of frames used.
   Warning:
     This function changed signature at version 0.4.1. Calling with
     the previous signature may cause error or return incorrect
     result.
   Parameters:
      * input (Tensor) -- the input tensor
      * n_fft (int) -- size of Fourier transform


hop_length (int, optional) -- the distance between
        neighboring sliding window frames. Default: "None" (treated as
        equal to "floor(n_fft / 4)")
win_length (int, optional) -- the size of window
    frame and STFT filter. Default: "None"  (treated as equal to
    "n_fft")
window (Tensor, optional) -- the optional window
    function. Default: "None" (treated as window of all 1 s)
center (bool, optional) -- whether to pad "input" on
    both sides so that the t-th frame is centered at time t \times
    \text{hop_length}. Default: "True"
pad_mode (str, optional) -- controls the padding
    method used when "center" is "True". Default: ""reflect""
normalized (bool, optional) -- controls whether to
    return the normalized STFT results Default: "False"
onesided (bool, optional) -- controls whether to


return half of results to avoid redundancy for real inputs.
        Default: "True" for real "input" and "window", "False"
        otherwise.
      * return_complex (bool, optional) --
        whether to return a complex tensor, or a real tensor with an
        extra last dimension for the real and imaginary components.
        Changed in version 2.0: "return_complex" is now a required
        argument for real inputs, as the default is being transitioned
        to "True".
        Deprecated since version 2.0: "return_complex=False" is
        deprecated, instead use "return_complex=True" Note that
        calling "torch.view_as_real()" on the output will recover the
        deprecated output format.
   Returns:
      A tensor containing the STFT result with shape described above
   Return type:
      Tensor


upsample_nearestclass torch.ao.nn.quantized.functional.upsample_nearest(input, size=None, scale_factor=None)
   Upsamples the input, using nearest neighbours' pixel values.
   Warning:
     This function is deprecated in favor of
     "torch.nn.quantized.functional.interpolate()". This is equivalent
     with "nn.quantized.functional.interpolate(..., mode='nearest')".
   Note:
     The input quantization parameters propagate to the output.
   Note:
     Only 2D inputs are supported
   Parameters:
      * input (Tensor) -- quantized input
      * size (int or Tuple[int, int] or
        Tuple[int, int, int]) -- output spatial size.
      * scale_factor (int) -- multiplier for spatial size. Has
        to be an integer.


torch.addmmtorch.addmm(input, mat1, mat2, , beta=1, alpha=1, out=None) -> Tensor
   Performs a matrix multiplication of the matrices "mat1" and "mat2".
   The matrix "input" is added to the final result.
   If "mat1" is a (n \times m) tensor, "mat2" is a (m \times p)
   tensor, then "input" must be broadcastable with a (n \times p)
   tensor and "out" will be a (n \times p) tensor.
   "alpha" and "beta" are scaling factors on matrix-vector product
   between "mat1" and "mat2" and the added matrix "input"
   respectively.
      \text{out} = \beta\ \text{input} + \alpha\ (\text{mat1}_i
      \mathbin{@} \text{mat2}_i)
   If "beta" is 0, then "input" will be ignored, and nan and inf
   in it will not be propagated.
   For inputs of type FloatTensor or DoubleTensor*, arguments
   "beta" and "alpha" must be real numbers, otherwise they should be
   integers.
   This operation has support for arguments with sparse layouts. If


"input" is sparse the result will have the same layout and if "out"
   is provided it must have the same layout as "input".
   Warning:
     Sparse support is a beta feature and some layout(s)/dtype/device
     combinations may not be supported, or may not have autograd
     support. If you notice missing functionality please open a
     feature request.
   This operator supports TensorFloat32.
   On certain ROCm devices, when using float16 inputs this module will
   use different precision for backward.
   Parameters:
      * input (Tensor) -- matrix to be added
      * mat1 (Tensor) -- the first matrix to be matrix
        multiplied
      * mat2 (Tensor) -- the second matrix to be matrix
        multiplied
   Keyword Arguments:
      * beta (Number, optional) -- multiplier for "input"
        (\beta)
      * alpha (Number, optional) -- multiplier for mat1 @
        mat2 (\alpha)
      * out (Tensor, optional) -- the output tensor.
   Example:


Example:
      >>> M = torch.randn(2, 3)
      >>> mat1 = torch.randn(2, 3)
      >>> mat2 = torch.randn(3, 3)
      >>> torch.addmm(M, mat1, mat2)
      tensor([[-4.8716,  1.4671, -1.3746],
              [ 0.7573, -3.9555, -2.8681]])


torch.Tensor.subtract_Tensor.subtract_(other, *, alpha=1) -> Tensor
   In-place version of "subtract()".


torch.Tensor.arcsinTensor.arcsin() -> Tensor
   See "torch.arcsin()"


torch.quantized_max_pool2dtorch.quantized_max_pool2d(input, kernel_size, stride=[], padding=0, dilation=1, ceil_mode=False) -> Tensor
   Applies a 2D max pooling over an input quantized tensor composed of
   several input planes.
   Parameters:
      * input (Tensor) -- quantized tensor
      * kernel_size ("list of int") -- the size of the sliding
        window
      * stride ("list of int", optional) -- the stride of the
        sliding window
      * padding ("list of int", optional) -- padding to be added
        on both sides, must be >= 0 and <= kernel_size / 2
      * dilation ("list of int", optional) -- The stride between
        elements within a sliding window, must be > 0. Default 1
      * ceil_mode (bool, optional) -- If True, will use ceil
        instead of floor to compute the output shape. Defaults to
        False.
   Returns:
      A quantized tensor with max_pool2d applied.
   Return type:
      Tensor
   Example:


Return type:
      Tensor
   Example:
      >>> qx = torch.quantize_per_tensor(torch.rand(2, 2, 2, 2), 1.5, 3, torch.quint8)
      >>> torch.quantized_max_pool2d(qx, [2,2])
      tensor([[[[1.5000]],
              [[1.5000]]],
              [[[0.0000]],
              [[0.0000]]]], size=(2, 2, 1, 1), dtype=torch.quint8,
          quantization_scheme=torch.per_tensor_affine, scale=1.5, zero_point=3)


ReLUclass torch.nn.ReLU(inplace=False)
   Applies the rectified linear unit function element-wise:
   \text{ReLU}(x) = (x)^+ = \max(0, x)
   Parameters:
      inplace (bool) -- can optionally do the operation in-
      place. Default: "False"
   Shape:
      * Input: (), where * means any number of dimensions.
      * Output: (), same shape as the input.
   [image]
   Examples:
        >>> m = nn.ReLU()
        >>> input = torch.randn(2)
        >>> output = m(input)
      An implementation of CReLU - https://arxiv.org/abs/1603.05201
        >>> m = nn.ReLU()
        >>> input = torch.randn(2).unsqueeze(0)
        >>> output = torch.cat((m(input), m(-input)))


hardtanhclass torch.ao.nn.quantized.functional.hardtanh(input, min_val=- 1.0, max_val=1.0, inplace=False)
   This is the quantized version of "hardtanh()".
   Return type:
      Tensor


torch.Tensor.put_Tensor.put_(index, source, accumulate=False) -> Tensor
   Copies the elements from "source" into the positions specified by
   "index". For the purpose of indexing, the "self" tensor is treated
   as if it were a 1-D tensor.
   "index" and "source" need to have the same number of elements, but
   not necessarily the same shape.
   If "accumulate" is "True", the elements in "source" are added to
   "self". If accumulate is "False", the behavior is undefined if
   "index" contain duplicate elements.
   Parameters:
      * index (LongTensor) -- the indices into self
      * source (Tensor) -- the tensor containing values to copy
        from
      * accumulate (bool) -- whether to accumulate into self
   Example:
      >>> src = torch.tensor([[4, 3, 5],
      ...                     [6, 7, 8]])
      >>> src.put_(torch.tensor([1, 3]), torch.tensor([9, 10]))
      tensor([[  4,   9,   5],
              [ 10,   7,   8]])


torch._foreach_trunctorch._foreach_trunc(self: List[Tensor]) -> List[Tensor]
   Apply "torch.trunc()" to each Tensor of the input list.


torch.Tensor.acoshTensor.acosh() -> Tensor
   See "torch.acosh()"


torch.Tensor.backwardTensor.backward(gradient=None, retain_graph=None, create_graph=False, inputs=None)
   Computes the gradient of current tensor w.r.t. graph leaves.
   The graph is differentiated using the chain rule. If the tensor is
   non-scalar (i.e. its data has more than one element) and requires
   gradient, the function additionally requires specifying "gradient".
   It should be a tensor of matching type and location, that contains
   the gradient of the differentiated function w.r.t. "self".
   This function accumulates gradients in the leaves - you might need
   to zero ".grad" attributes or set them to "None" before calling it.
   See Default gradient layouts for details on the memory layout of
   accumulated gradients.
   Note:
     If you run any forward ops, create "gradient", and/or call
     "backward" in a user-specified CUDA stream context, see Stream
     semantics of backward passes.
   Note:
     When "inputs" are provided and a given input is not a leaf, the


current implementation will call its grad_fn (though it is not
     strictly needed to get this gradients). It is an implementation
     detail on which the user should not rely. See https://github.com
     /pytorch/pytorch/pull/60521#issuecomment-867061780 for more
     details.
   Parameters:
      * gradient (Tensor or None) -- Gradient w.r.t. the
        tensor. If it is a tensor, it will be automatically converted
        to a Tensor that does not require grad unless "create_graph"
        is True. None values can be specified for scalar Tensors or
        ones that don't require grad. If a None value would be
        acceptable then this argument is optional.
      * retain_graph (bool, optional) -- If "False", the
        graph used to compute the grads will be freed. Note that in
        nearly all cases setting this option to True is not needed and
        often can be worked around in a much more efficient way.
        Defaults to the value of "create_graph".


Defaults to the value of "create_graph".
      * create_graph (bool, optional) -- If "True", graph of
        the derivative will be constructed, allowing to compute higher
        order derivative products. Defaults to "False".
      * inputs (sequence of Tensor) -- Inputs w.r.t. which the
        gradient will be accumulated into ".grad". All other Tensors
        will be ignored. If not provided, the gradient is accumulated
        into all the leaf Tensors that were used to compute the
        attr::tensors.


torch.pinversetorch.pinverse(input, rcond=1e-15) -> Tensor
   Alias for "torch.linalg.pinv()"


torch.Tensor.reshape_asTensor.reshape_as(other) -> Tensor
   Returns this tensor as the same shape as "other".
   "self.reshape_as(other)" is equivalent to
   "self.reshape(other.sizes())". This method returns a view if
   "other.sizes()" is compatible with the current shape. See
   "torch.Tensor.view()" on when it is possible to return a view.
   Please see "reshape()" for more information about "reshape".
   Parameters:
      other ("torch.Tensor") -- The result tensor has the same
      shape as "other".


torch.nn.modules.module.register_module_forward_pre_hooktorch.nn.modules.module.register_module_forward_pre_hook(hook)
   Registers a forward pre-hook common to all modules.
   Warning:
     This adds global state to the nn.module module and it is only
     intended for debugging/profiling purposes.
   The hook will be called every time before "forward()" is invoked.
   It should have the following signature:
      hook(module, input) -> None or modified input
   The input contains only the positional arguments given to the
   module. Keyword arguments won't be passed to the hooks and only to
   the "forward". The hook can modify the input. User can either
   return a tuple or a single modified value in the hook. We will wrap
   the value into a tuple if a single value is returned(unless that
   value is already a tuple).
   This hook has precedence over the specific module hooks registered
   with "register_forward_pre_hook".
   Returns:


with "register_forward_pre_hook".
   Returns:
      a handle that can be used to remove the added hook by calling
      "handle.remove()"
   Return type:
      "torch.utils.hooks.RemovableHandle"


torch.nn.functional.prelutorch.nn.functional.prelu(input, weight) -> Tensor
   Applies element-wise the function \text{PReLU}(x) = \max(0,x) +
   \text{weight} * \min(0,x) where weight is a learnable parameter.
   Note:
     weight is expected to be a scalar or 1-D tensor. If weight is
     1-D, its size must match the number of input channels, determined
     by input.size(1) when input.dim() >= 2, otherwise 1. In the
     1-D case, note that when input has dim > 2, weight can be
     expanded to the shape of input in a way that is not possible
     using normal broadcasting semantics.
   See "PReLU" for more details.


default_per_channel_weight_fake_quanttorch.quantization.fake_quantize.default_per_channel_weight_fake_quant
   alias of functools.partial(,
   observer=, quant_min=-128, quant_max=127,
   dtype=torch.qint8, qscheme=torch.per_channel_symmetric,
   reduce_range=False, ch_axis=0){}


torch.fft.irffttorch.fft.irfft(input, n=None, dim=- 1, norm=None, *, out=None) -> Tensor
   Computes the inverse of "rfft()".
   "input" is interpreted as a one-sided Hermitian signal in the
   Fourier domain, as produced by "rfft()". By the Hermitian property,
   the output will be real-valued.
   Note:
     Some input frequencies must be real-valued to satisfy the
     Hermitian property. In these cases the imaginary component will
     be ignored. For example, any imaginary component in the zero-
     frequency term cannot be represented in a real output and so will
     always be ignored.
   Note:
     The correct interpretation of the Hermitian input depends on the
     length of the original data, as given by "n". This is because
     each input shape could correspond to either an odd or even length
     signal. By default, the signal is assumed to be even length and
     odd signals will not round-trip properly. So, it is recommended
     to always pass the signal length "n".
   Note:


Note:
     Supports torch.half and torch.chalf on CUDA with GPU Architecture
     SM53 or greater. However it only supports powers of 2 signal
     length in every transformed dimension. With default arguments,
     size of the transformed dimension should be (2^n + 1) as argument
     n defaults to even output size = 2 * (transformed_dim_size - 1)
   Parameters:
      * input (Tensor) -- the input tensor representing a half-
        Hermitian signal
      * n (int, optional) -- Output signal length. This
        determines the length of the output signal. If given, the
        input will either be zero-padded or trimmed to this length
        before computing the real IFFT. Defaults to even output:
        "n=2(input.size(dim) - 1)".
      * dim (int, optional) -- The dimension along which to
        take the one dimensional real IFFT.
      * norm (str, optional*) --
        Normalization mode. For the backward transform ("irfft()"),
        these correspond to:


these correspond to:
        * ""forward"" - no normalization
        * ""backward"" - normalize by "1/n"
        * ""ortho"" - normalize by "1/sqrt(n)" (making the real IFFT
          orthonormal)
        Calling the forward transform ("rfft()") with the same
        normalization mode will apply an overall normalization of
        "1/n" between the two transforms. This is required to make
        "irfft()" the exact inverse.
        Default is ""backward"" (normalize by "1/n").
   Keyword Arguments:
      out (Tensor, optional) -- the output tensor.
   -[ Example ]-

t = torch.linspace(0, 1, 5)
t
   tensor([0.0000, 0.2500, 0.5000, 0.7500, 1.0000])
T = torch.fft.rfft(t)
T
   tensor([ 2.5000+0.0000j, -0.6250+0.8602j, -0.6250+0.2031j])
   Without specifying the output length to "irfft()", the output will
   not round-trip properly because the input is odd-length:
torch.fft.irfft(T)
   tensor([0.1562, 0.3511, 0.7812, 1.2114])


tensor([0.1562, 0.3511, 0.7812, 1.2114])
   So, it is recommended to always pass the signal length "n":

roundtrip = torch.fft.irfft(T, t.numel())
torch.testing.assert_close(roundtrip, t, check_stride=False)


torch.hamming_windowtorch.hamming_window(window_length, periodic=True, alpha=0.54, beta=0.46, *, dtype=None, layout=torch.strided, device=None, requires_grad=False) -> Tensor
   Hamming window function.
      w[n] = \alpha - \beta\ \cos \left( \frac{2 \pi n}{N - 1}
      \right),
   where N is the full window size.
   The input "window_length" is a positive integer controlling the
   returned window size. "periodic" flag determines whether the
   returned window trims off the last duplicate value from the
   symmetric window and is ready to be used as a periodic window with
   functions like "torch.stft()". Therefore, if "periodic" is true,
   the N in above formula is in fact \text{window_length} + 1. Also,
   we always have "torch.hamming_window(L, periodic=True)" equal to
   "torch.hamming_window(L + 1, periodic=False)[:-1])".
   Note:
     If "window_length" =1, the returned window contains a single
     value 1.
   Note:
     This is a generalized version of "torch.hann_window()".


Parameters:
      * window_length (int) -- the size of returned window
      * periodic (bool, optional) -- If True, returns a
        window to be used as periodic function. If False, return a
        symmetric window.
      * alpha (float, optional) -- The coefficient \alpha in
        the equation above
      * beta (float, optional) -- The coefficient \beta in
        the equation above
   Keyword Arguments:
      * dtype ("torch.dtype", optional) -- the desired data type
        of returned tensor. Default: if "None", uses a global default
        (see "torch.set_default_tensor_type()"). Only floating point
        types are supported.
      * layout ("torch.layout", optional) -- the desired layout of
        returned window tensor. Only "torch.strided" (dense layout) is
        supported.
      * device ("torch.device", optional) -- the desired device of
        returned tensor. Default: if "None", uses the current device


for the default tensor type (see
        "torch.set_default_tensor_type()"). "device" will be the CPU
        for CPU tensor types and the current CUDA device for CUDA
        tensor types.
      * requires_grad (bool, optional) -- If autograd should
        record operations on the returned tensor. Default: "False".
   Returns:
      A 1-D tensor of size (\text{window_length},) containing the
      window.
   Return type:
      Tensor


torch.Tensor.matrix_powerTensor.matrix_power(n) -> Tensor
   Note:
     "matrix_power()" is deprecated, use "torch.linalg.matrix_power()"
     instead.
   Alias for "torch.linalg.matrix_power()"


BackendConfigclass torch.ao.quantization.backend_config.BackendConfig(name='')
   Config that defines the set of patterns that can be quantized on a
   given backend, and how reference quantized models can be produced
   from these patterns.
   A pattern in this context refers to a module, a functional, an
   operator, or a directed acyclic graph of the above. Each pattern
   supported on the target backend can be individually configured
   through "BackendPatternConfig" in terms of:
   1. The supported input/output activation, weight, and bias data
      types
   2. How observers and quant/dequant ops are inserted in order to
      construct the reference pattern, and
   3. (Optionally) Fusion, QAT, and reference module mappings.
   The format of the patterns is described in: https://github.com/pyt
   orch/pytorch/blob/master/torch/ao/quantization/backend_config/READ
   ME.md
   Example usage:
      import torch
      from torch.ao.quantization.backend_config import (
          BackendConfig,


BackendConfig,
          BackendPatternConfig,
          DTypeConfig,
          ObservationType,
      )
      weighted_int8_dtype_config = DTypeConfig(
          input_dtype=torch.quint8,
          output_dtype=torch.quint8,
          weight_dtype=torch.qint8,
          bias_dtype=torch.float)
      def fuse_conv2d_relu(is_qat, conv, relu):
          return torch.ao.nn.intrinsic.ConvReLU2d(conv, relu)
      # For quantizing Linear
      linear_config = BackendPatternConfig(torch.nn.Linear)             .set_observation_type(ObservationType.OUTPUT_USE_DIFFERENT_OBSERVER_AS_INPUT)             .add_dtype_config(weighted_int8_dtype_config)             .set_root_module(torch.nn.Linear)             .set_qat_module(torch.ao.nn.qat.Linear)             .set_reference_quantized_module(torch.ao.nn.quantized.reference.Linear)
      # For fusing Conv2d + ReLU into ConvReLU2d


For fusing Conv2d + ReLU into ConvReLU2d
  conv_relu_config = BackendPatternConfig((torch.nn.Conv2d, torch.nn.ReLU))             .set_observation_type(ObservationType.OUTPUT_USE_DIFFERENT_OBSERVER_AS_INPUT)             .add_dtype_config(weighted_int8_dtype_config)             .set_fused_module(torch.ao.nn.intrinsic.ConvReLU2d)             .set_fuser_method(fuse_conv2d_relu)
  # For quantizing ConvReLU2d
  fused_conv_relu_config = BackendPatternConfig(torch.ao.nn.intrinsic.ConvReLU2d)             .set_observation_type(ObservationType.OUTPUT_USE_DIFFERENT_OBSERVER_AS_INPUT)             .add_dtype_config(weighted_int8_dtype_config)             .set_root_module(torch.nn.Conv2d)             .set_qat_module(torch.ao.nn.intrinsic.qat.ConvReLU2d)             .set_reference_quantized_module(torch.ao.nn.quantized.reference.Conv2d)


backend_config = BackendConfig("my_backend")             .set_backend_pattern_config(linear_config)             .set_backend_pattern_config(conv_relu_config)             .set_backend_pattern_config(fused_conv_relu_config)
   property configs: List[BackendPatternConfig]
      Return a copy of the list of configs set in this
      BackendConfig.
   classmethod from_dict(backend_config_dict)
      Create a "BackendConfig" from a dictionary with the following
      items:
         "name": the name of the target backend
         "configs": a list of dictionaries that each represents a
         BackendPatternConfig
      Return type:
         BackendConfig
   set_backend_pattern_config(config)
      Set the config for an pattern that can be run on the target
      backend. This overrides any existing config for the given
      pattern.
      Return type:
         BackendConfig
   set_backend_pattern_configs(configs)
      Set the configs for patterns that can be run on the target


backend. This overrides any existing config for a given pattern
      if it was previously registered already.
      Return type:
         BackendConfig
   set_name(name)
      Set the name of the target backend.
      Return type:
         BackendConfig
   to_dict()
      Convert this "BackendConfig" to a dictionary with the items
      described in "from_dict()".
      Return type:
         Dict[str, Any]


torch.Tensor.as_subclassTensor.as_subclass(cls) -> Tensor
   Makes a "cls" instance with the same data pointer as "self".
   Changes in the output mirror changes in "self", and the output
   stays attached to the autograd graph. "cls" must be a subclass of
   "Tensor".


torch.Tensor.cumprod_Tensor.cumprod_(dim, dtype=None) -> Tensor
   In-place version of "cumprod()"


torch.Tensor.flipudTensor.flipud() -> Tensor
   See "torch.flipud()"


torch.zerostorch.zeros(size, , out=None, dtype=None, layout=torch.strided, device=None, requires_grad=False) -> Tensor
   Returns a tensor filled with the scalar value 0, with the shape
   defined by the variable argument "size".
   Parameters:
      size (int...) -- a sequence of integers defining the
      shape of the output tensor. Can be a variable number of
      arguments or a collection like a list or tuple.
   Keyword Arguments:
      * out (Tensor, optional) -- the output tensor.
      * dtype ("torch.dtype", optional) -- the desired data type
        of returned tensor. Default: if "None", uses a global default
        (see "torch.set_default_tensor_type()").
      * layout ("torch.layout", optional) -- the desired layout of
        returned Tensor. Default: "torch.strided".
      * device ("torch.device", optional) -- the desired device of
        returned tensor. Default: if "None", uses the current device
        for the default tensor type (see


for the default tensor type (see
        "torch.set_default_tensor_type()"). "device" will be the CPU
        for CPU tensor types and the current CUDA device for CUDA
        tensor types.
      * requires_grad (bool, optional) -- If autograd should
        record operations on the returned tensor. Default: "False".
   Example:
      >>> torch.zeros(2, 3)
      tensor([[ 0.,  0.,  0.],
              [ 0.,  0.,  0.]])
      >>> torch.zeros(5)
      tensor([ 0.,  0.,  0.,  0.,  0.])


torch.Tensor.swapaxesTensor.swapaxes(axis0, axis1) -> Tensor
   See "torch.swapaxes()"


torch.jit.savetorch.jit.save(m, f, _extra_files=None)
   Save an offline version of this module for use in a separate
   process. The saved module serializes all of the methods,
   submodules, parameters, and attributes of this module. It can be
   loaded into the C++ API using "torch::jit::load(filename)" or into
   the Python API with "torch.jit.load".
   To be able to save a module, it must not make any calls to native
   Python functions.  This means that all submodules must be
   subclasses of "ScriptModule" as well.
   Danger:
     All modules, no matter their device, are always loaded onto the
     CPU during loading.  This is different from "torch.load()"'s
     semantics and may change in the future.
   Parameters:
      * m -- A "ScriptModule" to save.
      * f -- A file-like object (has to implement write and flush)
        or a string containing a file name.
      * _extra_files -- Map from filename to contents which will
        be stored as part of f.
   Note:


be stored as part of f.
   Note:
     torch.jit.save attempts to preserve the behavior of some
     operators across versions. For example, dividing two integer
     tensors in PyTorch 1.5 performed floor division, and if the
     module containing that code is saved in PyTorch 1.5 and loaded in
     PyTorch 1.6 its division behavior will be preserved. The same
     module saved in PyTorch 1.6 will fail to load in PyTorch 1.5,
     however, since the behavior of division changed in 1.6, and 1.5
     does not know how to replicate the 1.6 behavior.
   Example:
      import torch
      import io
      class MyModule(torch.nn.Module):
          def forward(self, x):
              return x + 10
      m = torch.jit.script(MyModule())
      # Save to file
      torch.jit.save(m, 'scriptmodule.pt')
      # This line is equivalent to the previous
      m.save("scriptmodule.pt")
      # Save to io.BytesIO buffer
      buffer = io.BytesIO()
      torch.jit.save(m, buffer)
      # Save with extra files


Save with extra files
  extra_files = {'foo.txt': b'bar'}
  torch.jit.save(m, 'scriptmodule.pt', _extra_files=extra_files)


Sequentialclass torch.nn.Sequential(*args: Module)
class torch.nn.Sequential(arg: OrderedDict[str, Module])
   A sequential container. Modules will be added to it in the order
   they are passed in the constructor. Alternatively, an "OrderedDict"
   of modules can be passed in. The "forward()" method of "Sequential"
   accepts any input and forwards it to the first module it contains.
   It then "chains" outputs to inputs sequentially for each subsequent
   module, finally returning the output of the last module.
   The value a "Sequential" provides over manually calling a sequence
   of modules is that it allows treating the whole container as a
   single module, such that performing a transformation on the
   "Sequential" applies to each of the modules it stores (which are
   each a registered submodule of the "Sequential").
   What's the difference between a "Sequential" and a
   "torch.nn.ModuleList"? A "ModuleList" is exactly what it sounds


like--a list for storing "Module" s! On the other hand, the layers
   in a "Sequential" are connected in a cascading way.
   Example:
      # Using Sequential to create a small model. When model is run,
      # input will first be passed to Conv2d(1,20,5). The output of
      # Conv2d(1,20,5) will be used as the input to the first
      # ReLU; the output of the first ReLU will become the input
      # for Conv2d(20,64,5). Finally, the output of
      # Conv2d(20,64,5) will be used as input to the second ReLU
      model = nn.Sequential(
                nn.Conv2d(1,20,5),
                nn.ReLU(),
                nn.Conv2d(20,64,5),
                nn.ReLU()
              )
      # Using Sequential with OrderedDict. This is functionally the
      # same as the above code
      model = nn.Sequential(OrderedDict([
                ('conv1', nn.Conv2d(1,20,5)),
                ('relu1', nn.ReLU()),
                ('conv2', nn.Conv2d(20,64,5)),
                ('relu2', nn.ReLU())


('relu2', nn.ReLU())
              ]))
   append(module)
      Appends a given module to the end.
      Parameters:
         module (nn.Module) -- module to append
      Return type:
         Sequential


torch.onestorch.ones(size, , out=None, dtype=None, layout=torch.strided, device=None, requires_grad=False) -> Tensor
   Returns a tensor filled with the scalar value 1, with the shape
   defined by the variable argument "size".
   Parameters:
      size (int...) -- a sequence of integers defining the
      shape of the output tensor. Can be a variable number of
      arguments or a collection like a list or tuple.
   Keyword Arguments:
      * out (Tensor, optional) -- the output tensor.
      * dtype ("torch.dtype", optional) -- the desired data type
        of returned tensor. Default: if "None", uses a global default
        (see "torch.set_default_tensor_type()").
      * layout ("torch.layout", optional) -- the desired layout of
        returned Tensor. Default: "torch.strided".
      * device ("torch.device", optional) -- the desired device of
        returned tensor. Default: if "None", uses the current device
        for the default tensor type (see


for the default tensor type (see
        "torch.set_default_tensor_type()"). "device" will be the CPU
        for CPU tensor types and the current CUDA device for CUDA
        tensor types.
      * requires_grad (bool, optional) -- If autograd should
        record operations on the returned tensor. Default: "False".
   Example:
      >>> torch.ones(2, 3)
      tensor([[ 1.,  1.,  1.],
              [ 1.,  1.,  1.]])
      >>> torch.ones(5)
      tensor([ 1.,  1.,  1.,  1.,  1.])


torch.arcsintorch.arcsin(input, *, out=None) -> Tensor
   Alias for "torch.asin()".


torch.meantorch.mean(input, , dtype=None) -> Tensor
   Returns the mean value of all elements in the "input" tensor.
   Parameters:
      input (Tensor) -- the input tensor.
   Keyword Arguments:
      dtype ("torch.dtype", optional) -- the desired data type of
      returned tensor. If specified, the input tensor is casted to
      "dtype" before the operation is performed. This is useful for
      preventing data type overflows. Default: None.
   Example:
      >>> a = torch.randn(1, 3)
      >>> a
      tensor([[ 0.2294, -0.5481,  1.3288]])
      >>> torch.mean(a)
      tensor(0.3367)
   torch.mean(input, dim, keepdim=False, , dtype=None, out=None) -> Tensor
   Returns the mean value of each row of the "input" tensor in the
   given dimension "dim". If "dim" is a list of dimensions, reduce
   over all of them.
   If "keepdim" is "True", the output tensor is of the same size as
   "input" except in the dimension(s) "dim" where it is of size 1.


Otherwise, "dim" is squeezed (see "torch.squeeze()"), resulting in
   the output tensor having 1 (or "len(dim)") fewer dimension(s).
   Parameters:
      * input (Tensor) -- the input tensor.
      * dim (int or tuple of ints) -- the dimension or
        dimensions to reduce.
      * keepdim (bool) -- whether the output tensor has "dim"
        retained or not.
   Keyword Arguments:
      * dtype ("torch.dtype", optional) -- the desired data type
        of returned tensor. If specified, the input tensor is casted
        to "dtype" before the operation is performed. This is useful
        for preventing data type overflows. Default: None.
      * out (Tensor, optional) -- the output tensor.
   See also:
     "torch.nanmean()" computes the mean value of non-NaN elements.
   Example:
      >>> a = torch.randn(4, 4)
      >>> a
      tensor([[-0.3841,  0.6320,  0.4254, -0.7384],
              [-0.9644,  1.0131, -0.6549, -1.4279],


[-0.2951, -1.3350, -0.7694,  0.5600],
              [ 1.0842, -0.9580,  0.3623,  0.2343]])
      >>> torch.mean(a, 1)
      tensor([-0.0163, -0.5085, -0.4599,  0.1807])
      >>> torch.mean(a, 1, True)
      tensor([[-0.0163],
              [-0.5085],
              [-0.4599],
              [ 0.1807]])


torch.fft.ffttorch.fft.fft(input, n=None, dim=- 1, norm=None, , out=None) -> Tensor
   Computes the one dimensional discrete Fourier transform of "input".
   Note:
     The Fourier domain representation of any real signal satisfies
     the Hermitian property: X[i] = conj(X[-i]). This function
     always returns both the positive and negative frequency terms
     even though, for real inputs, the negative frequencies are
     redundant. "rfft()" returns the more compact one-sided
     representation where only the positive frequencies are returned.
   Note:
     Supports torch.half and torch.chalf on CUDA with GPU Architecture
     SM53 or greater. However it only supports powers of 2 signal
     length in every transformed dimension.
   Parameters:
      * input (Tensor) -- the input tensor
      * n (int, optional*) -- Signal length. If given, the
        input will either be zero-padded or trimmed to this length
        before computing the FFT.


before computing the FFT.
      * dim (int, optional) -- The dimension along which to
        take the one dimensional FFT.
      * norm (str, optional) --
        Normalization mode. For the forward transform ("fft()"), these
        correspond to:
        * ""forward"" - normalize by "1/n"
        * ""backward"" - no normalization
        * ""ortho"" - normalize by "1/sqrt(n)" (making the FFT
          orthonormal)
        Calling the backward transform ("ifft()") with the same
        normalization mode will apply an overall normalization of
        "1/n" between the two transforms. This is required to make
        "ifft()" the exact inverse.
        Default is ""backward"" (no normalization).
   Keyword Arguments:
      out (Tensor, optional) -- the output tensor.
   -[ Example ]-

t = torch.arange(4)
t
   tensor([0, 1, 2, 3])
torch.fft.fft(t)
   tensor([ 6.+0.j, -2.+2.j, -2.+0.j, -2.-2.j])
t = torch.tensor([0.+1.j, 2.+3.j, 4.+5.j, 6.+7.j])


torch.fft.fft(t)
   tensor([12.+16.j, -8.+0.j, -4.-4.j,  0.-8.j])


torch.Tensor.varTensor.var(dim=None, *, correction=1, keepdim=False) -> Tensor
   See "torch.var()"


torch.erfctorch.erfc(input, *, out=None) -> Tensor
   Alias for "torch.special.erfc()".


torch.nn.functional.one_hottorch.nn.functional.one_hot(tensor, num_classes=- 1) -> LongTensor
   Takes LongTensor with index values of shape "()" and returns a
   tensor of shape "(, num_classes)" that have zeros everywhere
   except where the index of last dimension matches the corresponding
   value of the input tensor, in which case it will be 1.
   See also One-hot on Wikipedia .
   Parameters:
      * tensor (LongTensor) -- class values of any shape.
      * num_classes (int) -- Total number of classes. If set to
        -1, the number of classes will be inferred as one greater than
        the largest class value in the input tensor.
   Returns:
      LongTensor that has one more dimension with 1 values at the
      index of last dimension indicated by the input, and 0 everywhere
      else.
   -[ Examples ]-

F.one_hot(torch.arange(0, 5) % 3)
   tensor([[1, 0, 0],
           [0, 1, 0],
           [0, 0, 1],
           [1, 0, 0],
           [0, 1, 0]])


[1, 0, 0],
           [0, 1, 0]])

F.one_hot(torch.arange(0, 5) % 3, num_classes=5)
   tensor([[1, 0, 0, 0, 0],
           [0, 1, 0, 0, 0],
           [0, 0, 1, 0, 0],
           [1, 0, 0, 0, 0],
           [0, 1, 0, 0, 0]])
F.one_hot(torch.arange(0, 6).view(3,2) % 3)
   tensor([[[1, 0, 0],
            [0, 1, 0]],
           [[0, 0, 1],
            [1, 0, 0]],
           [[0, 1, 0],
            [0, 0, 1]]])


torch.Tensor.tileTensor.tile(*reps) -> Tensor
   See "torch.tile()"


torch.Tensor.log2Tensor.log2() -> Tensor
   See "torch.log2()"


torch.Tensor.lcm_Tensor.lcm_(other) -> Tensor
   In-place version of "lcm()"


torch.cholesky_solvetorch.cholesky_solve(input, input2, upper=False, , out=None) -> Tensor
   Solves a linear system of equations with a positive semidefinite
   matrix to be inverted given its Cholesky factor matrix u.
   If "upper" is "False", u is and lower triangular and c is
   returned such that:
      c = (u u^T)^{{-1}} b
   If "upper" is "True" or not provided, u is upper triangular and c
   is returned such that:
      c = (u^T u)^{{-1}} b
   torch.cholesky_solve(b, u) can take in 2D inputs b, u or inputs
   that are batches of 2D matrices. If the inputs are batches, then
   returns batched outputs c
   Supports real-valued and complex-valued inputs. For the complex-
   valued inputs the transpose operator above is the conjugate
   transpose.
   Parameters:
      * input (Tensor) -- input matrix b of size (, m, k),
        where * is zero or more batch dimensions
      * input2 (Tensor) -- input matrix u of size (*, m, m),


where * is zero of more batch dimensions composed of upper or
        lower triangular Cholesky factor
      * upper (bool, optional) -- whether to consider the
        Cholesky factor as a lower or upper triangular matrix.
        Default: "False".
   Keyword Arguments:
      out (Tensor, optional) -- the output tensor for c
   Example:
      >>> a = torch.randn(3, 3)
      >>> a = torch.mm(a, a.t()) # make symmetric positive definite
      >>> u = torch.linalg.cholesky(a)
      >>> a
      tensor([[ 0.7747, -1.9549,  1.3086],
              [-1.9549,  6.7546, -5.4114],
              [ 1.3086, -5.4114,  4.8733]])
      >>> b = torch.randn(3, 2)
      >>> b
      tensor([[-0.6355,  0.9891],
              [ 0.1974,  1.4706],
              [-0.4115, -0.6225]])
      >>> torch.cholesky_solve(b, u)
      tensor([[ -8.1625,  19.6097],
              [ -5.8398,  14.2387],
              [ -4.3771,  10.4173]])
      >>> torch.mm(a.inverse(), b)
      tensor([[ -8.1626,  19.6097],


tensor([[ -8.1626,  19.6097],
              [ -5.8398,  14.2387],
              [ -4.3771,  10.4173]])


torch.tensor_splittorch.tensor_split(input, indices_or_sections, dim=0) -> List of Tensors
   Splits a tensor into multiple sub-tensors, all of which are views
   of "input", along dimension "dim" according to the indices or
   number of sections specified by "indices_or_sections". This
   function is based on NumPy's "numpy.array_split()".
   Parameters:
      * input (Tensor) -- the tensor to split
      * indices_or_sections (Tensor, int or list or
        tuple of ints) --
        If "indices_or_sections" is an integer "n" or a zero
        dimensional long tensor with value "n", "input" is split into
        "n" sections along dimension "dim". If "input" is divisible by
        "n" along dimension "dim", each section will be of equal size,
        "input.size(dim) / n". If "input" is not divisible by "n", the
        sizes of the first "int(input.size(dim) % n)" sections will
        have size "int(input.size(dim) / n) + 1", and the rest will


have size "int(input.size(dim) / n)".
        If "indices_or_sections" is a list or tuple of ints, or a one-
        dimensional long tensor, then "input" is split along dimension
        "dim" at each of the indices in the list, tuple or tensor. For
        instance, "indices_or_sections=[2, 3]" and "dim=0" would
        result in the tensors "input[:2]", "input[2:3]", and
        "input[3:]".
        If "indices_or_sections" is a tensor, it must be a zero-
        dimensional or one-dimensional long tensor on the CPU.
      * dim (int, optional) -- dimension along which to
        split the tensor. Default: "0"
   Example:
      >>> x = torch.arange(8)
      >>> torch.tensor_split(x, 3)
      (tensor([0, 1, 2]), tensor([3, 4, 5]), tensor([6, 7]))
      >>> x = torch.arange(7)
      >>> torch.tensor_split(x, 3)
      (tensor([0, 1, 2]), tensor([3, 4]), tensor([5, 6]))
      >>> torch.tensor_split(x, (1, 6))
      (tensor([0]), tensor([1, 2, 3, 4, 5]), tensor([6]))


x = torch.arange(14).reshape(2, 7)
      >>> x
      tensor([[ 0,  1,  2,  3,  4,  5,  6],
              [ 7,  8,  9, 10, 11, 12, 13]])
      >>> torch.tensor_split(x, 3, dim=1)
      (tensor([[0, 1, 2],
              [7, 8, 9]]),
       tensor([[ 3,  4],
              [10, 11]]),
       tensor([[ 5,  6],
              [12, 13]]))
      >>> torch.tensor_split(x, (1, 6), dim=1)
      (tensor([[0],
              [7]]),
       tensor([[ 1,  2,  3,  4,  5],
              [ 8,  9, 10, 11, 12]]),
       tensor([[ 6],
              [13]]))


torch.fft.rfftntorch.fft.rfftn(input, s=None, dim=None, norm=None, , out=None) -> Tensor
   Computes the N-dimensional discrete Fourier transform of real
   "input".
   The FFT of a real signal is Hermitian-symmetric, "X[i_1, ..., i_n]
   = conj(X[-i_1, ..., -i_n])" so the full "fftn()" output contains
   redundant information. "rfftn()" instead omits the negative
   frequencies in the last dimension.
   Note:
     Supports torch.half on CUDA with GPU Architecture SM53 or
     greater. However it only supports powers of 2 signal length in
     every transformed dimensions.
   Parameters:
      * input (Tensor) -- the input tensor
      * s (Tuple[int], *optional) -- Signal size in the
        transformed dimensions. If given, each dimension "dim[i]" will
        either be zero-padded or trimmed to the length "s[i]" before
        computing the real FFT. If a length "-1" is specified, no
        padding is done in that dimension. Default: "s =


[input.size(d) for d in dim]"
      * dim (Tuple[int], optional) -- Dimensions to be
        transformed. Default: all dimensions, or the last "len(s)"
        dimensions if "s" is given.
      * norm (str, optional) --
        Normalization mode. For the forward transform ("rfftn()"),
        these correspond to:
        * ""forward"" - normalize by "1/n"
        * ""backward"" - no normalization
        * ""ortho"" - normalize by "1/sqrt(n)" (making the real FFT
          orthonormal)
        Where "n = prod(s)" is the logical FFT size. Calling the
        backward transform ("irfftn()") with the same normalization
        mode will apply an overall normalization of "1/n" between the
        two transforms. This is required to make "irfftn()" the exact
        inverse.
        Default is ""backward"" (no normalization).
   Keyword Arguments:
      out (Tensor, optional) -- the output tensor.
   -[ Example ]-

t = torch.rand(10, 10)


-[ Example ]-

t = torch.rand(10, 10)
rfftn = torch.fft.rfftn(t)
rfftn.size()
   torch.Size([10, 6])
   Compared against the full output from "fftn()", we have all
   elements up to the Nyquist frequency.
fftn = torch.fft.fftn(t)
torch.testing.assert_close(fftn[..., :6], rfftn, check_stride=False)
   The discrete Fourier transform is separable, so "rfftn()" here is
   equivalent to a combination of "fft()" and "rfft()":
two_ffts = torch.fft.fft(torch.fft.rfft(t, dim=1), dim=0)
torch.testing.assert_close(rfftn, two_ffts, check_stride=False)


torch.randpermtorch.randperm(n, , generator=None, out=None, dtype=torch.int64, layout=torch.strided, device=None, requires_grad=False, pin_memory=False) -> Tensor
   Returns a random permutation of integers from "0" to "n - 1".
   Parameters:
      n (int) -- the upper bound (exclusive)
   Keyword Arguments:
      * generator ("torch.Generator", optional) -- a pseudorandom
        number generator for sampling
      * out (Tensor, optional) -- the output tensor.
      * dtype ("torch.dtype", optional) -- the desired data type
        of returned tensor. Default: "torch.int64".
      * layout ("torch.layout", optional) -- the desired layout of
        returned Tensor. Default: "torch.strided".
      * device* ("torch.device", optional) -- the desired device of
        returned tensor. Default: if "None", uses the current device
        for the default tensor type (see
        "torch.set_default_tensor_type()"). "device" will be the CPU


for CPU tensor types and the current CUDA device for CUDA
        tensor types.
      * requires_grad (bool, optional) -- If autograd should
        record operations on the returned tensor. Default: "False".
      * pin_memory (bool, optional) -- If set, returned
        tensor would be allocated in the pinned memory. Works only for
        CPU tensors. Default: "False".
   Example:
      >>> torch.randperm(4)
      tensor([2, 1, 0, 3])


torch.nn.functional.tanhshrinktorch.nn.functional.tanhshrink(input) -> Tensor
   Applies element-wise, \text{Tanhshrink}(x) = x - \text{Tanh}(x)
   See "Tanhshrink" for more details.


torch.func.replace_all_batch_norm_modules_torch.func.replace_all_batch_norm_modules_(root)
   In place updates "root" by setting the "running_mean" and
   "running_var" to be None and setting track_running_stats to be
   False for any nn.BatchNorm module in "root"
   Return type:
      Module


hardswishclass torch.ao.nn.quantized.functional.hardswish(input, scale, zero_point)
   This is the quantized version of "hardswish()".
   Parameters:
      * input (Tensor) -- quantized input
      * scale (float) -- quantization scale of the output tensor
      * zero_point (int) -- quantization zero point of the
        output tensor
   Return type:
      Tensor


Transformerclass torch.nn.Transformer(d_model=512, nhead=8, num_encoder_layers=6, num_decoder_layers=6, dim_feedforward=2048, dropout=0.1, activation=, custom_encoder=None, custom_decoder=None, layer_norm_eps=1e-05, batch_first=False, norm_first=False, device=None, dtype=None)
   A transformer model. User is able to modify the attributes as
   needed. The architecture is based on the paper "Attention Is All
   You Need". Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob
   Uszkoreit, Llion Jones, Aidan N Gomez, Lukasz Kaiser, and Illia
   Polosukhin. 2017. Attention is all you need. In Advances in Neural
   Information Processing Systems, pages 6000-6010.
   Parameters:
      * d_model (int) -- the number of expected features in the
        encoder/decoder inputs (default=512).
      * nhead (int) -- the number of heads in the
        multiheadattention models (default=8).
      * num_encoder_layers (int) -- the number of sub-encoder-


layers in the encoder (default=6).
      * num_decoder_layers (int) -- the number of sub-decoder-
        layers in the decoder (default=6).
      * dim_feedforward (int) -- the dimension of the
        feedforward network model (default=2048).
      * dropout (float) -- the dropout value (default=0.1).
      * activation (Union[str,
        Callable[[Tensor], Tensor]]) -- the
        activation function of encoder/decoder intermediate layer, can
        be a string ("relu" or "gelu") or a unary callable. Default:
        relu
      * custom_encoder (Optional[Any]) -- custom encoder
        (default=None).
      * custom_decoder (Optional[Any]) -- custom decoder
        (default=None).
      * layer_norm_eps (float) -- the eps value in layer
        normalization components (default=1e-5).
      * batch_first (bool) -- If "True", then the input and
        output tensors are provided as (batch, seq, feature). Default:


"False" (seq, batch, feature).
      * norm_first (bool) -- if "True", encoder and decoder
        layers will perform LayerNorms before other attention and
        feedforward operations, otherwise after. Default: "False"
        (after).
   Examples::
      >>> transformer_model = nn.Transformer(nhead=16, num_encoder_layers=12)
      >>> src = torch.rand((10, 32, 512))
      >>> tgt = torch.rand((20, 32, 512))
      >>> out = transformer_model(src, tgt)
   Note: A full example to apply nn.Transformer module for the word
   language model is available in
   https://github.com/pytorch/examples/tree/master/word_language_model
   forward(src, tgt, src_mask=None, tgt_mask=None, memory_mask=None, src_key_padding_mask=None, tgt_key_padding_mask=None, memory_key_padding_mask=None)
      Take in and process masked source/target sequences.
      Parameters:
         * src (Tensor) -- the sequence to the encoder
           (required).
         * tgt (Tensor) -- the sequence to the decoder


(required).
         * src_mask (Optional[Tensor]) -- the additive
           mask for the src sequence (optional).
         * tgt_mask (Optional[Tensor]) -- the additive
           mask for the tgt sequence (optional).
         * memory_mask (Optional[Tensor]) -- the additive
           mask for the encoder output (optional).
         * src_key_padding_mask (Optional[Tensor]) -- the
           ByteTensor mask for src keys per batch (optional).
         * tgt_key_padding_mask (Optional[Tensor]) -- the
           ByteTensor mask for tgt keys per batch (optional).
         * memory_key_padding_mask (Optional[Tensor]) --
           the ByteTensor mask for memory keys per batch (optional).
      Return type:
         Tensor
      Shape:
         * src: (S, E) for unbatched input, (S, N, E) if
           batch_first=False or (N, S, E) if batch_first=True.
         * tgt: (T, E) for unbatched input, (T, N, E) if


batch_first=False or (N, T, E) if batch_first=True.
         * src_mask: (S, S) or (N\cdot\text{num_heads}, S, S).
         * tgt_mask: (T, T) or (N\cdot\text{num_heads}, T, T).
         * memory_mask: (T, S).
         * src_key_padding_mask: (S) for unbatched input otherwise (N,
           S).
         * tgt_key_padding_mask: (T) for unbatched input otherwise (N,
           T).
         * memory_key_padding_mask: (S) for unbatched input otherwise
           (N, S).
         Note: [src/tgt/memory]_mask ensures that position i is
         allowed to attend the unmasked positions. If a ByteTensor is
         provided, the non-zero positions are not allowed to attend
         while the zero positions will be unchanged. If a BoolTensor
         is provided, positions with "True" are not allowed to attend
         while "False" values will be unchanged. If a FloatTensor is
         provided, it will be added to the attention weight.


[src/tgt/memory]_key_padding_mask provides specified elements
         in the key to be ignored by the attention. If a ByteTensor is
         provided, the non-zero positions will be ignored while the
         zero positions will be unchanged. If a BoolTensor is
         provided, the positions with the value of "True" will be
         ignored while the position with the value of "False" will be
         unchanged.
         * output: (T, E) for unbatched input, (T, N, E) if
           batch_first=False or (N, T, E) if batch_first=True.
         Note: Due to the multi-head attention architecture in the
         transformer model, the output sequence length of a
         transformer is same as the input sequence (i.e. target)
         length of the decoder.
         where S is the source sequence length, T is the target
         sequence length, N is the batch size, E is the feature number
      -[ Examples ]-
      >>> output = transformer_model(src, tgt, src_mask=src_mask, tgt_mask=tgt_mask)


static generate_square_subsequent_mask(sz, device='cpu')
      Generate a square mask for the sequence. The masked positions
      are filled with float('-inf'). Unmasked positions are filled
      with float(0.0).
      Return type:
         Tensor


default_placeholder_observertorch.quantization.observer.default_placeholder_observer
   alias of "PlaceholderObserver"


torch.sparse_csr_tensortorch.sparse_csr_tensor(crow_indices, col_indices, values, size=None, , dtype=None, device=None, requires_grad=False, check_invariants=None) -> Tensor
   Constructs a sparse tensor in CSR (Compressed Sparse Row) with
   specified values at the given "crow_indices" and "col_indices".
   Sparse matrix multiplication operations in CSR format are typically
   faster than that for sparse tensors in COO format. Make you have a
   look at the note on the data type of the indices.
   Note:
     If the "device" argument is not specified the device of the given
     "values" and indices tensor(s) must match. If, however, the
     argument is specified the input Tensors will be converted to the
     given device and in turn determine the device of the constructed
     sparse tensor.
   Parameters:
      * crow_indices (array_like) -- (B+1)-dimensional array of
        size "(batchsize, nrows + 1)".  The last element of each


batch is the number of non-zeros. This tensor encodes the
        index in values and col_indices depending on where the given
        row starts. Each successive number in the tensor subtracted by
        the number before it denotes the number of elements in a given
        row.
      * col_indices (array_like) -- Column co-ordinates of each
        element in values. (B+1)-dimensional tensor with the same
        length as values.
      * values (array_list) -- Initial values for the tensor.
        Can be a list, tuple, NumPy "ndarray", scalar, and other types
        that represents a (1+K)-dimensional tensor where "K" is the
        number of dense dimensions.
      * size (list, tuple, "torch.Size", optional) -- Size of the
        sparse tensor: "(batchsize, nrows, ncols, densesize)". If
        not provided, the size will be inferred as the minimum size
        big enough to hold all non-zero elements.
   Keyword Arguments:


Keyword Arguments:
      * dtype ("torch.dtype", optional) -- the desired data type
        of returned tensor.  Default: if None, infers data type from
        "values".
      * device ("torch.device", optional) -- the desired device of
        returned tensor.  Default: if None, uses the current device
        for the default tensor type (see
        "torch.set_default_tensor_type()"). "device" will be the CPU
        for CPU tensor types and the current CUDA device for CUDA
        tensor types.
      * requires_grad (bool, optional) -- If autograd should
        record operations on the returned tensor. Default: "False".
      * check_invariants (bool, optional) -- If sparse
        tensor invariants are checked. Default: as returned by
        "torch.sparse.check_sparse_tensor_invariants.is_enabled()",
        initially False.
   Example::
      >>> crow_indices = [0, 2, 4]
      >>> col_indices = [0, 1, 0, 1]
      >>> values = [1, 2, 3, 4]


values = [1, 2, 3, 4]
      >>> torch.sparse_csr_tensor(torch.tensor(crow_indices, dtype=torch.int64),
      ...                         torch.tensor(col_indices, dtype=torch.int64),
      ...                         torch.tensor(values), dtype=torch.double)
      tensor(crow_indices=tensor([0, 2, 4]),
             col_indices=tensor([0, 1, 0, 1]),
             values=tensor([1., 2., 3., 4.]), size=(2, 2), nnz=4,
             dtype=torch.float64, layout=torch.sparse_csr)


torch.column_stacktorch.column_stack(tensors, , out=None) -> Tensor
   Creates a new tensor by horizontally stacking the tensors in
   "tensors".
   Equivalent to "torch.hstack(tensors)", except each zero or one
   dimensional tensor "t" in "tensors" is first reshaped into a
   "(t.numel(), 1)" column before being stacked horizontally.
   Parameters:
      tensors (sequence of Tensors) -- sequence of tensors to
      concatenate
   Keyword Arguments:
      out (Tensor, optional*) -- the output tensor.
   Example:
      >>> a = torch.tensor([1, 2, 3])
      >>> b = torch.tensor([4, 5, 6])
      >>> torch.column_stack((a, b))
      tensor([[1, 4],
          [2, 5],
          [3, 6]])
      >>> a = torch.arange(5)
      >>> b = torch.arange(10).reshape(5, 2)
      >>> torch.column_stack((a, b, b))
      tensor([[0, 0, 1, 0, 1],
              [1, 2, 3, 2, 3],
              [2, 4, 5, 4, 5],
              [3, 6, 7, 6, 7],
              [4, 8, 9, 8, 9]])


torch.index_reducetorch.index_reduce(input, dim, index, source, reduce, *, include_self=True, out=None) -> Tensor
   See "index_reduce_()" for function description.


torch.Tensor.negTensor.neg() -> Tensor
   See "torch.neg()"


torch.Tensor.is_complexTensor.is_complex() -> bool
   Returns True if the data type of "self" is a complex data type.


torch.nn.utils.parametrizations.orthogonaltorch.nn.utils.parametrizations.orthogonal(module, name='weight', orthogonal_map=None, , use_trivialization=True)
   Applies an orthogonal or unitary parametrization to a matrix or a
   batch of matrices.
   Letting \mathbb{K} be \mathbb{R} or \mathbb{C}, the parametrized
   matrix Q \in \mathbb{K}^{m \times n} is orthogonal as
      \begin{align}     Q^{\text{H}}Q &= \mathrm{I}_n
      \mathrlap{\qquad \text{if }m \geq n}\     QQ^{\text{H}} &=
      \mathrm{I}_m \mathrlap{\qquad \text{if }m < n} \end{align}
   where Q^{\text{H}} is the conjugate transpose when Q is complex and
   the transpose when Q is real-valued, and \mathrm{I}_n is the
   n-dimensional identity matrix. In plain words, Q will have
   orthonormal columns whenever m \geq n and orthonormal rows
   otherwise.
   If the tensor has more than two dimensions, we consider it as a
   batch of matrices of shape (..., m, n)*.


batch of matrices of shape (..., m, n).
   The matrix Q may be parametrized via three different
   "orthogonal_map" in terms of the original tensor:
   * ""matrix_exp""/""cayley"": the "matrix_exp()" Q = \exp(A) and the
     Cayley map Q = (\mathrm{I}_n + A/2)(\mathrm{I}_n - A/2)^{-1} are
     applied to a skew-symmetric A to give an orthogonal matrix.
   * ""householder"": computes a product of Householder reflectors
     ("householder_product()").
   ""matrix_exp""/""cayley"" often make the parametrized weight
   converge faster than ""householder"", but they are slower to
   compute for very thin or very wide matrices.
   If "use_trivialization=True" (default), the parametrization
   implements the "Dynamic Trivialization Framework", where an extra
   matrix B \in \mathbb{K}^{n \times n} is stored under
   "module.parametrizations.weight[0].base". This helps the
   convergence of the parametrized layer at the expense of some extra
   memory use. See Trivializations for Gradient-Based Optimization on


Manifolds .
   Initial value of Q: If the original tensor is not parametrized and
   "use_trivialization=True" (default), the initial value of Q is that
   of the original tensor if it is orthogonal (or unitary in the
   complex case) and it is orthogonalized via the QR decomposition
   otherwise (see "torch.linalg.qr()"). Same happens when it is not
   parametrized and "orthogonal_map="householder"" even when
   "use_trivialization=False". Otherwise, the initial value is the
   result of the composition of all the registered parametrizations
   applied to the original tensor.
   Note:
     This function is implemented using the parametrization
     functionality in "register_parametrization()".
   Parameters:
      * module (nn.Module) -- module on which to register the
        parametrization.
      * name (str, optional) -- name of the tensor to make
        orthogonal. Default: ""weight"".
      * orthogonal_map (str, optional) -- One of the


following: ""matrix_exp"", ""cayley"", ""householder"".
        Default: ""matrix_exp"" if the matrix is square or complex,
        ""householder"" otherwise.
      * use_trivialization (bool, optional) -- whether to
        use the dynamic trivialization framework. Default: "True".
   Returns:
      The original module with an orthogonal parametrization
      registered to the specified weight
   Return type:
      Module
   Example:
      >>> orth_linear = orthogonal(nn.Linear(20, 40))
      >>> orth_linear
      ParametrizedLinear(
      in_features=20, out_features=40, bias=True
      (parametrizations): ModuleDict(
          (weight): ParametrizationList(
          (0): _Orthogonal()
          )
      )
      )
      >>> Q = orth_linear.weight
      >>> torch.dist(Q.T @ Q, torch.eye(20))
      tensor(4.9332e-07)


torch.Tensor.anyTensor.any(dim=None, keepdim=False) -> Tensor
   See "torch.any()"


torch.Tensor.distTensor.dist(other, p=2) -> Tensor
   See "torch.dist()"


torch.cuda.device_counttorch.cuda.device_count()
   Returns the number of GPUs available.
   Return type:
      int


SyncBatchNormclass torch.nn.SyncBatchNorm(num_features, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True, process_group=None, device=None, dtype=None)
   Applies Batch Normalization over a N-Dimensional input (a mini-
   batch of [N-2]D inputs with additional channel dimension) as
   described in the paper Batch Normalization: Accelerating Deep
   Network Training by Reducing Internal Covariate Shift .
      y = \frac{x - \mathrm{E}[x]}{ \sqrt{\mathrm{Var}[x] + \epsilon}}
      * \gamma + \beta
   The mean and standard-deviation are calculated per-dimension over
   all mini-batches of the same process groups. \gamma and \beta are
   learnable parameter vectors of size C (where C is the input
   size). By default, the elements of \gamma are sampled from
   \mathcal{U}(0, 1) and the elements of \beta are set to 0. The
   standard-deviation is calculated via the biased estimator,
   equivalent to torch.var(input, unbiased=False).


Also by default, during training this layer keeps running estimates
   of its computed mean and variance, which are then used for
   normalization during evaluation. The running estimates are kept
   with a default "momentum" of 0.1.
   If "track_running_stats" is set to "False", this layer then does
   not keep running estimates, and batch statistics are instead used
   during evaluation time as well.
   Note:
     This "momentum" argument is different from one used in optimizer
     classes and the conventional notion of momentum. Mathematically,
     the update rule for running statistics here is \hat{x}_\text{new}
     = (1 - \text{momentum}) \times \hat{x} + \text{momentum} \times
     x_t, where \hat{x} is the estimated statistic and x_t is the new
     observed value.
   Because the Batch Normalization is done for each channel in the "C"
   dimension, computing statistics on "(N, +)" slices, it's common
   terminology to call this Volumetric Batch Normalization or Spatio-


temporal Batch Normalization.
   Currently "SyncBatchNorm" only supports "DistributedDataParallel"
   (DDP) with single GPU per process. Use
   "torch.nn.SyncBatchNorm.convert_sync_batchnorm()" to convert
   "BatchNormD" layer to "SyncBatchNorm" before wrapping Network with
   DDP.
   Parameters:
      * num_features (int) -- C from an expected input of size
        (N, C, +)
      * eps (float) -- a value added to the denominator for
        numerical stability. Default: "1e-5"
      * momentum (float) -- the value used for the running_mean
        and running_var computation. Can be set to "None" for
        cumulative moving average (i.e. simple average). Default: 0.1
      * affine (bool) -- a boolean value that when set to
        "True", this module has learnable affine parameters. Default:
        "True"
      * track_running_stats (bool*) -- a boolean value that when
        set to "True", this module tracks the running mean and


variance, and when set to "False", this module does not track
        such statistics, and initializes statistics buffers
        "running_mean" and "running_var" as "None". When these buffers
        are "None", this module always uses batch statistics. in both
        training and eval modes. Default: "True"
      * process_group (Optional[Any]) -- synchronization
        of stats happen within each process group individually.
        Default behavior is synchronization across the whole world
   Shape:
      * Input: (N, C, +)
      * Output: (N, C, +) (same shape as input)
   Note:
     Synchronization of batchnorm statistics occurs only while
     training, i.e. synchronization is disabled when "model.eval()" is
     set or if "self.training" is otherwise "False".
   Examples:
      >>> # With Learnable Parameters
      >>> m = nn.SyncBatchNorm(100)
      >>> # creating process group (optional)
      >>> # ranks is a list of int identifying rank ids.
      >>> ranks = list(range(8))


ranks = list(range(8))
      >>> r1, r2 = ranks[:4], ranks[4:]
      >>> # Note: every rank calls into new_group for every
      >>> # process group created, even if that rank is not
      >>> # part of the group.
      >>> process_groups = [torch.distributed.new_group(pids) for pids in [r1, r2]]
      >>> process_group = process_groups[0 if dist.get_rank() <= 3 else 1]
      >>> # Without Learnable Parameters
      >>> m = nn.BatchNorm3d(100, affine=False, process_group=process_group)
      >>> input = torch.randn(20, 100, 35, 45, 10)
      >>> output = m(input)
      >>> # network is nn.BatchNorm layer
      >>> sync_bn_network = nn.SyncBatchNorm.convert_sync_batchnorm(network, process_group)
      >>> # only single gpu per process is currently supported
      >>> ddp_sync_bn_network = torch.nn.parallel.DistributedDataParallel(
      >>>                         sync_bn_network,
      >>>                         device_ids=[args.local_rank],


                    output_device=args.local_rank)

classmethod convert_sync_batchnorm(module, process_group=None)
      Helper function to convert all "BatchNormD" layers in the model
      to "torch.nn.SyncBatchNorm" layers.
      Parameters:
         * module (nn.Module) -- module containing one or more
           "BatchNormD" layers
         * process_group (optional) -- process group to scope
           synchronization, default is the whole world
      Returns:
         The original "module" with the converted
         "torch.nn.SyncBatchNorm" layers. If the original "module" is
         a "BatchNorm*D" layer, a new "torch.nn.SyncBatchNorm" layer
         object will be returned instead.
      Example:
         >>> # Network with nn.BatchNorm layer
         >>> module = torch.nn.Sequential(
         >>>            torch.nn.Linear(20, 100),
         >>>            torch.nn.BatchNorm1d(100),
         >>>          ).cuda()
         >>> # creating process group (optional)


creating process group (optional)
     >>> # ranks is a list of int identifying rank ids.
     >>> ranks = list(range(8))
     >>> r1, r2 = ranks[:4], ranks[4:]
     >>> # Note: every rank calls into new_group for every
     >>> # process group created, even if that rank is not
     >>> # part of the group.
     >>> process_groups = [torch.distributed.new_group(pids) for pids in [r1, r2]]
     >>> process_group = process_groups[0 if dist.get_rank() <= 3 else 1]
     >>> sync_bn_module = torch.nn.SyncBatchNorm.convert_sync_batchnorm(module, process_group)


LambdaLRclass torch.optim.lr_scheduler.LambdaLR(optimizer, lr_lambda, last_epoch=- 1, verbose=False)
   Sets the learning rate of each parameter group to the initial lr
   times a given function. When last_epoch=-1, sets initial lr as lr.
   Parameters:
      * optimizer (Optimizer) -- Wrapped optimizer.
      * lr_lambda (function or list) -- A function which
        computes a multiplicative factor given an integer parameter
        epoch, or a list of such functions, one for each group in
        optimizer.param_groups.
      * last_epoch (int) -- The index of last epoch. Default:
        -1.
      * verbose (bool) -- If "True", prints a message to stdout
        for each update. Default: "False".
   -[ Example ]-

Assuming optimizer has two groups.
lambda1 = lambda epoch: epoch // 30
lambda2 = lambda epoch: 0.95 ** epoch
scheduler = LambdaLR(optimizer, lr_lambda=[lambda1, lambda2])
for epoch in range(100):
    train(...)


train(...)
validate(...)
scheduler.step()

get_last_lr()
      Return last computed learning rate by current scheduler.
   load_state_dict(state_dict)
      Loads the schedulers state.
      When saving or loading the scheduler, please make sure to also
      save or load the state of the optimizer.
      Parameters:
         state_dict (dict) -- scheduler state. Should be an
         object returned from a call to "state_dict()".
   print_lr(is_verbose, group, lr, epoch=None)
      Display the current learning rate.
   state_dict()
      Returns the state of the scheduler as a "dict".
      It contains an entry for every variable in self.dict which
      is not the optimizer. The learning rate lambda functions will
      only be saved if they are callable objects and not if they are
      functions or lambdas.
      When saving or loading the scheduler, please make sure to also
      save or load the state of the optimizer.


torch.eyetorch.eye(n, m=None, , out=None, dtype=None, layout=torch.strided, device=None, requires_grad=False) -> Tensor
   Returns a 2-D tensor with ones on the diagonal and zeros elsewhere.
   Parameters:
      * n (int) -- the number of rows
      * m (int, optional) -- the number of columns with
        default being "n"
   Keyword Arguments:
      * out (Tensor, optional) -- the output tensor.
      * dtype ("torch.dtype", optional) -- the desired data type
        of returned tensor. Default: if "None", uses a global default
        (see "torch.set_default_tensor_type()").
      * layout ("torch.layout", optional) -- the desired layout of
        returned Tensor. Default: "torch.strided".
      * device* ("torch.device", optional) -- the desired device of
        returned tensor. Default: if "None", uses the current device
        for the default tensor type (see
        "torch.set_default_tensor_type()"). "device" will be the CPU


for CPU tensor types and the current CUDA device for CUDA
        tensor types.
      * requires_grad (bool, optional) -- If autograd should
        record operations on the returned tensor. Default: "False".
   Returns:
      A 2-D tensor with ones on the diagonal and zeros elsewhere
   Return type:
      Tensor
   Example:
      >>> torch.eye(3)
      tensor([[ 1.,  0.,  0.],
              [ 0.,  1.,  0.],
              [ 0.,  0.,  1.]])