shrinath-suresh/blogs-docs-splitted huggingface dataset:
Dropout2d
class torch.nn.Dropout2d(p=0.5, inplace=False)
Randomly zero out entire channels (a channel is a 2D feature map,
   e.g., the j-th channel of the i-th sample in the batched input is a
   2D tensor \text{input}[i, j]). Each channel will be zeroed out
   independently on every forward call with probability "p" using
   samples from a Bernoulli distribution.
Usually the input comes from "nn.Conv2d" modules.
As described in the paper Efficient Object Localization Using
   Convolutional Networks , if adjacent pixels within feature maps are
   strongly correlated (as is normally the case in early convolution
   layers) then i.i.d. dropout will not regularize the activations and
   will otherwise just result in an effective learning rate decrease.
In this case, "nn.Dropout2d()" will help promote independence
   between feature maps and should be used instead.
Parameters:
      * p (float, optional) -- probability of an element to
        be zero-ed.
be zero-ed.
  * **inplace** (*bool**, **optional*) -- If set to "True", will
    do this operation in-place

Warning:
 Due to historical reasons, this class will perform 1D channel-
 wise dropout for 3D inputs (as done by "nn.Dropout1d"). Thus, it
 currently does NOT support inputs without a batch dimension of
 shape (C, H, W). This behavior will change in a future release to
 interpret 3D inputs as no-batch-dim inputs. To maintain the old
 behavior, switch to "nn.Dropout1d".

Shape:
      * Input: (N, C, H, W) or (N, C, L).
  * Output: (N, C, H, W) or (N, C, L) (same shape as input).

Examples:
  >>> m = nn.Dropout2d(p=0.2)
  >>> input = torch.randn(20, 16, 32, 32)
  >>> output = m(input)
torch.linalg.svd
torch.linalg.svd(A, full_matrices=True, *, driver=None, out=None)
Computes the singular value decomposition (SVD) of a matrix.
Letting \mathbb{K} be \mathbb{R} or \mathbb{C}, the full SVD of
   a matrix A \in \mathbb{K}^{m \times n}, if k = min(m,n), is
   defined as
  A = U \operatorname{diag}(S) V^{\text{H}} \mathrlap{\qquad U \in
  \mathbb{K}^{m \times m}, S \in \mathbb{R}^k, V \in \mathbb{K}^{n
  \times n}}

where \operatorname{diag}(S) \in \mathbb{K}^{m \times n},
   V^{\text{H}} is the conjugate transpose when V is complex, and the
   transpose when V is real-valued. The matrices  U, V (and thus
   V^{\text{H}}) are orthogonal in the real case, and unitary in the
   complex case.
When m > n (resp. m < n) we can drop the last m - n (resp. n
   - m) columns of U (resp. V) to form the reduced SVD:
  A = U \operatorname{diag}(S) V^{\text{H}} \mathrlap{\qquad U \in
\mathbb{K}^{m \times k}, S \in \mathbb{R}^k, V \in \mathbb{K}^{k
      \times n}}
where \operatorname{diag}(S) \in \mathbb{K}^{k \times k}. In this
   case, U and V also have orthonormal columns.
Supports input of float, double, cfloat and cdouble dtypes. Also
   supports batches of matrices, and if "A" is a batch of matrices
   then the output has the same batch dimensions.
The returned decomposition is a named tuple (U, S, Vh) which
   corresponds to U, S, V^{\text{H}} above.
The singular values are returned in descending order.
The parameter "full_matrices" chooses between the full (default)
   and reduced SVD.
The "driver" kwarg may be used in CUDA with a cuSOLVER backend to
   choose the algorithm used to compute the SVD. The choice of a
   driver is a trade-off between accuracy and speed.


If "A" is well-conditioned (its condition number is not too
     large), or you do not mind some precision loss.

For a general matrix: 'gesvdj' (Jacobi method)
If "A" is tall or wide (m >> n or m << n): 'gesvda'
       (Approximate method)


If "A" is not well-conditioned or precision is relevant:
     'gesvd' (QR based)


By default ("driver"= None), we call 'gesvdj' and, if it fails,
   we fallback to 'gesvd'.
Differences with numpy.linalg.svd:

Unlike numpy.linalg.svd, this function always returns a tuple
     of three tensors and it doesn't support compute_uv argument.
     Please use "torch.linalg.svdvals()", which computes only the
     singular values, instead of compute_uv=False.

Note:
 When "full_matrices"*= True*, the gradients with respect to
 *U[..., :, min(m, n):]* and *Vh[..., min(m, n):, :]* will be
 ignored, as those vectors can be arbitrary bases of the
 corresponding subspaces.

Warning:
 The returned tensors *U* and *V* are not unique, nor are they
 continuous with respect to "A". Due to this lack of uniqueness,
different hardware and software may compute different singular
     vectors.This non-uniqueness is caused by the fact that
     multiplying any pair of singular vectors u_k, v_k by -1 in the
     real case or by e^{i \phi}, \phi \in \mathbb{R} in the complex
     case produces another two valid singular vectors of the matrix.
     For this reason, the loss function shall not depend on this e^{i
     \phi} quantity, as it is not well-defined. This is checked for
     complex inputs when computing the gradients of this function. As
     such, when inputs are complex and are on a CUDA device, the
     computation of the gradients of this function synchronizes that
     device with the CPU.
Warning:
 Gradients computed using *U* or *Vh* will only be finite when "A"
 does not have repeated singular values. If "A" is rectangular,
 additionally, zero must also not be one of its singular values.
 Furthermore, if the distance between any two singular values is
close to zero, the gradient will be numerically unstable, as it
     depends on the singular values \sigma_i through the computation
     of \frac{1}{\min_{i \neq j} \sigma_i^2 - \sigma_j^2}. In the
     rectangular case, the gradient will also be numerically unstable
     when "A" has small singular values, as it also depends on the
     computation of \frac{1}{\sigma_i}.
See also:
 "torch.linalg.svdvals()" computes only the singular values.
 Unlike "torch.linalg.svd()", the gradients of "svdvals()" are
 always numerically stable.

 "torch.linalg.eig()" for a function that computes another type of
 spectral decomposition of a matrix. The eigendecomposition works
 just on square matrices.

 "torch.linalg.eigh()" for a (faster) function that computes the
 eigenvalue decomposition for Hermitian and symmetric matrices.

 "torch.linalg.qr()" for another (much faster) decomposition that
 works on general matrices.

Parameters:
works on general matrices.
Parameters:
      * A (Tensor) -- tensor of shape (, m, n)* where *** is
        zero or more batch dimensions.
  * **full_matrices** (*bool**, **optional*) -- controls whether
    to compute the full or reduced SVD, and consequently, the
    shape of the returned tensors *U* and *Vh*. Default: *True*.

Keyword Arguments:
      * driver (str, optional) -- name of the cuSOLVER
        method to be used. This keyword argument only works on CUDA
        inputs. Available options are: None, gesvd, gesvdj, and
        gesvda. Default: None.
  * **out** (*tuple**, **optional*) -- output tuple of three
    tensors. Ignored if *None*.

Returns:
      A named tuple (U, S, Vh) which corresponds to U, S,
      V^{\text{H}} above.
  *S* will always be real-valued, even when "A" is complex. It
  will also be ordered in descending order.

  *U* and *Vh* will have the same dtype as "A". The left / right
singular vectors will be given by the columns of U and the
      rows of Vh respectively.
Examples:
  >>> A = torch.randn(5, 3)
  >>> U, S, Vh = torch.linalg.svd(A, full_matrices=False)
  >>> U.shape, S.shape, Vh.shape
  (torch.Size([5, 3]), torch.Size([3]), torch.Size([3, 3]))
  >>> torch.dist(A, U @ torch.diag(S) @ Vh)
  tensor(1.0486e-06)

  >>> U, S, Vh = torch.linalg.svd(A)
  >>> U.shape, S.shape, Vh.shape
  (torch.Size([5, 5]), torch.Size([3]), torch.Size([3, 3]))
  >>> torch.dist(A, U[:, :3] @ torch.diag(S) @ Vh)
  tensor(1.0486e-06)

  >>> A = torch.randn(7, 5, 3)
  >>> U, S, Vh = torch.linalg.svd(A, full_matrices=False)
  >>> torch.dist(A, U @ torch.diag_embed(S) @ Vh)
  tensor(3.0957e-06)
torch.Tensor.record_stream
Tensor.record_stream(stream)
Ensures that the tensor memory is not reused for another tensor
   until all current work queued on "stream" are complete.
Note:
 The caching allocator is aware of only the stream where a tensor
 was allocated. Due to the awareness, it already correctly manages
 the life cycle of tensors on only one stream. But if a tensor is
 used on a stream different from the stream of origin, the
 allocator might reuse the memory unexpectedly. Calling this
 method lets the allocator know which streams have used the
 tensor.
LazyBatchNorm3d
class torch.nn.LazyBatchNorm3d(eps=1e-05, momentum=0.1, affine=True, track_running_stats=True, device=None, dtype=None)
A "torch.nn.BatchNorm3d" module with lazy initialization of the
   "num_features" argument of the "BatchNorm3d" that is inferred from
   the "input.size(1)". The attributes that will be lazily initialized
   are weight, bias, running_mean and running_var.
Check the "torch.nn.modules.lazy.LazyModuleMixin" for further
   documentation on lazy modules and their limitations.
Parameters:
      * eps (float) -- a value added to the denominator for
        numerical stability. Default: 1e-5
  * **momentum** (*float*) -- the value used for the running_mean
    and running_var computation. Can be set to "None" for
    cumulative moving average (i.e. simple average). Default: 0.1

  * **affine** (*bool*) -- a boolean value that when set to
    "True", this module has learnable affine parameters. Default:
    "True"
"True"
  * **track_running_stats** (*bool*) -- a boolean value that when
    set to "True", this module tracks the running mean and
    variance, and when set to "False", this module does not track
    such statistics, and initializes statistics buffers
    "running_mean" and "running_var" as "None". When these buffers
    are "None", this module always uses batch statistics. in both
    training and eval modes. Default: "True"

cls_to_become
  alias of "BatchNorm3d"
MultiLabelSoftMarginLoss
class torch.nn.MultiLabelSoftMarginLoss(weight=None, size_average=None, reduce=None, reduction='mean')
Creates a criterion that optimizes a multi-label one-versus-all
   loss based on max-entropy, between input x and target y of size (N,
   C). For each sample in the minibatch:
  loss(x, y) = - \frac{1}{C} * \sum_i y[i] * \log((1 +
  \exp(-x[i]))^{-1})                  + (1-y[i]) *
  \log\left(\frac{\exp(-x[i])}{(1 + \exp(-x[i]))}\right)

where i \in \left{0, \; \cdots , \; \text{x.nElement}() -
   1\right}, y[i] \in \left{0, \; 1\right}.
Parameters:
      * weight (Tensor, optional) -- a manual rescaling
        weight given to each class. If given, it has to be a Tensor of
        size C. Otherwise, it is treated as if having all ones.
  * **size_average** (*bool**, **optional*) -- Deprecated (see
    "reduction"). By default, the losses are averaged over each
loss element in the batch. Note that for some losses, there
        are multiple elements per sample. If the field "size_average"
        is set to "False", the losses are instead summed for each
        minibatch. Ignored when "reduce" is "False". Default: "True"
  * **reduce** (*bool**, **optional*) -- Deprecated (see
    "reduction"). By default, the losses are averaged or summed
    over observations for each minibatch depending on
    "size_average". When "reduce" is "False", returns a loss per
    batch element instead and ignores "size_average". Default:
    "True"

  * **reduction** (*str**, **optional*) -- Specifies the reduction
    to apply to the output: "'none'" | "'mean'" | "'sum'".
    "'none'": no reduction will be applied, "'mean'": the sum of
    the output will be divided by the number of elements in the
    output, "'sum'": the output will be summed. Note:
    "size_average" and "reduce" are in the process of being
deprecated, and in the meantime, specifying either of those
        two args will override "reduction". Default: "'mean'"
Shape:
      * Input: (N, C) where N is the batch size and C is the
        number of classes.
  * Target: (N, C), label targets padded by -1 ensuring same shape
    as the input.

  * Output: scalar. If "reduction" is "'none'", then (N).
torch.nn.functional.fold
torch.nn.functional.fold(input, output_size, kernel_size, dilation=1, padding=0, stride=1)
Combines an array of sliding local blocks into a large containing
   tensor.
Warning:
 Currently, only unbatched (3D) or batched (4D) image-like output
 tensors are supported.

See "torch.nn.Fold" for details
Return type:
      Tensor
torch.autograd.graph.Node.register_hook
abstract Node.register_hook(fn)
Registers a backward hook.
The hook will be called every time a gradient with respect to the
   Node is computed. The hook should have the following signature:
  hook(grad_inputs: Tuple[Tensor], grad_outputs: Tuple[Tensor]) -> Tuple[Tensor] or None

The hook should not modify its argument, but it can optionally
   return a new gradient which will be used in place of
   "grad_outputs".
This function returns a handle with a method "handle.remove()" that
   removes the hook from the module.
Note:
 See Backward Hooks execution for more information on how when
 this hook is executed, and how its execution is ordered relative
 to other hooks.

Example:
  >>> import torch
  >>> a = torch.tensor([0., 0., 0.], requires_grad=True)
  >>> b = a.clone()
  >>> assert isinstance(b.grad_fn, torch.autograd.graph.Node)
handle = b.grad_fn.register_hook(lambda gI, gO: (gO[0] * 2,))
      >>> b.sum().backward(retain_graph=True)
      >>> print(a.grad)
      tensor([2., 2., 2.])
      >>> handle.remove() # Removes the hook
      >>> a.grad = None
      >>> b.sum().backward(retain_graph=True)
      >>> print(a.grad)
      tensor([1., 1., 1.])



Return type:
      RemovableHandle
torch.histc
torch.histc(input, bins=100, min=0, max=0, *, out=None) -> Tensor
Computes the histogram of a tensor.
The elements are sorted into equal width bins between "min" and
   "max". If "min" and "max" are both zero, the minimum and maximum
   values of the data are used.
Elements lower than min and higher than max and "NaN" elements are
   ignored.
Parameters:
      * input (Tensor) -- the input tensor.
  * **bins** (*int*) -- number of histogram bins

  * **min** (*Scalar*) -- lower end of the range (inclusive)

  * **max** (*Scalar*) -- upper end of the range (inclusive)

Keyword Arguments:
      out (Tensor, optional) -- the output tensor.
Returns:
      Histogram represented as a tensor
Return type:
      Tensor
Example:
  >>> torch.histc(torch.tensor([1., 2, 1]), bins=4, min=0, max=3)
  tensor([ 0.,  2.,  1.,  0.])
torch.linalg.tensorsolve
torch.linalg.tensorsolve(A, B, dims=None, *, out=None) -> Tensor
Computes the solution X to the system torch.tensordot(A, X) =
   B.
If m is the product of the first "B".ndim  dimensions of "A"
   and n is the product of the rest of the dimensions, this function
   expects m and n to be equal.
The returned tensor x satisfies tensordot("A", x, dims=x.ndim)
   == "B". x has shape "A"[B.ndim:].
If "dims" is specified, "A" will be reshaped as
  A = movedim(A, dims, range(len(dims) - A.ndim + 1, 0))

Supports inputs of float, double, cfloat and cdouble dtypes.
See also:
 "torch.linalg.tensorinv()" computes the multiplicative inverse of
 "torch.tensordot()".

Parameters:
      * A (Tensor) -- tensor to solve for. Its shape must
        satisfy prod("A".shape[:"B".ndim]) ==
        prod("A".shape["B".ndim:]).
  * **B** (*Tensor*) -- tensor of shape "A"*.shape[:*"B"*.ndim]*.
dims (Tuple[int], optional) -- dimensions of
        "A" to be moved. If None, no dimensions are moved. Default:
        None.

Keyword Arguments:
      out (Tensor, optional) -- output tensor. Ignored if
      None. Default: None.
Raises:
      RuntimeError -- if the reshaped "A".view(m, m) with m as
      above  is not     invertible or the product of the first "ind"
      dimensions is not equal     to the product of the rest of the
      dimensions.
Examples:
  >>> A = torch.eye(2 * 3 * 4).reshape((2 * 3, 4, 2, 3, 4))
  >>> B = torch.randn(2 * 3, 4)
  >>> X = torch.linalg.tensorsolve(A, B)
  >>> X.shape
  torch.Size([2, 3, 4])
  >>> torch.allclose(torch.tensordot(A, X, dims=X.ndim), B)
  True

  >>> A = torch.randn(6, 4, 4, 3, 2)
  >>> B = torch.randn(4, 3, 2)
  >>> X = torch.linalg.tensorsolve(A, B, dims=(0, 2))
  >>> X.shape
  torch.Size([6, 4])
  >>> A = A.permute(1, 3, 4, 0, 2)
torch.Tensor.new_full
Tensor.new_full(size, fill_value, *, dtype=None, device=None, requires_grad=False, layout=torch.strided, pin_memory=False) -> Tensor
Returns a Tensor of size "size" filled with "fill_value". By
   default, the returned Tensor has the same "torch.dtype" and
   "torch.device" as this tensor.
Parameters:
      fill_value (scalar) -- the number to fill the output
      tensor with.
Keyword Arguments:
      * dtype ("torch.dtype", optional) -- the desired type of
        returned tensor. Default: if None, same "torch.dtype" as this
        tensor.
  * **device** ("torch.device", optional) -- the desired device of
    returned tensor. Default: if None, same "torch.device" as this
    tensor.

  * **requires_grad** (*bool**, **optional*) -- If autograd should
    record operations on the returned tensor. Default: "False".

  * **layout** ("torch.layout", optional) -- the desired layout of
returned Tensor. Default: "torch.strided".
  * **pin_memory** (*bool**, **optional*) -- If set, returned
    tensor would be allocated in the pinned memory. Works only for
    CPU tensors. Default: "False".

Example:
  >>> tensor = torch.ones((2,), dtype=torch.float64)
  >>> tensor.new_full((3, 4), 3.141592)
  tensor([[ 3.1416,  3.1416,  3.1416,  3.1416],
          [ 3.1416,  3.1416,  3.1416,  3.1416],
          [ 3.1416,  3.1416,  3.1416,  3.1416]], dtype=torch.float64)
torch.sspaddmm
torch.sspaddmm(input, mat1, mat2, *, beta=1, alpha=1, out=None) -> Tensor
Matrix multiplies a sparse tensor "mat1" with a dense tensor
   "mat2", then adds the sparse tensor "input" to the result.
Note: This function is equivalent to "torch.addmm()", except
   "input" and "mat1" are sparse.
Parameters:
      * input (Tensor) -- a sparse matrix to be added
  * **mat1** (*Tensor*) -- a sparse matrix to be matrix multiplied

  * **mat2** (*Tensor*) -- a dense matrix to be matrix multiplied

Keyword Arguments:
      * beta (Number, optional) -- multiplier for "mat"
        (\beta)
  * **alpha** (*Number**, **optional*) -- multiplier for mat1 @
    mat2 (\alpha)

  * **out** (*Tensor**, **optional*) -- the output tensor.
ParameterList
class torch.nn.ParameterList(values=None)
Holds parameters in a list.
"ParameterList" can be used like a regular Python list, but Tensors
   that are "Parameter" are properly registered, and will be visible
   by all "Module" methods.
Note that the constructor, assigning an element of the list, the
   "append()" method and the "extend()" method will convert any
   "Tensor" into "Parameter".
Parameters:
      parameters (iterable, optional) -- an iterable of
      elements to add to the list.
Example:
  class MyModule(nn.Module):
      def __init__(self):
          super(MyModule, self).__init__()
          self.params = nn.ParameterList([nn.Parameter(torch.randn(10, 10)) for i in range(10)])

      def forward(self, x):
          # ParameterList can act as an iterable, or be indexed using ints
          for i, p in enumerate(self.params):
              x = self.params[i // 2].mm(x) + p.mm(x)
          return x
return x
append(value)
  Appends a given value at the end of the list.

  Parameters:
     **value** (*Any*) -- value to append

  Return type:
     *ParameterList*

extend(values)
  Appends values from a Python iterable to the end of the list.

  Parameters:
     **values** (*iterable*) -- iterable of values to append

  Return type:
     *ParameterList*
torch.sinh
torch.sinh(input, *, out=None) -> Tensor
Returns a new tensor with the hyperbolic sine of the elements of
   "input".
  \text{out}_{i} = \sinh(\text{input}_{i})

Parameters:
      input (Tensor) -- the input tensor.
Keyword Arguments:
      out (Tensor, optional) -- the output tensor.
Example:
  >>> a = torch.randn(4)
  >>> a
  tensor([ 0.5380, -0.8632, -0.1265,  0.9399])
  >>> torch.sinh(a)
  tensor([ 0.5644, -0.9744, -0.1268,  1.0845])

Note:
 When "input" is on the CPU, the implementation of torch.sinh may
 use the Sleef library, which rounds very large results to
 infinity or negative infinity. See here for details.
inference_mode
class torch.inference_mode(mode=True)
Context-manager that enables or disables inference mode
InferenceMode is a new context manager analogous to "no_grad" to be
   used when you are certain your operations will have no interactions
   with autograd (e.g., model training). Code run under this mode gets
   better performance by disabling view tracking and version counter
   bumps. Note that unlike some other mechanisms that locally enable
   or disable grad, entering inference_mode also disables to forward-
   mode AD.
This context manager is thread local; it will not affect
   computation in other threads.
Also functions as a decorator. (Make sure to instantiate with
   parenthesis.)
Note:
 Inference mode is one of several mechanisms that can enable or
 disable gradients locally see Locally disabling gradient
 computation for more information on how they compare.

Parameters:
Parameters:
      mode (bool) -- Flag whether to enable or disable inference
      mode
Example::
      >>> import torch
      >>> x = torch.ones(1, 2, 3, requires_grad=True)
      >>> with torch.inference_mode():
      ...     y = x * x
      >>> y.requires_grad
      False
      >>> y._version
      Traceback (most recent call last):
      File "", line 1, in 
      RuntimeError: Inference tensors do not track version counter.
      >>> @torch.inference_mode()
      ... def func(x):
      ...     return x * x
      >>> out = func(x)
      >>> out.requires_grad
      False
convert
class torch.quantization.convert(module, mapping=None, inplace=False, remove_qconfig=True, is_reference=False, convert_custom_config_dict=None)
Converts submodules in input module to a different module according
   to mapping by calling from_float method on the target module
   class. And remove qconfig at the end if remove_qconfig is set to
   True.
Parameters:
      * module -- prepared and calibrated module
  * **mapping** -- a dictionary that maps from source module type
    to target module type, can be overwritten to allow swapping
    user defined Modules

  * **inplace** -- carry out model transformations in-place, the
    original module is mutated

  * **convert_custom_config_dict** -- custom configuration
    dictionary for convert function

  # Example of convert_custom_config_dict:
  convert_custom_config_dict = {
      # user will manually define the corresponding quantized
graph
class torch.cuda.graph(cuda_graph, pool=None, stream=None)
Context-manager that captures CUDA work into a
   "torch.cuda.CUDAGraph" object for later replay.
See CUDA Graphs for a general introduction, detailed use, and
   constraints.
Parameters:
      * cuda_graph (torch.cuda.CUDAGraph) -- Graph object used
        for capture.
  * **pool** (*optional*) -- Opaque token (returned by a call to
    "graph_pool_handle()" or "other_Graph_instance.pool()")
    hinting this graph's capture may share memory from the
    specified pool. See Graph memory management.

  * **stream** (*torch.cuda.Stream**, **optional*) -- If supplied,
    will be set as the current stream in the context. If not
    supplied, "graph" sets its own internal side stream as the
    current stream in the context.

Note:
 For effective memory sharing, if you pass a "pool" used by a
 previous capture and the previous capture used an explicit