shrinath-suresh/blogs-docs-splitted huggingface dataset:
False
      >>> torch.allclose(torch.tensor([10000., 1e-08]), torch.tensor([10000.1, 1e-09]))
      True
      >>> torch.allclose(torch.tensor([1.0, float('nan')]), torch.tensor([1.0, float('nan')]))
      False
      >>> torch.allclose(torch.tensor([1.0, float('nan')]), torch.tensor([1.0, float('nan')]), equal_nan=True)
      True
FeatureAlphaDropout
class torch.nn.FeatureAlphaDropout(p=0.5, inplace=False)
Randomly masks out entire channels (a channel is a feature map,
   e.g. the j-th channel of the i-th sample in the batch input is a
   tensor \text{input}[i, j]) of the input tensor). Instead of setting
   activations to zero, as in regular Dropout, the activations are set
   to the negative saturation value of the SELU activation function.
   More details can be found in the paper Self-Normalizing Neural
   Networks .
Each element will be masked independently for each sample on every
   forward call with probability "p" using samples from a Bernoulli
   distribution. The elements to be masked are randomized on every
   forward call, and scaled and shifted to maintain zero mean and unit
   variance.
Usually the input comes from "nn.AlphaDropout" modules.
As described in the paper Efficient Object Localization Using
   Convolutional Networks , if adjacent pixels within feature maps are
strongly correlated (as is normally the case in early convolution
   layers) then i.i.d. dropout will not regularize the activations and
   will otherwise just result in an effective learning rate decrease.
In this case, "nn.AlphaDropout()" will help promote independence
   between feature maps and should be used instead.
Parameters:
      * p (float, optional) -- probability of an element to
        be zeroed. Default: 0.5
  * **inplace** (*bool**, **optional*) -- If set to "True", will
    do this operation in-place

Shape:
      * Input: (N, C, D, H, W) or (C, D, H, W).
  * Output: (N, C, D, H, W) or (C, D, H, W) (same shape as input).

Examples:
  >>> m = nn.FeatureAlphaDropout(p=0.2)
  >>> input = torch.randn(20, 16, 4, 32, 32)
  >>> output = m(input)
torch.cuda.memory_reserved
torch.cuda.memory_reserved(device=None)
Returns the current GPU memory managed by the caching allocator in
   bytes for a given device.
Parameters:
      device (torch.device or int, optional) -- selected
      device. Returns statistic for the current device, given by
      "current_device()", if "device" is "None" (default).
Return type:
      int
Note:
 See Memory management for more details about GPU memory
 management.
torch.fft.ifft2
torch.fft.ifft2(input, s=None, dim=(- 2, - 1), norm=None, *, out=None) -> Tensor
Computes the 2 dimensional inverse discrete Fourier transform of
   "input". Equivalent to "ifftn()" but IFFTs only the last two
   dimensions by default.
Note:
 Supports torch.half and torch.chalf on CUDA with GPU Architecture
 SM53 or greater. However it only supports powers of 2 signal
 length in every transformed dimensions.

Parameters:
      * input (Tensor) -- the input tensor
  * **s** (*Tuple**[**int**]**, **optional*) -- Signal size in the
    transformed dimensions. If given, each dimension "dim[i]" will
    either be zero-padded or trimmed to the length "s[i]" before
    computing the IFFT. If a length "-1" is specified, no padding
    is done in that dimension. Default: "s = [input.size(d) for d
    in dim]"

  * **dim** (*Tuple**[**int**]**, **optional*) -- Dimensions to be
    transformed. Default: last two dimensions.
norm (str, optional) --Normalization mode. For the backward transform ("ifft2()"),
these correspond to:

* ""forward"" - no normalization

* ""backward"" - normalize by "1/n"

* ""ortho"" - normalize by "1/sqrt(n)" (making the IFFT
  orthonormal)

Where "n = prod(s)" is the logical IFFT size. Calling the
forward transform ("fft2()") with the same normalization mode
will apply an overall normalization of "1/n" between the two
transforms. This is required to make "ifft2()" the exact
inverse.

Default is ""backward"" (normalize by "1/n").



Keyword Arguments:
      out (Tensor, optional) -- the output tensor.
-[ Example ]-



x = torch.rand(10, 10, dtype=torch.complex64)
ifft2 = torch.fft.ifft2(x)



The discrete Fourier transform is separable, so "ifft2()" here is
   equivalent to two one-dimensional "ifft()" calls:
ConvTranspose3d
class torch.nn.ConvTranspose3d(in_channels, out_channels, kernel_size, stride=1, padding=0, output_padding=0, groups=1, bias=True, dilation=1, padding_mode='zeros', device=None, dtype=None)
Applies a 3D transposed convolution operator over an input image
   composed of several input planes. The transposed convolution
   operator multiplies each input value element-wise by a learnable
   kernel, and sums over the outputs from all input feature planes.
This module can be seen as the gradient of Conv3d with respect to
   its input. It is also known as a fractionally-strided convolution
   or a deconvolution (although it is not an actual deconvolution
   operation as it does not compute a true inverse of convolution).
   For more information, see the visualizations here and the
   Deconvolutional Networks paper.
This module supports TensorFloat32.
On certain ROCm devices, when using float16 inputs this module will
   use different precision for backward.
use different precision for backward.


"stride" controls the stride for the cross-correlation.


"padding" controls the amount of implicit zero padding on both
     sides for "dilation * (kernel_size - 1) - padding" number of
     points. See note below for details.


"output_padding" controls the additional size added to one side
     of the output shape. See note below for details.


"dilation" controls the spacing between the kernel points; also
     known as the Ã  trous algorithm. It is harder to describe, but the
     link here has a nice visualization of what "dilation" does.


"groups" controls the connections between inputs and outputs.
     "in_channels" and "out_channels" must both be divisible by
     "groups". For example,
* At groups=1, all inputs are convolved to all outputs.

* At groups=2, the operation becomes equivalent to having two
  conv layers side by side, each seeing half the input
channels and producing half the output channels, and both
          subsequently concatenated.
    * At groups= "in_channels", each input channel is convolved
      with its own set of filters (of size
      \frac{\text{out\_channels}}{\text{in\_channels}}).

The parameters "kernel_size", "stride", "padding", "output_padding"
   can either be:
  * a single "int" -- in which case the same value is used for the
    depth, height and width dimensions

  * a "tuple" of three ints -- in which case, the first *int* is
    used for the depth dimension, the second *int* for the height
    dimension and the third *int* for the width dimension

Note:
 The "padding" argument effectively adds "dilation * (kernel_size
 - 1) - padding" amount of zero padding to both sizes of the
 input. This is set so that when a "Conv3d" and a
 "ConvTranspose3d" are initialized with same parameters, they are
inverses of each other in regard to the input and output shapes.
     However, when "stride > 1", "Conv3d" maps multiple input shapes
     to the same output shape. "output_padding" is provided to resolve
     this ambiguity by effectively increasing the calculated output
     shape on one side. Note that "output_padding" is only used to
     find output shape, but does not actually add zero-padding to
     output.
Note:
 In some circumstances when given tensors on a CUDA device and
 using CuDNN, this operator may select a nondeterministic
 algorithm to increase performance. If this is undesirable, you
 can try to make the operation deterministic (potentially at a
 performance cost) by setting "torch.backends.cudnn.deterministic
 = True". See Reproducibility for more information.

Parameters:
      * in_channels (int) -- Number of channels in the input
        image
  * **out_channels** (*int*) -- Number of channels produced by the
    convolution
convolution
  * **kernel_size** (*int** or **tuple*) -- Size of the convolving
    kernel

  * **stride** (*int** or **tuple**, **optional*) -- Stride of the
    convolution. Default: 1

  * **padding** (*int** or **tuple**, **optional*) -- "dilation *
    (kernel_size - 1) - padding" zero-padding will be added to
    both sides of each dimension in the input. Default: 0

  * **output_padding** (*int** or **tuple**, **optional*) --
    Additional size added to one side of each dimension in the
    output shape. Default: 0

  * **groups** (*int**, **optional*) -- Number of blocked
    connections from input channels to output channels. Default: 1

  * **bias** (*bool**, **optional*) -- If "True", adds a learnable
    bias to the output. Default: "True"

  * **dilation** (*int** or **tuple**, **optional*) -- Spacing
    between kernel elements. Default: 1

Shape:
      * Input: (N, C_{in}, D_{in}, H_{in}, W_{in}) or (C_{in}, D_{in},
H_{in}, W_{in})
  * Output: (N, C_{out}, D_{out}, H_{out}, W_{out}) or (C_{out},
    D_{out}, H_{out}, W_{out}), where

     D_{out} = (D_{in} - 1) \times \text{stride}[0] - 2 \times
     \text{padding}[0] + \text{dilation}[0]           \times
     (\text{kernel\_size}[0] - 1) + \text{output\_padding}[0] + 1

     H_{out} = (H_{in} - 1) \times \text{stride}[1] - 2 \times
     \text{padding}[1] + \text{dilation}[1]           \times
     (\text{kernel\_size}[1] - 1) + \text{output\_padding}[1] + 1

     W_{out} = (W_{in} - 1) \times \text{stride}[2] - 2 \times
     \text{padding}[2] + \text{dilation}[2]           \times
     (\text{kernel\_size}[2] - 1) + \text{output\_padding}[2] + 1

Variables:
      * weight (Tensor) -- the learnable weights of the module
        of shape (\text{in_channels},
        \frac{\text{out_channels}}{\text{groups}},
        \text{kernel_size[0]}, \text{kernel_size[1]},
\text{kernel_size[2]}). The values of these weights are
        sampled from \mathcal{U}(-\sqrt{k}, \sqrt{k}) where k =
        \frac{groups}{C_\text{out} *
        \prod_{i=0}^{2}\text{kernel_size}[i]}
  * **bias** (*Tensor*) -- the learnable bias of the module of
    shape (out_channels) If "bias" is "True", then the values of
    these weights are sampled from \mathcal{U}(-\sqrt{k},
    \sqrt{k}) where k = \frac{groups}{C_\text{out} *
    \prod_{i=0}^{2}\text{kernel\_size}[i]}

Examples:
  >>> # With square kernels and equal stride
  >>> m = nn.ConvTranspose3d(16, 33, 3, stride=2)
  >>> # non-square kernels and unequal stride and with padding
  >>> m = nn.ConvTranspose3d(16, 33, (3, 5, 2), stride=(2, 1, 1), padding=(0, 4, 2))
  >>> input = torch.randn(20, 16, 10, 50, 100)
  >>> output = m(input)
torch.cholesky_inverse
torch.cholesky_inverse(input, upper=False, *, out=None) -> Tensor
Computes the inverse of a symmetric positive-definite matrix A
   using its Cholesky factor u: returns matrix "inv". The inverse is
   computed using LAPACK routines "dpotri" and "spotri" (and the
   corresponding MAGMA routines).
If "upper" is "False", u is lower triangular such that the returned
   tensor is
  inv = (uu^{{T}})^{{-1}}

If "upper" is "True" or not provided, u is upper triangular such
   that the returned tensor is
  inv = (u^T u)^{{-1}}

Supports input of float, double, cfloat and cdouble dtypes. Also
   supports batches of matrices, and if A is a batch of matrices then
   the output has the same batch dimensions.
Parameters:
      * input (Tensor) -- the input tensor A of size (*, n, n),
        consisting of symmetric positive-definite matrices where * is
        zero or more batch dimensions.
zero or more batch dimensions.
  * **upper** (*bool**, **optional*) -- flag that indicates
    whether to return a upper or lower triangular matrix. Default:
    False

Keyword Arguments:
      out (Tensor, optional) -- the output tensor for inv
Example:
  >>> a = torch.randn(3, 3)
  >>> a = torch.mm(a, a.t()) + 1e-05 * torch.eye(3) # make symmetric positive definite
  >>> u = torch.linalg.cholesky(a)
  >>> a
  tensor([[  0.9935,  -0.6353,   1.5806],
          [ -0.6353,   0.8769,  -1.7183],
          [  1.5806,  -1.7183,  10.6618]])
  >>> torch.cholesky_inverse(u)
  tensor([[ 1.9314,  1.2251, -0.0889],
          [ 1.2251,  2.4439,  0.2122],
          [-0.0889,  0.2122,  0.1412]])
  >>> a.inverse()
  tensor([[ 1.9314,  1.2251, -0.0889],
          [ 1.2251,  2.4439,  0.2122],
          [-0.0889,  0.2122,  0.1412]])
  >>> a = torch.randn(3, 2, 2) # Example for batched input
GroupNorm
class torch.nn.GroupNorm(num_groups, num_channels, eps=1e-05, affine=True, device=None, dtype=None)
Applies Group Normalization over a mini-batch of inputs as
   described in the paper Group Normalization
  y = \frac{x - \mathrm{E}[x]}{ \sqrt{\mathrm{Var}[x] + \epsilon}}
  * \gamma + \beta

The input channels are separated into "num_groups" groups, each
   containing "num_channels / num_groups" channels. "num_channels"
   must be divisible by "num_groups". The mean and standard-deviation
   are calculated separately over the each group. \gamma and \beta are
   learnable per-channel affine transform parameter vectors of size
   "num_channels" if "affine" is "True". The standard-deviation is
   calculated via the biased estimator, equivalent to
   torch.var(input, unbiased=False).
This layer uses statistics computed from input data in both
   training and evaluation modes.
Parameters:
      * num_groups (int) -- number of groups to separate the
channels into
  * **num_channels** (*int*) -- number of channels expected in
    input

  * **eps** (*float*) -- a value added to the denominator for
    numerical stability. Default: 1e-5

  * **affine** (*bool*) -- a boolean value that when set to
    "True", this module has learnable per-channel affine
    parameters initialized to ones (for weights) and zeros (for
    biases). Default: "True".

Shape:
      * Input: (N, C, *) where C=\text{num_channels}
  * Output: (N, C, *) (same shape as input)

Examples:
  >>> input = torch.randn(20, 6, 10, 10)
  >>> # Separate 6 channels into 3 groups
  >>> m = nn.GroupNorm(3, 6)
  >>> # Separate 6 channels into 6 groups (equivalent with InstanceNorm)
  >>> m = nn.GroupNorm(6, 6)
  >>> # Put all 6 channels into a single group (equivalent with LayerNorm)
  >>> m = nn.GroupNorm(1, 6)
  >>> # Activating the module
  >>> output = m(input)
torch.cuda.caching_allocator_alloc
torch.cuda.caching_allocator_alloc(size, device=None, stream=None)
Performs a memory allocation using the CUDA memory allocator.
Memory is allocated for a given device and a stream, this function
   is intended to be used for interoperability with other frameworks.
   Allocated memory is released through "caching_allocator_delete()".
Parameters:
      * size (int) -- number of bytes to be allocated.
  * **device** (*torch.device** or **int**, **optional*) --
    selected device. If it is "None" the default CUDA device is
    used.

  * **stream** (*torch.cuda.Stream** or **int**, **optional*) --
    selected stream. If is "None" then the default stream for the
    selected device is used.

Note:
 See Memory management for more details about GPU memory
 management.
BatchNorm1d
class torch.nn.BatchNorm1d(num_features, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True, device=None, dtype=None)
Applies Batch Normalization over a 2D or 3D input as described in
   the paper Batch Normalization: Accelerating Deep Network Training
   by Reducing Internal Covariate Shift .
  y = \frac{x - \mathrm{E}[x]}{\sqrt{\mathrm{Var}[x] + \epsilon}}
  * \gamma + \beta

The mean and standard-deviation are calculated per-dimension over
   the mini-batches and \gamma and \beta are learnable parameter
   vectors of size C (where C is the number of features or
   channels of the input). By default, the elements of \gamma are set
   to 1 and the elements of \beta are set to 0. The standard-deviation
   is calculated via the biased estimator, equivalent to
   torch.var(input, unbiased=False).
Also by default, during training this layer keeps running estimates
   of its computed mean and variance, which are then used for
normalization during evaluation. The running estimates are kept
   with a default "momentum" of 0.1.
If "track_running_stats" is set to "False", this layer then does
   not keep running estimates, and batch statistics are instead used
   during evaluation time as well.
Note:
 This "momentum" argument is different from one used in optimizer
 classes and the conventional notion of momentum. Mathematically,
 the update rule for running statistics here is \hat{x}_\text{new}
 = (1 - \text{momentum}) \times \hat{x} + \text{momentum} \times
 x_t, where \hat{x} is the estimated statistic and x_t is the new
 observed value.

Because the Batch Normalization is done over the C dimension,
   computing statistics on (N, L) slices, it's common terminology to
   call this Temporal Batch Normalization.
Parameters:
      * num_features (int) -- number of features or channels C
        of the input
  * **eps** (*float*) -- a value added to the denominator for
numerical stability. Default: 1e-5
  * **momentum** (*float*) -- the value used for the running_mean
    and running_var computation. Can be set to "None" for
    cumulative moving average (i.e. simple average). Default: 0.1

  * **affine** (*bool*) -- a boolean value that when set to
    "True", this module has learnable affine parameters. Default:
    "True"

  * **track_running_stats** (*bool*) -- a boolean value that when
    set to "True", this module tracks the running mean and
    variance, and when set to "False", this module does not track
    such statistics, and initializes statistics buffers
    "running_mean" and "running_var" as "None". When these buffers
    are "None", this module always uses batch statistics. in both
    training and eval modes. Default: "True"

Shape:
      * Input: (N, C) or (N, C, L), where N is the batch size, C is
        the number of features or channels, and L is the sequence
        length
length
  * Output: (N, C) or (N, C, L) (same shape as input)

Examples:
  >>> # With Learnable Parameters
  >>> m = nn.BatchNorm1d(100)
  >>> # Without Learnable Parameters
  >>> m = nn.BatchNorm1d(100, affine=False)
  >>> input = torch.randn(20, 100)
  >>> output = m(input)
torch.nn.functional.hardsigmoid
torch.nn.functional.hardsigmoid(input, inplace=False)
Applies the element-wise function
  \text{Hardsigmoid}(x) = \begin{cases}     0 & \text{if~} x \le
  -3, \\     1 & \text{if~} x \ge +3, \\     x / 6 + 1 / 2 &
  \text{otherwise} \end{cases}

Parameters:
      inplace (bool) -- If set to "True", will do this operation
      in-place. Default: "False"
Return type:
      Tensor
See "Hardsigmoid" for more details.
torch.addcmul
torch.addcmul(input, tensor1, tensor2, *, value=1, out=None) -> Tensor
Performs the element-wise multiplication of "tensor1" by "tensor2",
   multiplies the result by the scalar "value" and adds it to "input".
  \text{out}_i = \text{input}_i + \text{value} \times
  \text{tensor1}_i \times \text{tensor2}_i

The shapes of "tensor", "tensor1", and "tensor2" must be
   broadcastable.
For inputs of type FloatTensor or DoubleTensor, "value" must be
   a real number, otherwise an integer.
Parameters:
      * input (Tensor) -- the tensor to be added
  * **tensor1** (*Tensor*) -- the tensor to be multiplied

  * **tensor2** (*Tensor*) -- the tensor to be multiplied

Keyword Arguments:
      * value (Number, optional) -- multiplier for tensor1
        .* tensor2
  * **out** (*Tensor**, **optional*) -- the output tensor.

Example:
  >>> t = torch.randn(1, 3)
  >>> t1 = torch.randn(3, 1)
  >>> t2 = torch.randn(1, 3)
MovingAverageMinMaxObserver
class torch.quantization.observer.MovingAverageMinMaxObserver(averaging_constant=0.01, dtype=torch.quint8, qscheme=torch.per_tensor_affine, reduce_range=False, quant_min=None, quant_max=None, eps=1.1920928955078125e-07, **kwargs)
Observer module for computing the quantization parameters based on
   the moving average of the min and max values.
This observer computes the quantization parameters based on the
   moving averages of minimums and maximums of the incoming tensors.
   The module records the average minimum and maximum of incoming
   tensors, and uses this statistic to compute the quantization
   parameters.
Parameters:
      * averaging_constant -- Averaging constant for min/max.
  * **dtype** -- dtype argument to the *quantize* node needed to
    implement the reference model spec.

  * **qscheme** -- Quantization scheme to be used

  * **reduce_range** -- Reduces the range of the quantized data
type by 1 bit
  * **quant_min** -- Minimum quantization value. If unspecified,
    it will follow the 8-bit setup.

  * **quant_max** -- Maximum quantization value. If unspecified,
    it will follow the 8-bit setup.

  * **eps** (*Tensor*) -- Epsilon value for float32, Defaults to
    *torch.finfo(torch.float32).eps*.

The moving average min/max is computed as follows
  \begin{array}{ll}         x_\text{min} = \begin{cases}
  \min(X) & \text{if~}x_\text{min} = \text{None} \\             (1
  - c) x_\text{min} + c \min(X) & \text{otherwise}
  \end{cases}\\         x_\text{max} = \begin{cases}
  \max(X) & \text{if~}x_\text{max} = \text{None} \\             (1
  - c) x_\text{max} + c \max(X) & \text{otherwise}
  \end{cases}\\ \end{array}

where x_\text{min/max} is the running average min/max, X is is the
   incoming tensor, and c is the "averaging_constant".
The scale and zero point are then computed as in "MinMaxObserver".
Note:
Softsign
class torch.nn.Softsign
Applies the element-wise function:
  \text{SoftSign}(x) = \frac{x}{ 1 + |x|}

Shape:
      * Input: (*), where * means any number of dimensions.
  * Output: (*), same shape as the input.

[image]
Examples:
  >>> m = nn.Softsign()
  >>> input = torch.randn(2)
  >>> output = m(input)
torch.nn.utils.prune.remove
torch.nn.utils.prune.remove(module, name)
Removes the pruning reparameterization from a module and the
   pruning method from the forward hook. The pruned parameter named
   "name" remains permanently pruned, and the parameter named
   "name+'_orig'" is removed from the parameter list. Similarly, the
   buffer named "name+'_mask'" is removed from the buffers.
Note:
 Pruning itself is NOT undone or reversed!

Parameters:
      * module (nn.Module) -- module containing the tensor to
        prune
  * **name** (*str*) -- parameter name within "module" on which
    pruning will act.

-[ Examples ]-



m = random_unstructured(nn.Linear(5, 7), name='weight', amount=0.2)
m = remove(m, name='weight')
torch.take_along_dim
torch.take_along_dim(input, indices, dim, *, out=None) -> Tensor
Selects values from "input" at the 1-dimensional indices from
   "indices" along the given "dim".
Functions that return indices along a dimension, like
   "torch.argmax()" and "torch.argsort()", are designed to work with
   this function. See the examples below.
Note:
 This function is similar to NumPy's *take_along_axis*. See also
 "torch.gather()".

Parameters:
      * input (Tensor) -- the input tensor.
  * **indices** (*tensor*) -- the indices into "input". Must have
    long dtype.

  * **dim** (*int*) -- dimension to select along.

Keyword Arguments:
      out (Tensor, optional) -- the output tensor.
Example:
  >>> t = torch.tensor([[10, 30, 20], [60, 40, 50]])
  >>> max_idx = torch.argmax(t)
  >>> torch.take_along_dim(t, max_idx)
  tensor([60])
  >>> sorted_idx = torch.argsort(t, dim=1)
torch.nn.modules.module.register_module_forward_hook
torch.nn.modules.module.register_module_forward_hook(hook)
Registers a global forward hook for all the modules
Warning:
 This adds global state to the *nn.module* module and it is only
 intended for debugging/profiling purposes.

The hook will be called every time after "forward()" has computed
   an output. It should have the following signature:
  hook(module, input, output) -> None or modified output

The input contains only the positional arguments given to the
   module. Keyword arguments won't be passed to the hooks and only to
   the "forward". The hook can modify the output. It can modify the
   input inplace but it will not have effect on forward since this is
   called after "forward()" is called.
Returns:
      a handle that can be used to remove the added hook by calling
      "handle.remove()"
Return type:
      "torch.utils.hooks.RemovableHandle"