shrinath-suresh/blogs-docs-splitted huggingface dataset:
tensor types.
  * **requires_grad** (*bool**, **optional*) -- If autograd should
    record operations on the returned tensor. Default: "False".

  * **check_invariants** (*bool**, **optional*) -- If sparse
    tensor invariants are checked. Default: as returned by
    "torch.sparse.check_sparse_tensor_invariants.is_enabled()",
    initially False.

Example:
  >>> i = torch.tensor([[0, 1, 1],
  ...                   [2, 0, 2]])
  >>> v = torch.tensor([3, 4, 5], dtype=torch.float32)
  >>> torch.sparse_coo_tensor(i, v, [2, 4])
  tensor(indices=tensor([[0, 1, 1],
                         [2, 0, 2]]),
         values=tensor([3., 4., 5.]),
         size=(2, 4), nnz=3, layout=torch.sparse_coo)

  >>> torch.sparse_coo_tensor(i, v)  # Shape inference
  tensor(indices=tensor([[0, 1, 1],
                         [2, 0, 2]]),
         values=tensor([3., 4., 5.]),
         size=(2, 3), nnz=3, layout=torch.sparse_coo)
torch.sparse_coo_tensor(i, v, [2, 4],
      ...                         dtype=torch.float64,
      ...                         device=torch.device('cuda:0'))
      tensor(indices=tensor([[0, 1, 1],
                             [2, 0, 2]]),
             values=tensor([3., 4., 5.]),
             device='cuda:0', size=(2, 4), nnz=3, dtype=torch.float64,
             layout=torch.sparse_coo)



  # Create an empty sparse tensor with the following invariants:
  #   1. sparse_dim + dense_dim = len(SparseTensor.shape)
  #   2. SparseTensor._indices().shape = (sparse_dim, nnz)
  #   3. SparseTensor._values().shape = (nnz, SparseTensor.shape[sparse_dim:])
  #
  # For instance, to create an empty sparse tensor with nnz = 0, dense_dim = 0 and
  # sparse_dim = 1 (hence indices is a 2D tensor of shape = (1, 0))
  >>> S = torch.sparse_coo_tensor(torch.empty([1, 0]), [], [1])
  tensor(indices=tensor([], size=(1, 0)),
         values=tensor([], size=(0,)),
values=tensor([], size=(0,)),
             size=(1,), nnz=0, layout=torch.sparse_coo)
  # and to create an empty sparse tensor with nnz = 0, dense_dim = 1 and
  # sparse_dim = 1
  >>> S = torch.sparse_coo_tensor(torch.empty([1, 0]), torch.empty([0, 2]), [1, 2])
  tensor(indices=tensor([], size=(1, 0)),
         values=tensor([], size=(0, 2)),
         size=(1, 2), nnz=0, layout=torch.sparse_coo)
LeakyReLU
class torch.nn.LeakyReLU(negative_slope=0.01, inplace=False)
Applies the element-wise function:
  \text{LeakyReLU}(x) = \max(0, x) + \text{negative\_slope} *
  \min(0, x)

or
  \text{LeakyReLU}(x) = \begin{cases} x, & \text{ if } x \geq 0 \\
  \text{negative\_slope} \times x, & \text{ otherwise }
  \end{cases}

Parameters:
      * negative_slope (float) -- Controls the angle of the
        negative slope. Default: 1e-2
  * **inplace** (*bool*) -- can optionally do the operation in-
    place. Default: "False"

Shape:
      * Input: (*) where *** means, any number of additional
        dimensions
  * Output: (*), same shape as the input

[image]
Examples:
  >>> m = nn.LeakyReLU(0.1)
  >>> input = torch.randn(2)
  >>> output = m(input)
torch.nn.functional.leaky_relu
torch.nn.functional.leaky_relu(input, negative_slope=0.01, inplace=False) -> Tensor
Applies element-wise, \text{LeakyReLU}(x) = \max(0, x) +
   \text{negative_slope} * \min(0, x)
See "LeakyReLU" for more details.
Return type:
      Tensor
torch.Tensor.fill_diagonal_
Tensor.fill_diagonal_(fill_value, wrap=False) -> Tensor
Fill the main diagonal of a tensor that has at least 2-dimensions.
   When dims>2, all dimensions of input must be of equal length. This
   function modifies the input tensor in-place, and returns the input
   tensor.
Parameters:
      * fill_value (Scalar) -- the fill value
  * **wrap** (*bool*) -- the diagonal 'wrapped' after N columns
    for tall matrices.

Example:
  >>> a = torch.zeros(3, 3)
  >>> a.fill_diagonal_(5)
  tensor([[5., 0., 0.],
          [0., 5., 0.],
          [0., 0., 5.]])
  >>> b = torch.zeros(7, 3)
  >>> b.fill_diagonal_(5)
  tensor([[5., 0., 0.],
          [0., 5., 0.],
          [0., 0., 5.],
          [0., 0., 0.],
          [0., 0., 0.],
          [0., 0., 0.],
          [0., 0., 0.]])
  >>> c = torch.zeros(7, 3)
  >>> c.fill_diagonal_(5, wrap=True)
  tensor([[5., 0., 0.],
ConstantLR
class torch.optim.lr_scheduler.ConstantLR(optimizer, factor=0.3333333333333333, total_iters=5, last_epoch=- 1, verbose=False)
Decays the learning rate of each parameter group by a small
   constant factor until the number of epoch reaches a pre-defined
   milestone: total_iters. Notice that such decay can happen
   simultaneously with other changes to the learning rate from outside
   this scheduler. When last_epoch=-1, sets initial lr as lr.
Parameters:
      * optimizer (Optimizer) -- Wrapped optimizer.
  * **factor** (*float*) -- The number we multiply learning rate
    until the milestone. Default: 1./3.

  * **total_iters** (*int*) -- The number of steps that the
    scheduler decays the learning rate. Default: 5.

  * **last_epoch** (*int*) -- The index of the last epoch.
    Default: -1.

  * **verbose** (*bool*) -- If "True", prints a message to stdout
    for each update. Default: "False".

-[ Example ]-
-[ Example ]-



Assuming optimizer uses lr = 0.05 for all groups
lr = 0.025   if epoch == 0
lr = 0.025   if epoch == 1
lr = 0.025   if epoch == 2
lr = 0.025   if epoch == 3
lr = 0.05    if epoch >= 4
scheduler = ConstantLR(self.opt, factor=0.5, total_iters=4)
for epoch in range(100):
    train(...)
    validate(...)
    scheduler.step()



get_last_lr()
  Return last computed learning rate by current scheduler.

load_state_dict(state_dict)
  Loads the schedulers state.

  Parameters:
     **state_dict** (*dict*) -- scheduler state. Should be an
     object returned from a call to "state_dict()".

print_lr(is_verbose, group, lr, epoch=None)
  Display the current learning rate.

state_dict()
  Returns the state of the scheduler as a "dict".

  It contains an entry for every variable in self.__dict__ which
  is not the optimizer.
BNReLU2d
class torch.ao.nn.intrinsic.quantized.BNReLU2d(num_features, eps=1e-05, momentum=0.1, device=None, dtype=None)
A BNReLU2d module is a fused module of BatchNorm2d and ReLU
We adopt the same interface as "torch.ao.nn.quantized.BatchNorm2d".
Variables:
      torch.ao.nn.quantized.BatchNorm2d (Same as) --
ConvertCustomConfig
class torch.ao.quantization.fx.custom_config.ConvertCustomConfig
Custom configuration for "convert_fx()".
Example usage:
  convert_custom_config = ConvertCustomConfig()             .set_observed_to_quantized_mapping(ObservedCustomModule, QuantizedCustomModule)             .set_preserved_attributes(["attr1", "attr2"])

classmethod from_dict(convert_custom_config_dict)
  Create a "ConvertCustomConfig" from a dictionary with the
  following items:

     "observed_to_quantized_custom_module_class": a nested
     dictionary mapping from quantization mode to an inner mapping
     from observed module classes to quantized module classes,
     e.g.:: { "static": {FloatCustomModule: ObservedCustomModule},
     "dynamic": {FloatCustomModule: ObservedCustomModule},
     "weight_only": {FloatCustomModule: ObservedCustomModule} }
     "preserved_attributes": a list of attributes that persist
even if they are not used in "forward"
  This function is primarily for backward compatibility and may be
  removed in the future.

  Return type:
     *ConvertCustomConfig*

set_observed_to_quantized_mapping(observed_class, quantized_class, quant_type=QuantType.STATIC)
  Set the mapping from a custom observed module class to a custom
  quantized module class.

  The quantized module class must have a "from_observed" class
  method that converts the observed module class to the quantized
  module class.

  Return type:
     *ConvertCustomConfig*

set_preserved_attributes(attributes)
  Set the names of the attributes that will persist in the graph
  module even if they are not used in the model's "forward"
  method.

  Return type:
     *ConvertCustomConfig*

to_dict()
  Convert this "ConvertCustomConfig" to a dictionary with the
  items described in "from_dict()".

  Return type:
     *Dict*[str, *Any*]
torch.set_default_tensor_type
torch.set_default_tensor_type(t)
Sets the default "torch.Tensor" type to floating point tensor type
   "t". This type will also be used as default floating point type for
   type inference in "torch.tensor()".
The default floating point tensor type is initially
   "torch.FloatTensor".
Parameters:
      t (type or string) -- the floating point tensor type
      or its name
Example:
  >>> torch.tensor([1.2, 3]).dtype    # initial default for floating point is torch.float32
  torch.float32
  >>> torch.set_default_tensor_type(torch.DoubleTensor)
  >>> torch.tensor([1.2, 3]).dtype    # a new floating point tensor
  torch.float64
PairwiseDistance
class torch.nn.PairwiseDistance(p=2.0, eps=1e-06, keepdim=False)
Computes the pairwise distance between input vectors, or between
   columns of input matrices.
Distances are computed using "p"-norm, with constant "eps" added to
   avoid division by zero if "p" is negative, i.e.:
  \mathrm{dist}\left(x, y\right) = \left\Vert x-y + \epsilon e
  \right\Vert_p,

where e is the vector of ones and the "p"-norm is given by.
  \Vert x \Vert _p = \left( \sum_{i=1}^n  \vert x_i \vert ^ p
  \right) ^ {1/p}.

Parameters:
      * p (real, optional) -- the norm degree. Can be
        negative. Default: 2
  * **eps** (*float**, **optional*) -- Small value to avoid
    division by zero. Default: 1e-6

  * **keepdim** (*bool**, **optional*) -- Determines whether or
    not to keep the vector dimension. Default: False

Shape:
      * Input1: (N, D) or (D) where N = batch dimension and D =
        vector dimension
vector dimension*
  * Input2: (N, D) or (D), same shape as the Input1

  * Output: (N) or () based on input dimension. If "keepdim" is
    "True", then (N, 1) or (1) based on input dimension.

Examples::
      >>> pdist = nn.PairwiseDistance(p=2)
      >>> input1 = torch.randn(100, 128)
      >>> input2 = torch.randn(100, 128)
      >>> output = pdist(input1, input2)
torch.fft.ifftn
torch.fft.ifftn(input, s=None, dim=None, norm=None, *, out=None) -> Tensor
Computes the N dimensional inverse discrete Fourier transform of
   "input".
Note:
 Supports torch.half and torch.chalf on CUDA with GPU Architecture
 SM53 or greater. However it only supports powers of 2 signal
 length in every transformed dimensions.

Parameters:
      * input (Tensor) -- the input tensor
  * **s** (*Tuple**[**int**]**, **optional*) -- Signal size in the
    transformed dimensions. If given, each dimension "dim[i]" will
    either be zero-padded or trimmed to the length "s[i]" before
    computing the IFFT. If a length "-1" is specified, no padding
    is done in that dimension. Default: "s = [input.size(d) for d
    in dim]"

  * **dim** (*Tuple**[**int**]**, **optional*) -- Dimensions to be
    transformed. Default: all dimensions, or the last "len(s)"
    dimensions if "s" is given.
dimensions if "s" is given.
  * **norm** (*str**, **optional*) --

    Normalization mode. For the backward transform ("ifftn()"),
    these correspond to:

    * ""forward"" - no normalization

    * ""backward"" - normalize by "1/n"

    * ""ortho"" - normalize by "1/sqrt(n)" (making the IFFT
      orthonormal)

    Where "n = prod(s)" is the logical IFFT size. Calling the
    forward transform ("fftn()") with the same normalization mode
    will apply an overall normalization of "1/n" between the two
    transforms. This is required to make "ifftn()" the exact
    inverse.

    Default is ""backward"" (normalize by "1/n").

Keyword Arguments:
      out (Tensor, optional) -- the output tensor.
-[ Example ]-



x = torch.rand(10, 10, dtype=torch.complex64)
ifftn = torch.fft.ifftn(x)



The discrete Fourier transform is separable, so "ifftn()" here is
   equivalent to two one-dimensional "ifft()" calls:
torch.nn.functional.lp_pool1d
torch.nn.functional.lp_pool1d(input, norm_type, kernel_size, stride=None, ceil_mode=False)
Applies a 1D power-average pooling over an input signal composed of
   several input planes. If the sum of all inputs to the power of p
   is zero, the gradient is set to zero as well.
See "LPPool1d" for details.
Return type:
      Tensor
torch.frexp
torch.frexp(input, *, out=None) -> (Tensor mantissa, Tensor exponent)
Decomposes "input" into mantissa and exponent tensors such that
   \text{input} = \text{mantissa} \times 2^{\text{exponent}}.
The range of mantissa is the open interval (-1, 1).
Supports float inputs.
Parameters:
      input (Tensor) -- the input tensor
Keyword Arguments:
      out (tuple, optional) -- the output tensors
Example:
  >>> x = torch.arange(9.)
  >>> mantissa, exponent = torch.frexp(x)
  >>> mantissa
  tensor([0.0000, 0.5000, 0.5000, 0.7500, 0.5000, 0.6250, 0.7500, 0.8750, 0.5000])
  >>> exponent
  tensor([0, 1, 2, 2, 3, 3, 3, 3, 4], dtype=torch.int32)
  >>> torch.ldexp(mantissa, exponent)
  tensor([0., 1., 2., 3., 4., 5., 6., 7., 8.])
torch.vsplit
torch.vsplit(input, indices_or_sections) -> List of Tensors
Splits "input", a tensor with two or more dimensions, into multiple
   tensors vertically according to "indices_or_sections". Each split
   is a view of "input".
This is equivalent to calling torch.tensor_split(input,
   indices_or_sections, dim=0) (the split dimension is 0), except that
   if "indices_or_sections" is an integer it must evenly divide the
   split dimension or a runtime error will be thrown.
This function is based on NumPy's "numpy.vsplit()".
Parameters:
      * input (Tensor) -- tensor to split.
  * **indices_or_sections** (*int** or **list** or **tuple of
    ints*) -- See argument in "torch.tensor_split()".

Example::
      >>> t = torch.arange(16.0).reshape(4,4)
      >>> t
      tensor([[ 0.,  1.,  2.,  3.],
              [ 4.,  5.,  6.,  7.],
              [ 8.,  9., 10., 11.],
              [12., 13., 14., 15.]])
      >>> torch.vsplit(t, 2)
torch.vsplit(t, 2)
      (tensor([[0., 1., 2., 3.],
               [4., 5., 6., 7.]]),
       tensor([[ 8.,  9., 10., 11.],
               [12., 13., 14., 15.]]))
      >>> torch.vsplit(t, [3, 6])
      (tensor([[ 0.,  1.,  2.,  3.],
               [ 4.,  5.,  6.,  7.],
               [ 8.,  9., 10., 11.]]),
       tensor([[12., 13., 14., 15.]]),
       tensor([], size=(0, 4)))
no_grad
class torch.no_grad
Context-manager that disabled gradient calculation.
Disabling gradient calculation is useful for inference, when you
   are sure that you will not call "Tensor.backward()". It will reduce
   memory consumption for computations that would otherwise have
   requires_grad=True.
In this mode, the result of every computation will have
   requires_grad=False, even when the inputs have
   requires_grad=True.
This context manager is thread local; it will not affect
   computation in other threads.
Also functions as a decorator. (Make sure to instantiate with
   parenthesis.)
Note:
 No-grad is one of several mechanisms that can enable or disable
 gradients locally see Locally disabling gradient computation for
 more information on how they compare.

Note:
 This API does not apply to forward-mode AD. If you want to
 disable forward AD for a computation, you can unpack your dual
 tensors.

Example::
tensors.
Example::
      >>> x = torch.tensor([1.], requires_grad=True)
      >>> with torch.no_grad():
      ...     y = x * 2
      >>> y.requires_grad
      False
      >>> @torch.no_grad()
      ... def doubler(x):
      ...     return x * 2
      >>> z = doubler(x)
      >>> z.requires_grad
      False
ConvReLU1d
class torch.ao.nn.intrinsic.quantized.ConvReLU1d(in_channels, out_channels, kernel_size, stride=1, padding=0, dilation=1, groups=1, bias=True, padding_mode='zeros', device=None, dtype=None)
A ConvReLU1d module is a fused module of Conv1d and ReLU
We adopt the same interface as "torch.ao.nn.quantized.Conv1d".
Variables:
      torch.ao.nn.quantized.Conv1d (Same as) --
InstanceNorm1d
class torch.ao.nn.quantized.InstanceNorm1d(num_features, weight, bias, scale, zero_point, eps=1e-05, momentum=0.1, affine=False, track_running_stats=False, device=None, dtype=None)
This is the quantized version of "InstanceNorm1d".
Additional args:
      * scale - quantization scale of the output, type: double.
  * **zero_point** - quantization zero point of the output, type:
    long.
torch.cuda.max_memory_reserved
torch.cuda.max_memory_reserved(device=None)
Returns the maximum GPU memory managed by the caching allocator in
   bytes for a given device.
By default, this returns the peak cached memory since the beginning
   of this program. "reset_peak_memory_stats()" can be used to reset
   the starting point in tracking this metric. For example, these two
   functions can measure the peak cached memory amount of each
   iteration in a training loop.
Parameters:
      device (torch.device or int, optional) -- selected
      device. Returns statistic for the current device, given by
      "current_device()", if "device" is "None" (default).
Return type:
      int
Note:
 See Memory management for more details about GPU memory
 management.
torch.nn.functional.affine_grid
torch.nn.functional.affine_grid(theta, size, align_corners=None)
Generates a 2D or 3D flow field (sampling grid), given a batch of
   affine matrices "theta".
Note:
 This function is often used in conjunction with "grid_sample()"
 to build Spatial Transformer Networks .

Parameters:
      * theta (Tensor) -- input batch of affine matrices with
        shape (N \times 2 \times 3) for 2D or (N \times 3 \times 4)
        for 3D
  * **size** (*torch.Size*) -- the target output image size. (N
    \times C \times H \times W for 2D or N \times C \times D
    \times H \times W for 3D) Example: torch.Size((32, 3, 24, 24))

  * **align_corners** (*bool**, **optional*) -- if "True",
    consider "-1" and "1" to refer to the centers of the corner
    pixels rather than the image corners. Refer to "grid_sample()"
    for a more complete description. A grid generated by
"affine_grid()" should be passed to "grid_sample()" with the
        same setting for this option. Default: "False"
Returns:
      output Tensor of size (N \times H \times W \times 2)
Return type:
      output (Tensor)
Warning:
 When "align_corners = True", the grid positions depend on the
 pixel size relative to the input image size, and so the locations
 sampled by "grid_sample()" will differ for the same input given
 at different resolutions (that is, after being upsampled or
 downsampled). The default behavior up to version 1.2.0 was
 "align_corners = True". Since then, the default behavior has been
 changed to "align_corners = False", in order to bring it in line
 with the default for "interpolate()".

Warning:
 When "align_corners = True", 2D affine transforms on 1D data and
 3D affine transforms on 2D data (that is, when one of the spatial
 dimensions has unit size) are ill-defined, and not an intended
torch.cuda.get_rng_state
torch.cuda.get_rng_state(device='cuda')
Returns the random number generator state of the specified GPU as a
   ByteTensor.
Parameters:
      device (torch.device or int, optional) -- The
      device to return the RNG state of. Default: "'cuda'" (i.e.,
      "torch.device('cuda')", the current CUDA device).
Return type:
      Tensor
Warning:
 This function eagerly initializes CUDA.
torch.range
torch.range(start=0, end, step=1, *, out=None, dtype=None, layout=torch.strided, device=None, requires_grad=False) -> Tensor
Returns a 1-D tensor of size \left\lfloor \frac{\text{end} -
   \text{start}}{\text{step}} \right\rfloor + 1 with values from
   "start" to "end" with step "step". Step is the gap between two
   values in the tensor.
  \text{out}_{i+1} = \text{out}_i + \text{step}.

Warning:
 This function is deprecated and will be removed in a future
 release because its behavior is inconsistent with Python's range
 builtin. Instead, use "torch.arange()", which produces values in
 [start, end).

Parameters:
      * start (float) -- the starting value for the set of
        points. Default: "0".
  * **end** (*float*) -- the ending value for the set of points

  * **step** (*float*) -- the gap between each pair of adjacent
    points. Default: "1".

Keyword Arguments:
Keyword Arguments:
      * out (Tensor, optional) -- the output tensor.
  * **dtype** ("torch.dtype", optional) -- the desired data type
    of returned tensor. Default: if "None", uses a global default
    (see "torch.set_default_tensor_type()"). If *dtype* is not
    given, infer the data type from the other input arguments. If
    any of *start*, *end*, or *stop* are floating-point, the
    *dtype* is inferred to be the default dtype, see
    "get_default_dtype()". Otherwise, the *dtype* is inferred to
    be *torch.int64*.

  * **layout** ("torch.layout", optional) -- the desired layout of
    returned Tensor. Default: "torch.strided".

  * **device** ("torch.device", optional) -- the desired device of
    returned tensor. Default: if "None", uses the current device
    for the default tensor type (see
    "torch.set_default_tensor_type()"). "device" will be the CPU
    for CPU tensor types and the current CUDA device for CUDA
tensor types.
  * **requires_grad** (*bool**, **optional*) -- If autograd should
    record operations on the returned tensor. Default: "False".

Example:
  >>> torch.range(1, 4)
  tensor([ 1.,  2.,  3.,  4.])
  >>> torch.range(1, 4, 0.5)
  tensor([ 1.0000,  1.5000,  2.0000,  2.5000,  3.0000,  3.5000,  4.0000])
ConvBnReLU1d
class torch.ao.nn.intrinsic.qat.ConvBnReLU1d(in_channels, out_channels, kernel_size, stride=1, padding=0, dilation=1, groups=1, bias=None, padding_mode='zeros', eps=1e-05, momentum=0.1, freeze_bn=False, qconfig=None)
A ConvBnReLU1d module is a module fused from Conv1d, BatchNorm1d
   and ReLU, attached with FakeQuantize modules for weight, used in
   quantization aware training.
We combined the interface of "torch.nn.Conv1d" and
   "torch.nn.BatchNorm1d" and "torch.nn.ReLU".
Similar to torch.nn.Conv1d, with FakeQuantize modules initialized
   to default.
Variables:
      weight_fake_quant -- fake quant module for weight
torch.rand_like
torch.rand_like(input, *, dtype=None, layout=None, device=None, requires_grad=False, memory_format=torch.preserve_format) -> Tensor
Returns a tensor with the same size as "input" that is filled with
   random numbers from a uniform distribution on the interval [0, 1).
   "torch.rand_like(input)" is equivalent to "torch.rand(input.size(),
   dtype=input.dtype, layout=input.layout, device=input.device)".
Parameters:
      input (Tensor) -- the size of "input" will determine size
      of the output tensor.
Keyword Arguments:
      * dtype ("torch.dtype", optional) -- the desired data type
        of returned Tensor. Default: if "None", defaults to the dtype
        of "input".
  * **layout** ("torch.layout", optional) -- the desired layout of
    returned tensor. Default: if "None", defaults to the layout of
    "input".

  * **device** ("torch.device", optional) -- the desired device of
torch.Tensor.detach
Tensor.detach()
Returns a new Tensor, detached from the current graph.
The result will never require gradient.
This method also affects forward mode AD gradients and the result
   will never have forward mode AD gradients.
Note:
 Returned Tensor shares the same storage with the original one.
 In-place modifications on either of them will be seen, and may
 trigger errors in correctness checks. IMPORTANT NOTE: Previously,
 in-place size / stride / storage changes (such as *resize_* /
 *resize_as_* / *set_* / *transpose_*) to the returned tensor also
 update the original tensor. Now, these in-place changes will not
 update the original tensor anymore, and will instead trigger an
 error. For sparse tensors: In-place indices / values changes
 (such as *zero_* / *copy_* / *add_*) to the returned tensor will
 not update the original tensor anymore, and will instead trigger
 an error.