shrinath-suresh/blogs-docs-splitted huggingface dataset:
Generator
class torch.Generator(device='cpu')
Creates and returns a generator object that manages the state of
   the algorithm which produces pseudo random numbers. Used as a
   keyword argument in many In-place random sampling functions.
Parameters:
      device ("torch.device", optional) -- the desired device for
      the generator.
Returns:
      An torch.Generator object.
Return type:
      Generator
Example:
  >>> g_cpu = torch.Generator()
  >>> g_cuda = torch.Generator(device='cuda')

device
  Generator.device -> device

  Gets the current device of the generator.

  Example:

     >>> g_cpu = torch.Generator()
     >>> g_cpu.device
     device(type='cpu')

get_state() -> Tensor
  Returns the Generator state as a "torch.ByteTensor".

  Returns:
     A "torch.ByteTensor" which contains all the necessary bits to
     restore a Generator to a specific point in time.

  Return type:
     Tensor
Return type:
         Tensor
  Example:

     >>> g_cpu = torch.Generator()
     >>> g_cpu.get_state()

initial_seed() -> int
  Returns the initial seed for generating random numbers.

  Example:

     >>> g_cpu = torch.Generator()
     >>> g_cpu.initial_seed()
     2147483647

manual_seed(seed) -> Generator
  Sets the seed for generating random numbers. Returns a
  *torch.Generator* object. It is recommended to set a large seed,
  i.e. a number that has a good balance of 0 and 1 bits. Avoid
  having many 0 bits in the seed.

  Parameters:
     **seed** (*int*) -- The desired seed. Value must be within
     the inclusive range *[-0x8000_0000_0000_0000,
     0xffff_ffff_ffff_ffff]*. Otherwise, a RuntimeError is raised.
     Negative inputs are remapped to positive values with the
     formula *0xffff_ffff_ffff_ffff + seed*.

  Returns:
     An torch.Generator object.

  Return type:
     Generator
Return type:
         Generator
  Example:

     >>> g_cpu = torch.Generator()
     >>> g_cpu.manual_seed(2147483647)

seed() -> int
  Gets a non-deterministic random number from std::random_device
  or the current time and uses it to seed a Generator.

  Example:

     >>> g_cpu = torch.Generator()
     >>> g_cpu.seed()
     1516516984916

set_state(new_state) -> void
  Sets the Generator state.

  Parameters:
     **new_state** (*torch.ByteTensor*) -- The desired state.

  Example:

     >>> g_cpu = torch.Generator()
     >>> g_cpu_other = torch.Generator()
     >>> g_cpu.set_state(g_cpu_other.get_state())
torch.cummax
torch.cummax(input, dim, *, out=None)
Returns a namedtuple "(values, indices)" where "values" is the
   cumulative maximum of elements of "input" in the dimension "dim".
   And "indices" is the index location of each maximum value found in
   the dimension "dim".
  y_i = max(x_1, x_2, x_3, \dots, x_i)

Parameters:
      * input (Tensor) -- the input tensor.
  * **dim** (*int*) -- the dimension to do the operation over

Keyword Arguments:
      out (tuple, optional) -- the result tuple of two
      output tensors (values, indices)
Example:
  >>> a = torch.randn(10)
  >>> a
  tensor([-0.3449, -1.5447,  0.0685, -1.5104, -1.1706,  0.2259,  1.4696, -1.3284,
       1.9946, -0.8209])
  >>> torch.cummax(a, dim=0)
  torch.return_types.cummax(
      values=tensor([-0.3449, -0.3449,  0.0685,  0.0685,  0.0685,  0.2259,  1.4696,  1.4696,
       1.9946,  1.9946]),
      indices=tensor([0, 0, 2, 2, 2, 5, 6, 6, 8, 8]))
torch.nn.functional.upsample_nearest
torch.nn.functional.upsample_nearest(input, size=None, scale_factor=None)
Upsamples the input, using nearest neighbours' pixel values.
Warning:
 This function is deprecated in favor of
 "torch.nn.functional.interpolate()". This is equivalent with
 "nn.functional.interpolate(..., mode='nearest')".

Currently spatial and volumetric upsampling are supported (i.e.
   expected inputs are 4 or 5 dimensional).
Parameters:
      * input (Tensor) -- input
  * **size** (*int** or **Tuple**[**int**, **int**] or
    **Tuple**[**int**, **int**, **int**]*) -- output spatia size.

  * **scale_factor** (*int*) -- multiplier for spatial size. Has
    to be an integer.

Note:
 This operation may produce nondeterministic gradients when given
 tensors on a CUDA device. See Reproducibility for more
 information.
torch.Tensor.index_put_
Tensor.index_put_(indices, values, accumulate=False) -> Tensor
Puts values from the tensor "values" into the tensor "self" using
   the indices specified in "indices" (which is a tuple of Tensors).
   The expression "tensor.index_put_(indices, values)" is equivalent
   to "tensor[indices] = values". Returns "self".
If "accumulate" is "True", the elements in "values" are added to
   "self". If accumulate is "False", the behavior is undefined if
   indices contain duplicate elements.
Parameters:
      * indices (tuple of LongTensor) -- tensors used to index
        into self.
  * **values** (*Tensor*) -- tensor of same dtype as *self*.

  * **accumulate** (*bool*) -- whether to accumulate into self
FloatFunctional
class torch.ao.nn.quantized.FloatFunctional
State collector class for float operations.
The instance of this class can be used instead of the "torch."
   prefix for some operations. See example usage below.
Note:
 This class does not provide a "forward" hook. Instead, you must
 use one of the underlying functions (e.g. "add").

Examples:
  >>> f_add = FloatFunctional()
  >>> a = torch.tensor(3.0)
  >>> b = torch.tensor(4.0)
  >>> f_add.add(a, b)  # Equivalent to ``torch.add(a, b)``

Valid operation names:
      * add
  * cat

  * mul

  * add_relu

  * add_scalar

  * mul_scalar
ELU
class torch.nn.ELU(alpha=1.0, inplace=False)
Applies the Exponential Linear Unit (ELU) function, element-wise,
   as described in the paper: Fast and Accurate Deep Network Learning
   by Exponential Linear Units (ELUs).
ELU is defined as:
  \text{ELU}(x) = \begin{cases} x, & \text{ if } x > 0\\ \alpha *
  (\exp(x) - 1), & \text{ if } x \leq 0 \end{cases}

Parameters:
      * alpha (float) -- the \alpha value for the ELU
        formulation. Default: 1.0
  * **inplace** (*bool*) -- can optionally do the operation in-
    place. Default: "False"

Shape:
      * Input: (*), where * means any number of dimensions.
  * Output: (*), same shape as the input.

[image]
Examples:
  >>> m = nn.ELU()
  >>> input = torch.randn(2)
  >>> output = m(input)
torch.baddbmm
torch.baddbmm(input, batch1, batch2, *, beta=1, alpha=1, out=None) -> Tensor
Performs a batch matrix-matrix product of matrices in "batch1" and
   "batch2". "input" is added to the final result.
"batch1" and "batch2" must be 3-D tensors each containing the same
   number of matrices.
If "batch1" is a (b \times n \times m) tensor, "batch2" is a (b
   \times m \times p) tensor, then "input" must be broadcastable with
   a (b \times n \times p) tensor and "out" will be a (b \times n
   \times p) tensor. Both "alpha" and "beta" mean the same as the
   scaling factors used in "torch.addbmm()".
  \text{out}_i = \beta\ \text{input}_i + \alpha\ (\text{batch1}_i
  \mathbin{@} \text{batch2}_i)

If "beta" is 0, then "input" will be ignored, and nan and inf
   in it will not be propagated.
For inputs of type FloatTensor or DoubleTensor, arguments
   "beta" and "alpha" must be real numbers, otherwise they should be
   integers.
integers.
This operator supports TensorFloat32.
On certain ROCm devices, when using float16 inputs this module will
   use different precision for backward.
Parameters:
      * input (Tensor) -- the tensor to be added
  * **batch1** (*Tensor*) -- the first batch of matrices to be
    multiplied

  * **batch2** (*Tensor*) -- the second batch of matrices to be
    multiplied

Keyword Arguments:
      * beta (Number, optional) -- multiplier for "input"
        (\beta)
  * **alpha** (*Number**, **optional*) -- multiplier for
    \text{batch1} \mathbin{@} \text{batch2} (\alpha)

  * **out** (*Tensor**, **optional*) -- the output tensor.

Example:
  >>> M = torch.randn(10, 3, 5)
  >>> batch1 = torch.randn(10, 3, 4)
  >>> batch2 = torch.randn(10, 4, 5)
  >>> torch.baddbmm(M, batch1, batch2).size()
  torch.Size([10, 3, 5])
HistogramObserver
class torch.quantization.observer.HistogramObserver(bins=2048, upsample_rate=128, dtype=torch.quint8, qscheme=torch.per_tensor_affine, reduce_range=False, quant_min=None, quant_max=None, factory_kwargs=None, eps=1.1920928955078125e-07)
The module records the running histogram of tensor values along
   with min/max values. "calculate_qparams" will calculate scale and
   zero_point.
Parameters:
      * bins (int) -- Number of bins to use for the histogram
  * **upsample_rate** (*int*) -- Factor by which the histograms
    are upsampled, this is used to interpolate histograms with
    varying ranges across observations

  * **dtype** (*dtype*) -- dtype argument to the *quantize* node
    needed to implement the reference model spec

  * **qscheme** -- Quantization scheme to be used

  * **reduce_range** -- Reduces the range of the quantized data
    type by 1 bit
type by 1 bit
  * **eps** (*Tensor*) -- Epsilon value for float32, Defaults to
    *torch.finfo(torch.float32).eps*.

The scale and zero point are computed as follows:


Create the histogram of the incoming inputs.
         The histogram is computed continuously, and the ranges per
         bin change with every new tensor observed.


Search the distribution in the histogram for optimal min/max
      values.
         The search for the min/max values ensures the minimization of
         the quantization error with respect to the floating point
         model.


Compute the scale and zero point the same way as in the
         "MinMaxObserver"
torch.promote_types
torch.promote_types(type1, type2) -> dtype
Returns the "torch.dtype" with the smallest size and scalar kind
   that is not smaller nor of lower kind than either type1 or
   type2. See type promotion documentation for more information on
   the type promotion logic.
Parameters:
      * type1 ("torch.dtype") --
  * **type2** ("torch.dtype") --

Example:
  >>> torch.promote_types(torch.int32, torch.float32)
  torch.float32
  >>> torch.promote_types(torch.uint8, torch.long)
  torch.long
torch.linalg.lstsq
torch.linalg.lstsq(A, B, rcond=None, *, driver=None)
Computes a solution to the least squares problem of a system of
   linear equations.
Letting \mathbb{K} be \mathbb{R} or \mathbb{C}, the least squares
   problem for a linear system AX = B with A \in \mathbb{K}^{m
   \times n}, B \in \mathbb{K}^{m \times k} is defined as
  \min_{X \in \mathbb{K}^{n \times k}} \|AX - B\|_F

where |-|_F denotes the Frobenius norm.
Supports inputs of float, double, cfloat and cdouble dtypes. Also
   supports batches of matrices, and if the inputs are batches of
   matrices then the output has the same batch dimensions.
"driver" chooses the backend function that will be used. For CPU
   inputs the valid values are 'gels', 'gelsy', 'gelsd,
   'gelss'. To choose the best driver on CPU consider:

If "A" is well-conditioned (its condition number is not too
     large), or you do not mind some precision loss.
For a general matrix: 'gelsy' (QR with pivoting) (default)

If "A" is full-rank: 'gels' (QR)



If "A" is not well-conditioned.


'gelsd' (tridiagonal reduction and SVD)


But if you run into memory issues: 'gelss' (full SVD).




For CUDA input, the only valid driver is 'gels', which assumes
   that "A" is full-rank.
See also the full description of these drivers
"rcond" is used to determine the effective rank of the matrices in
   "A" when "driver" is one of ('gelsy', 'gelsd', 'gelss'). In
   this case, if \sigma_i are the singular values of A in decreasing
   order, \sigma_i will be rounded down to zero if \sigma_i \leq
   \text{rcond} \cdot \sigma_1. If "rcond"= None (default), "rcond"
   is set to the machine precision of the dtype of "A" times max(m,
   n).
This function returns the solution to the problem and some extra
   information in a named tuple of four tensors *(solution, residuals,
rank, singular_values). For inputs "A", "B" of shape (, m, n),
   (, m, k)* respectively, it contains


solution: the least squares solution. It has shape (, n, k)*.


residuals: the squared residuals of the solutions, that is,
     |AX - B|_F^2. It has shape equal to the batch dimensions of
     "A". It is computed when m > n and every matrix in "A" is full-
     rank, otherwise, it is an empty tensor. If "A" is a batch of
     matrices and any matrix in the batch is not full rank, then an
     empty tensor is returned. This behavior may change in a future
     PyTorch release.


rank: tensor of ranks of the matrices in "A". It has shape
     equal to the batch dimensions of "A". It is computed when
     "driver" is one of ('gelsy', 'gelsd', 'gelss'), otherwise
     it is an empty tensor.


singular_values: tensor of singular values of the matrices in
     "A". It has shape (, min(m, n))*. It is computed when "driver"
is one of ('gelsd', 'gelss'), otherwise it is an empty
     tensor.
Note:
 This function computes *X = *"A"*.pinverse() @ *"B" in a faster
 and more numerically stable way than performing the computations
 separately.

Warning:
 The default value of "rcond" may change in a future PyTorch
 release. It is therefore recommended to use a fixed value to
 avoid potential breaking changes.

Parameters:
      * A (Tensor) -- lhs tensor of shape (, m, n)* where ***
        is zero or more batch dimensions.
  * **B** (*Tensor*) -- rhs tensor of shape *(*, m, k)* where ***
    is zero or more batch dimensions.

  * **rcond** (*float**, **optional*) -- used to determine the
    effective rank of "A". If "rcond"*= None*, "rcond" is set to
    the machine precision of the dtype of "A" times *max(m, n)*.
    Default: *None*.

Keyword Arguments:
      driver (str, optional) -- name of the LAPACK/MAGMA
method to be used. If None, 'gelsy' is used for CPU inputs
      and 'gels' for CUDA inputs. Default: None.
Returns:
      A named tuple (solution, residuals, rank, singular_values).
Examples:
  >>> A = torch.randn(1,3,3)
  >>> A
  tensor([[[-1.0838,  0.0225,  0.2275],
       [ 0.2438,  0.3844,  0.5499],
       [ 0.1175, -0.9102,  2.0870]]])
  >>> B = torch.randn(2,3,3)
  >>> B
  tensor([[[-0.6772,  0.7758,  0.5109],
       [-1.4382,  1.3769,  1.1818],
       [-0.3450,  0.0806,  0.3967]],
      [[-1.3994, -0.1521, -0.1473],
       [ 1.9194,  1.0458,  0.6705],
       [-1.1802, -0.9796,  1.4086]]])
  >>> X = torch.linalg.lstsq(A, B).solution # A is broadcasted to shape (2, 3, 3)
  >>> torch.dist(X, torch.linalg.pinv(A) @ B)
  tensor(1.5152e-06)

  >>> S = torch.linalg.lstsq(A, B, driver='gelsd').singular_values
  >>> torch.dist(S, torch.linalg.svdvals(A))
  tensor(2.3842e-07)
LinearLR
class torch.optim.lr_scheduler.LinearLR(optimizer, start_factor=0.3333333333333333, end_factor=1.0, total_iters=5, last_epoch=- 1, verbose=False)
Decays the learning rate of each parameter group by linearly
   changing small multiplicative factor until the number of epoch
   reaches a pre-defined milestone: total_iters. Notice that such
   decay can happen simultaneously with other changes to the learning
   rate from outside this scheduler. When last_epoch=-1, sets initial
   lr as lr.
Parameters:
      * optimizer (Optimizer) -- Wrapped optimizer.
  * **start_factor** (*float*) -- The number we multiply learning
    rate in the first epoch. The multiplication factor changes
    towards end_factor in the following epochs. Default: 1./3.

  * **end_factor** (*float*) -- The number we multiply learning
    rate at the end of linear changing process. Default: 1.0.

  * **total_iters** (*int*) -- The number of iterations that
multiplicative factor reaches to 1. Default: 5.
  * **last_epoch** (*int*) -- The index of the last epoch.
    Default: -1.

  * **verbose** (*bool*) -- If "True", prints a message to stdout
    for each update. Default: "False".

-[ Example ]-



Assuming optimizer uses lr = 0.05 for all groups
lr = 0.025    if epoch == 0
lr = 0.03125  if epoch == 1
lr = 0.0375   if epoch == 2
lr = 0.04375  if epoch == 3
lr = 0.05    if epoch >= 4
scheduler = LinearLR(self.opt, start_factor=0.5, total_iters=4)
for epoch in range(100):
    train(...)
    validate(...)
    scheduler.step()



get_last_lr()
  Return last computed learning rate by current scheduler.

load_state_dict(state_dict)
  Loads the schedulers state.

  Parameters:
     **state_dict** (*dict*) -- scheduler state. Should be an
     object returned from a call to "state_dict()".
LazyBatchNorm2d
class torch.nn.LazyBatchNorm2d(eps=1e-05, momentum=0.1, affine=True, track_running_stats=True, device=None, dtype=None)
A "torch.nn.BatchNorm2d" module with lazy initialization of the
   "num_features" argument of the "BatchNorm2d" that is inferred from
   the "input.size(1)". The attributes that will be lazily initialized
   are weight, bias, running_mean and running_var.
Check the "torch.nn.modules.lazy.LazyModuleMixin" for further
   documentation on lazy modules and their limitations.
Parameters:
      * eps (float) -- a value added to the denominator for
        numerical stability. Default: 1e-5
  * **momentum** (*float*) -- the value used for the running_mean
    and running_var computation. Can be set to "None" for
    cumulative moving average (i.e. simple average). Default: 0.1

  * **affine** (*bool*) -- a boolean value that when set to
    "True", this module has learnable affine parameters. Default:
    "True"
"True"
  * **track_running_stats** (*bool*) -- a boolean value that when
    set to "True", this module tracks the running mean and
    variance, and when set to "False", this module does not track
    such statistics, and initializes statistics buffers
    "running_mean" and "running_var" as "None". When these buffers
    are "None", this module always uses batch statistics. in both
    training and eval modes. Default: "True"

cls_to_become
  alias of "BatchNorm2d"
torch.copysign
torch.copysign(input, other, *, out=None) -> Tensor
Create a new floating-point tensor with the magnitude of "input"
   and the sign of "other", elementwise.
  \text{out}_{i} = \begin{cases}     -|\text{input}_{i}| &
  \text{if } \text{other}_{i} \leq -0.0 \\      |\text{input}_{i}|
  & \text{if } \text{other}_{i} \geq 0.0 \\ \end{cases}

Supports broadcasting to a common shape, and integer and float
   inputs.
Parameters:
      * input (Tensor) -- magnitudes.
  * **other** (*Tensor** or **Number*) -- contains value(s) whose
    signbit(s) are applied to the magnitudes in "input".

Keyword Arguments:
      out (Tensor, optional) -- the output tensor.
Example:
  >>> a = torch.randn(5)
  >>> a
  tensor([-1.2557, -0.0026, -0.5387,  0.4740, -0.9244])
  >>> torch.copysign(a, 1)
  tensor([1.2557, 0.0026, 0.5387, 0.4740, 0.9244])
  >>> a = torch.randn(4, 4)
  >>> a
a = torch.randn(4, 4)
      >>> a
      tensor([[ 0.7079,  0.2778, -1.0249,  0.5719],
              [-0.0059, -0.2600, -0.4475, -1.3948],
              [ 0.3667, -0.9567, -2.5757, -0.1751],
              [ 0.2046, -0.0742,  0.2998, -0.1054]])
      >>> b = torch.randn(4)
      tensor([ 0.2373,  0.3120,  0.3190, -1.1128])
      >>> torch.copysign(a, b)
      tensor([[ 0.7079,  0.2778,  1.0249, -0.5719],
              [ 0.0059,  0.2600,  0.4475, -1.3948],
              [ 0.3667,  0.9567,  2.5757, -0.1751],
              [ 0.2046,  0.0742,  0.2998, -0.1054]])
      >>> a = torch.tensor([1.])
      >>> b = torch.tensor([-0.])
      >>> torch.copysign(a, b)
      tensor([-1.])



Note:
 copysign handles signed zeros. If the other argument has a
 negative zero (-0), the corresponding output value will be
 negative.
torch.pca_lowrank
torch.pca_lowrank(A, q=None, center=True, niter=2)
Performs linear Principal Component Analysis (PCA) on a low-rank
   matrix, batches of such matrices, or sparse matrix.
This function returns a namedtuple "(U, S, V)" which is the nearly
   optimal approximation of a singular value decomposition of a
   centered matrix A such that A = U diag(S) V^T.
Note:
 The relation of "(U, S, V)" to PCA is as follows:

 * A is a data matrix with "m" samples and "n" features

 * the V columns represent the principal directions

 * S ** 2 / (m - 1) contains the eigenvalues of A^T A / (m - 1)
   which is the covariance of "A" when "center=True" is provided.

 * "matmul(A, V[:, :k])" projects data to the first k principal
   components

Note:
 Different from the standard SVD, the size of returned matrices
 depend on the specified rank and q values as follows:

    * U is m x q matrix

    * S is q-vector
S is q-vector* V is n x q matrix



Note:
 To obtain repeatable results, reset the seed for the pseudorandom
 number generator

Parameters:
      * A (Tensor) -- the input tensor of size (*, m, n)
  * **q** (*int**, **optional*) -- a slightly overestimated rank
    of A. By default, "q = min(6, m, n)".

  * **center** (*bool**, **optional*) -- if True, center the input
    tensor, otherwise, assume that the input is centered.

  * **niter** (*int**, **optional*) -- the number of subspace
    iterations to conduct; niter must be a nonnegative integer,
    and defaults to 2.

Return type:
      Tuple[Tensor, Tensor, Tensor]
References:
  - Nathan Halko, Per-Gunnar Martinsson, and Joel Tropp, Finding
    structure with randomness: probabilistic algorithms for
    constructing approximate matrix decompositions,
    arXiv:0909.4061 [math.NA; math.PR], 2009 (available at
torch.unique
torch.unique(input, sorted=True, return_inverse=False, return_counts=False, dim=None) -> Tuple[Tensor, Tensor, Tensor]
Returns the unique elements of the input tensor.
Note:
 This function is different from "torch.unique_consecutive()" in
 the sense that this function also eliminates non-consecutive
 duplicate values.

Note:
 Currently in the CUDA implementation and the CPU implementation
 when dim is specified, *torch.unique* always sort the tensor at
 the beginning regardless of the *sort* argument. Sorting could be
 slow, so if your input tensor is already sorted, it is
 recommended to use "torch.unique_consecutive()" which avoids the
 sorting.

Parameters:
      * input (Tensor) -- the input tensor
  * **sorted** (*bool*) -- Whether to sort the unique elements in
    ascending order before returning as output.

  * **return_inverse** (*bool*) -- Whether to also return the
indices for where elements in the original input ended up in
        the returned unique list.
  * **return_counts** (*bool*) -- Whether to also return the
    counts for each unique element.

  * **dim** (*int*) -- the dimension to apply unique. If "None",
    the unique of the flattened input is returned. default: "None"

Returns:
      A tensor or a tuple of tensors containing
     * **output** (*Tensor*): the output list of unique scalar
       elements.

     * **inverse_indices** (*Tensor*): (optional) if
       "return_inverse" is True, there will be an additional
       returned tensor (same shape as input) representing the
       indices for where elements in the original input map to in
       the output; otherwise, this function will only return a
       single tensor.

     * **counts** (*Tensor*): (optional) if "return_counts" is
       True, there will be an additional returned tensor (same
shape as output or output.size(dim), if dim was specified)
           representing the number of occurrences for each unique
           value or tensor.
Return type:
      (Tensor, Tensor (optional), Tensor (optional))
Example:
  >>> output = torch.unique(torch.tensor([1, 3, 2, 3], dtype=torch.long))
  >>> output
  tensor([1, 2, 3])

  >>> output, inverse_indices = torch.unique(
  ...     torch.tensor([1, 3, 2, 3], dtype=torch.long), sorted=True, return_inverse=True)
  >>> output
  tensor([1, 2, 3])
  >>> inverse_indices
  tensor([0, 2, 1, 2])

  >>> output, inverse_indices = torch.unique(
  ...     torch.tensor([[1, 3], [2, 3]], dtype=torch.long), sorted=True, return_inverse=True)
  >>> output
  tensor([1, 2, 3])
  >>> inverse_indices
  tensor([[0, 2],
          [1, 2]])