shrinath-suresh/blogs-docs-splitted huggingface dataset:
the element sizes of the dtypes.
  * The strides of all dimensions, except the last dimension, must
    be divisible by the ratio between the element sizes of the
    dtypes.

If any of the above conditions are not met, an error is thrown.
Warning:
 This overload is not supported by TorchScript, and using it in a
 Torchscript program will cause undefined behavior.

Parameters:
      dtype ("torch.dtype") -- the desired dtype
Example:
  >>> x = torch.randn(4, 4)
  >>> x
  tensor([[ 0.9482, -0.0310,  1.4999, -0.5316],
          [-0.1520,  0.7472,  0.5617, -0.8649],
          [-2.4724, -0.0334, -0.2976, -0.8499],
          [-0.2109,  1.9913, -0.9607, -0.6123]])
  >>> x.dtype
  torch.float32

  >>> y = x.view(torch.int32)
  >>> y
  tensor([[ 1064483442, -1124191867,  1069546515, -1089989247],
          [-1105482831,  1061112040,  1057999968, -1084397505],
[-1071760287, -1123489973, -1097310419, -1084649136],
              [-1101533110,  1073668768, -1082790149, -1088634448]],
          dtype=torch.int32)
      >>> y[0, 0] = 1000000000
      >>> x
      tensor([[ 0.0047, -0.0310,  1.4999, -0.5316],
              [-0.1520,  0.7472,  0.5617, -0.8649],
              [-2.4724, -0.0334, -0.2976, -0.8499],
              [-0.2109,  1.9913, -0.9607, -0.6123]])
  >>> x.view(torch.cfloat)
  tensor([[ 0.0047-0.0310j,  1.4999-0.5316j],
          [-0.1520+0.7472j,  0.5617-0.8649j],
          [-2.4724-0.0334j, -0.2976-0.8499j],
          [-0.2109+1.9913j, -0.9607-0.6123j]])
  >>> x.view(torch.cfloat).size()
  torch.Size([4, 2])

  >>> x.view(torch.uint8)
  tensor([[  0, 202, 154,  59, 182, 243, 253, 188, 185, 252, 191,  63, 240,  22,
             8, 191],
          [227, 165,  27, 190, 128,  72,  63,  63, 146, 203,  15,  63,  22, 106,
            93, 191],
93, 191],
              [205,  59,  30, 192, 112, 206,   8, 189,   7,  95, 152, 190,  12, 147,
                89, 191],
              [ 43, 246,  87, 190, 235, 226, 254,  63, 111, 240, 117, 191, 177, 191,
                28, 191]], dtype=torch.uint8)
      >>> x.view(torch.uint8).size()
      torch.Size([4, 16])
MultiStepLR
class torch.optim.lr_scheduler.MultiStepLR(optimizer, milestones, gamma=0.1, last_epoch=- 1, verbose=False)
Decays the learning rate of each parameter group by gamma once the
   number of epoch reaches one of the milestones. Notice that such
   decay can happen simultaneously with other changes to the learning
   rate from outside this scheduler. When last_epoch=-1, sets initial
   lr as lr.
Parameters:
      * optimizer (Optimizer) -- Wrapped optimizer.
  * **milestones** (*list*) -- List of epoch indices. Must be
    increasing.

  * **gamma** (*float*) -- Multiplicative factor of learning rate
    decay. Default: 0.1.

  * **last_epoch** (*int*) -- The index of last epoch. Default:
    -1.

  * **verbose** (*bool*) -- If "True", prints a message to stdout
    for each update. Default: "False".

-[ Example ]-



Assuming optimizer uses lr = 0.05 for all groups
lr = 0.05     if epoch < 30
lr = 0.05     if epoch < 30
lr = 0.005    if 30 <= epoch < 80
lr = 0.0005   if epoch >= 80
scheduler = MultiStepLR(optimizer, milestones=[30,80], gamma=0.1)
for epoch in range(100):
    train(...)
    validate(...)
    scheduler.step()



get_last_lr()
  Return last computed learning rate by current scheduler.

load_state_dict(state_dict)
  Loads the schedulers state.

  Parameters:
     **state_dict** (*dict*) -- scheduler state. Should be an
     object returned from a call to "state_dict()".

print_lr(is_verbose, group, lr, epoch=None)
  Display the current learning rate.

state_dict()
  Returns the state of the scheduler as a "dict".

  It contains an entry for every variable in self.__dict__ which
  is not the optimizer.
torch.autograd.function.FunctionCtx.save_for_backward
FunctionCtx.save_for_backward(*tensors)
Saves given tensors for a future call to "backward()".
"save_for_backward" should be called at most once, only from inside
   the "forward()" method, and only with tensors.
All tensors intended to be used in the backward pass should be
   saved with "save_for_backward" (as opposed to directly on "ctx") to
   prevent incorrect gradients and memory leaks, and enable the
   application of saved tensor hooks. See
   "torch.autograd.graph.saved_tensors_hooks".
Note that if intermediary tensors, tensors that are neither inputs
   nor outputs of "forward()", are saved for backward, your custom
   Function may not support double backward. Custom Functions that do
   not support double backward should decorate their "backward()"
   method with "@once_differentiable" so that performing double
backward raises an error. If you'd like to support double backward,
   you can either recompute intermediaries based on the inputs during
   backward or return the intermediaries as the outputs of the custom
   Function. See the double backward tutorial for more details.
In "backward()", saved tensors can be accessed through the
   "saved_tensors" attribute. Before returning them to the user, a
   check is made to ensure they weren't used in any in-place operation
   that modified their content.
Arguments can also be "None". This is a no-op.
See Extending torch.autograd for more details on how to use this
   method.
Example::
      >>> class Func(Function):
      >>>     @staticmethod
      >>>     def forward(ctx, x: torch.Tensor, y: torch.Tensor, z: int):
      >>>         w = x * z
      >>>         out = x * y + y * z + w * y
      >>>         ctx.save_for_backward(x, y, w, out)
      >>>         ctx.z = z  # z is not a tensor
      >>>         return out
      >>>
return out
  >>>
  >>>     @staticmethod
  >>>     @once_differentiable
  >>>     def backward(ctx, grad_out):
  >>>         x, y, w, out = ctx.saved_tensors
  >>>         z = ctx.z
  >>>         gx = grad_out * (y + y * z)
  >>>         gy = grad_out * (x + z + w)
  >>>         gz = None
  >>>         return gx, gy, gz
  >>>
  >>> a = torch.tensor(1., requires_grad=True, dtype=torch.double)
  >>> b = torch.tensor(2., requires_grad=True, dtype=torch.double)
  >>> c = 4
  >>> d = Func.apply(a, b, c)
torch.full
torch.full(size, fill_value, *, out=None, dtype=None, layout=torch.strided, device=None, requires_grad=False) -> Tensor
Creates a tensor of size "size" filled with "fill_value". The
   tensor's dtype is inferred from "fill_value".
Parameters:
      * size (int...) -- a list, tuple, or "torch.Size" of
        integers defining the shape of the output tensor.
  * **fill_value** (*Scalar*) -- the value to fill the output
    tensor with.

Keyword Arguments:
      * out (Tensor, optional) -- the output tensor.
  * **dtype** ("torch.dtype", optional) -- the desired data type
    of returned tensor. Default: if "None", uses a global default
    (see "torch.set_default_tensor_type()").

  * **layout** ("torch.layout", optional) -- the desired layout of
    returned Tensor. Default: "torch.strided".

  * **device** ("torch.device", optional) -- the desired device of
returned tensor. Default: if "None", uses the current device
        for the default tensor type (see
        "torch.set_default_tensor_type()"). "device" will be the CPU
        for CPU tensor types and the current CUDA device for CUDA
        tensor types.
  * **requires_grad** (*bool**, **optional*) -- If autograd should
    record operations on the returned tensor. Default: "False".

Example:
  >>> torch.full((2, 3), 3.141592)
  tensor([[ 3.1416,  3.1416,  3.1416],
          [ 3.1416,  3.1416,  3.1416]])
torch.nanquantile
torch.nanquantile(input, q, dim=None, keepdim=False, *, interpolation='linear', out=None) -> Tensor
This is a variant of "torch.quantile()" that "ignores" "NaN"
   values, computing the quantiles "q" as if "NaN" values in "input"
   did not exist. If all values in a reduced row are "NaN" then the
   quantiles for that reduction will be "NaN". See the documentation
   for "torch.quantile()".
Parameters:
      * input (Tensor) -- the input tensor.
  * **q** (*float** or **Tensor*) -- a scalar or 1D tensor of
    quantile values in the range [0, 1]

  * **dim** (*int*) -- the dimension to reduce.

  * **keepdim** (*bool*) -- whether the output tensor has "dim"
    retained or not.

Keyword Arguments:
      * interpolation (str) -- interpolation method to use when
        the desired quantile lies between two data points. Can be
        "linear", "lower", "higher", "midpoint" and "nearest". Default
        is "linear".
is "linear".
  * **out** (*Tensor**, **optional*) -- the output tensor.

Example:
  >>> t = torch.tensor([float('nan'), 1, 2])
  >>> t.quantile(0.5)
  tensor(nan)
  >>> t.nanquantile(0.5)
  tensor(1.5000)
  >>> t = torch.tensor([[float('nan'), float('nan')], [1, 2]])
  >>> t
  tensor([[nan, nan],
          [1., 2.]])
  >>> t.nanquantile(0.5, dim=0)
  tensor([1., 2.])
  >>> t.nanquantile(0.5, dim=1)
  tensor([   nan, 1.5000])
torch.aminmax
torch.aminmax(input, *, dim=None, keepdim=False, out=None) -> (Tensor min, Tensor max)
Computes the minimum and maximum values of the "input" tensor.
Parameters:
      input (Tensor) -- The input tensor
Keyword Arguments:
      * dim (Optional[int]) -- The dimension along which
        to compute the values. If None, computes the values over the
        entire "input" tensor. Default is None.
  * **keepdim** (*bool*) -- If *True*, the reduced dimensions will
    be kept in the output tensor as dimensions with size 1 for
    broadcasting, otherwise they will be removed, as if calling
    ("torch.squeeze()"). Default is *False*.

  * **out** (*Optional**[**Tuple**[**Tensor**, **Tensor**]**]*) --
    Optional tensors on which to write the result. Must have the
    same shape and dtype as the expected output. Default is
    *None*.

Returns:
      A named tuple (min, max) containing the minimum and maximum
values.
Raises:
      RuntimeError -- If any of the dimensions to compute the
      values over has size 0.
Note:
 NaN values are propagated to the output if at least one value is
 NaN.

See also:
 "torch.amin()" computes just the minimum value "torch.amax()"
 computes just the maximum value

Example:
  >>> torch.aminmax(torch.tensor([1, -3, 5]))
  torch.return_types.aminmax(
  min=tensor(-3),
  max=tensor(5))

  >>> # aminmax propagates NaNs
  >>> torch.aminmax(torch.tensor([1, -3, 5, torch.nan]))
  torch.return_types.aminmax(
  min=tensor(nan),
  max=tensor(nan))

  >>> t = torch.arange(10).view(2, 5)
  >>> t
  tensor([[0, 1, 2, 3, 4],
          [5, 6, 7, 8, 9]])
  >>> t.aminmax(dim=0, keepdim=True)
  torch.return_types.aminmax(
  min=tensor([[0, 1, 2, 3, 4]]),
  max=tensor([[5, 6, 7, 8, 9]]))
torch.autograd.functional.jacobian
torch.autograd.functional.jacobian(func, inputs, create_graph=False, strict=False, vectorize=False, strategy='reverse-mode')
Function that computes the Jacobian of a given function.
Parameters:
      * func (function) -- a Python function that takes Tensor
        inputs and returns a tuple of Tensors or a Tensor.
  * **inputs** (*tuple of Tensors** or **Tensor*) -- inputs to the
    function "func".

  * **create_graph** (*bool**, **optional*) -- If "True", the
    Jacobian will be computed in a differentiable manner. Note
    that when "strict" is "False", the result can not require
    gradients or be disconnected from the inputs.  Defaults to
    "False".

  * **strict** (*bool**, **optional*) -- If "True", an error will
    be raised when we detect that there exists an input such that
    all the outputs are independent of it. If "False", we return a
Tensor of zeros as the jacobian for said inputs, which is the
        expected mathematical value. Defaults to "False".
  * **vectorize** (*bool**, **optional*) -- This feature is
    experimental. Please consider using "torch.func.jacrev()" or
    "torch.func.jacfwd()" instead if you are looking for something
    less experimental and more performant. When computing the
    jacobian, usually we invoke "autograd.grad" once per row of
    the jacobian. If this flag is "True", we perform only a single
    "autograd.grad" call with "batched_grad=True" which uses the
    vmap prototype feature. Though this should lead to performance
    improvements in many cases, because this feature is still
    experimental, there may be performance cliffs. See
    "torch.autograd.grad()"'s "batched_grad" parameter for more
    information.

  * **strategy** (*str**, **optional*) -- Set to ""forward-mode""
or ""reverse-mode"" to determine whether the Jacobian will be
        computed with forward or reverse mode AD. Currently,
        ""forward-mode"" requires "vectorized=True". Defaults to
        ""reverse-mode"". If "func" has more outputs than inputs,
        ""forward-mode"" tends to be more performant. Otherwise,
        prefer to use ""reverse-mode"".
Returns:
      if there is a single input and output, this will be a single
      Tensor containing the Jacobian for the linearized inputs and
      output. If one of the two is a tuple, then the Jacobian will be
      a tuple of Tensors. If both of them are tuples, then the
      Jacobian will be a tuple of tuple of Tensors where
      "Jacobian[i][j]" will contain the Jacobian of the "i"th output
      and "j"th input and will have as size the concatenation of the
      sizes of the corresponding output and the corresponding input
      and will have same dtype and device as the corresponding input.
If strategy is "forward-mode", the dtype will be that of the
      output; otherwise, the input.
Return type:
      Jacobian (Tensor or nested tuple of Tensors)
-[ Example ]-



def exp_reducer(x):
   ...     return x.exp().sum(dim=1)
inputs = torch.rand(2, 2)
jacobian(exp_reducer, inputs)
   tensor([[[1.4917, 2.4352],
            [0.0000, 0.0000]],
           [[0.0000, 0.0000],
            [2.4369, 2.3799]]])
jacobian(exp_reducer, inputs, create_graph=True)
   tensor([[[1.4917, 2.4352],
            [0.0000, 0.0000]],
           [[0.0000, 0.0000],
            [2.4369, 2.3799]]], grad_fn=)
def exp_adder(x, y):
   ...     return 2 * x.exp() + 3 * y
inputs = (torch.rand(2), torch.rand(2))
jacobian(exp_adder, inputs)
   (tensor([[2.8052, 0.0000],
           [0.0000, 3.3963]]),
    tensor([[3., 0.],
            [0., 3.]]))
torch.nonzero
torch.nonzero(input, *, out=None, as_tuple=False) -> LongTensor or tuple of LongTensors
Note:
 "torch.nonzero(..., as_tuple=False)" (default) returns a 2-D
 tensor where each row is the index for a nonzero
 value."torch.nonzero(..., as_tuple=True)" returns a tuple of 1-D
 index tensors, allowing for advanced indexing, so
 "x[x.nonzero(as_tuple=True)]" gives all nonzero values of tensor
 "x". Of the returned tuple, each index tensor contains nonzero
 indices for a certain dimension.See below for more details on the
 two behaviors.When "input" is on CUDA, "torch.nonzero()" causes
 host-device synchronization.

When "as_tuple" is "False" (default):
Returns a tensor containing the indices of all non-zero elements of
   "input".  Each row in the result contains the indices of a non-zero
   element in "input". The result is sorted lexicographically, with
   the last index changing the fastest (C-style).
the last index changing the fastest (C-style).
If "input" has n dimensions, then the resulting indices tensor
   "out" is of size (z \times n), where z is the total number of non-
   zero elements in the "input" tensor.
When "as_tuple" is "True":
Returns a tuple of 1-D tensors, one for each dimension in "input",
   each containing the indices (in that dimension) of all non-zero
   elements of "input" .
If "input" has n dimensions, then the resulting tuple contains n
   tensors of size z, where z is the total number of non-zero elements
   in the "input" tensor.
As a special case, when "input" has zero dimensions and a nonzero
   scalar value, it is treated as a one-dimensional tensor with one
   element.
Parameters:
      input (Tensor) -- the input tensor.
Keyword Arguments:
      out (LongTensor, optional) -- the output tensor
      containing indices
Returns:
      If "as_tuple" is "False", the output tensor containing indices.
If "as_tuple" is "True", one 1-D tensor for each dimension,
      containing the indices of each nonzero element along that
      dimension.
Return type:
      LongTensor or tuple of LongTensor
Example:
  >>> torch.nonzero(torch.tensor([1, 1, 1, 0, 1]))
  tensor([[ 0],
          [ 1],
          [ 2],
          [ 4]])
  >>> torch.nonzero(torch.tensor([[0.6, 0.0, 0.0, 0.0],
  ...                             [0.0, 0.4, 0.0, 0.0],
  ...                             [0.0, 0.0, 1.2, 0.0],
  ...                             [0.0, 0.0, 0.0,-0.4]]))
  tensor([[ 0,  0],
          [ 1,  1],
          [ 2,  2],
          [ 3,  3]])
  >>> torch.nonzero(torch.tensor([1, 1, 1, 0, 1]), as_tuple=True)
  (tensor([0, 1, 2, 4]),)
  >>> torch.nonzero(torch.tensor([[0.6, 0.0, 0.0, 0.0],
  ...                             [0.0, 0.4, 0.0, 0.0],
  ...                             [0.0, 0.0, 1.2, 0.0],
torch.set_default_dtype
torch.set_default_dtype(d)
Sets the default floating point dtype to "d". Supports
   torch.float32 and torch.float64 as inputs. Other dtypes may be
   accepted without complaint but are not supported and are unlikely
   to work as expected.
When PyTorch is initialized its default floating point dtype is
   torch.float32, and the intent of set_default_dtype(torch.float64)
   is to facilitate NumPy-like type inference. The default floating
   point dtype is used to:


Implicitly determine the default complex dtype. When the default
      floating point type is float32 the default complex dtype is
      complex64, and when the default floating point type is float64
      the default complex type is complex128.


Infer the dtype for tensors constructed using Python floats or
      complex Python numbers. See examples below.


Determine the result of type promotion between bool and integer
tensors and Python floats and complex Python numbers.
Parameters:
      d ("torch.dtype") -- the floating point dtype to make the
      default. Either torch.float32 or torch.float64.
-[ Example ]-



initial default for floating point is torch.float32
Python floats are interpreted as float32
torch.tensor([1.2, 3]).dtype
   torch.float32
initial default for floating point is torch.complex64
Complex Python numbers are interpreted as complex64
torch.tensor([1.2, 3j]).dtype
   torch.complex64
torch.set_default_dtype(torch.float64)
Python floats are now interpreted as float64
torch.tensor([1.2, 3]).dtype    # a new floating point tensor
   torch.float64
Complex Python numbers are now interpreted as complex128
torch.tensor([1.2, 3j]).dtype   # a new complex tensor
   torch.complex128
RandomStructured
class torch.nn.utils.prune.RandomStructured(amount, dim=- 1)
Prune entire (currently unpruned) channels in a tensor at random.
Parameters:
      * amount (int or float) -- quantity of parameters to
        prune. If "float", should be between 0.0 and 1.0 and represent
        the fraction of parameters to prune. If "int", it represents
        the absolute number of parameters to prune.
  * **dim** (*int**, **optional*) -- index of the dim along which
    we define channels to prune. Default: -1.

classmethod apply(module, name, amount, dim=- 1)
  Adds the forward pre-hook that enables pruning on the fly and
  the reparametrization of a tensor in terms of the original
  tensor and the pruning mask.

  Parameters:
     * **module** (*nn.Module*) -- module containing the tensor to
       prune

     * **name** (*str*) -- parameter name within "module" on which
       pruning will act.
pruning will act.
     * **amount** (*int** or **float*) -- quantity of parameters
       to prune. If "float", should be between 0.0 and 1.0 and
       represent the fraction of parameters to prune. If "int", it
       represents the absolute number of parameters to prune.

     * **dim** (*int**, **optional*) -- index of the dim along
       which we define channels to prune. Default: -1.

apply_mask(module)
  Simply handles the multiplication between the parameter being
  pruned and the generated mask. Fetches the mask and the original
  tensor from the module and returns the pruned version of the
  tensor.

  Parameters:
     **module** (*nn.Module*) -- module containing the tensor to
     prune

  Returns:
     pruned version of the input tensor

  Return type:
     pruned_tensor (torch.Tensor)

compute_mask(t, default_mask)
  Computes and returns a mask for the input tensor "t". Starting
from a base "default_mask" (which should be a mask of ones if
      the tensor has not been pruned yet), generate a random mask to
      apply on top of the "default_mask" by randomly zeroing out
      channels along the specified dim of the tensor.
  Parameters:
     * **t** (*torch.Tensor*) -- tensor representing the parameter
       to prune

     * **default_mask** (*torch.Tensor*) -- Base mask from
       previous pruning iterations, that need to be respected
       after the new mask is applied. Same dims as "t".

  Returns:
     mask to apply to "t", of same dims as "t"

  Return type:
     mask (torch.Tensor)

  Raises:
     **IndexError** -- if "self.dim >= len(t.shape)"

prune(t, default_mask=None, importance_scores=None)
  Computes and returns a pruned version of input tensor "t"
  according to the pruning rule specified in "compute_mask()".

  Parameters:
     * **t** (*torch.Tensor*) -- tensor to prune (of same
dimensions as "default_mask").
     * **importance_scores** (*torch.Tensor*) -- tensor of
       importance scores (of same shape as "t") used to compute
       mask for pruning "t". The values in this tensor indicate
       the importance of the corresponding elements in the "t"
       that is being pruned. If unspecified or None, the tensor
       "t" will be used in its place.

     * **default_mask** (*torch.Tensor**, **optional*) -- mask
       from previous pruning iteration, if any. To be considered
       when determining what portion of the tensor that pruning
       should act on. If None, default to a mask of ones.

  Returns:
     pruned version of tensor "t".

remove(module)
  Removes the pruning reparameterization from a module. The pruned
  parameter named "name" remains permanently pruned, and the
  parameter named "name+'_orig'" is removed from the parameter
set_grad_enabled
class torch.set_grad_enabled(mode)
Context-manager that sets gradient calculation on or off.
"set_grad_enabled" will enable or disable grads based on its
   argument "mode". It can be used as a context-manager or as a
   function.
This context manager is thread local; it will not affect
   computation in other threads.
Parameters:
      mode (bool) -- Flag whether to enable grad ("True"), or
      disable ("False"). This can be used to conditionally enable
      gradients.
Note:
 set_grad_enabled is one of several mechanisms that can enable or
 disable gradients locally see Locally disabling gradient
 computation for more information on how they compare.

Note:
 This API does not apply to forward-mode AD.

Example::
      >>> x = torch.tensor([1.], requires_grad=True)
      >>> is_train = False
      >>> with torch.set_grad_enabled(is_train):
      ...     y = x * 2
      >>> y.requires_grad
      False
torch.nn.functional.fractional_max_pool2d
torch.nn.functional.fractional_max_pool2d(args, *kwargs)
Applies 2D fractional max pooling over an input signal composed of
   several input planes.
Fractional MaxPooling is described in detail in the paper
   Fractional MaxPooling by Ben Graham
The max-pooling operation is applied in kH \times kW regions by a
   stochastic step size determined by the target output size. The
   number of output features is equal to the number of input planes.
Parameters:
      * kernel_size -- the size of the window to take a max over.
        Can be a single number k (for a square kernel of k \times k)
        or a tuple (kH, kW)
  * **output_size** -- the target output size of the image of the
    form oH \times oW. Can be a tuple *(oH, oW)* or a single
    number oH for a square image oH \times oH

  * **output_ratio** -- If one wants to have an output size as a