shrinath-suresh/blogs-docs-splitted huggingface dataset:
as training progresses.
  Parameters:
     **param_group** (*dict*) -- Specifies what Tensors should be
     optimized along with group specific optimization options.

load_state_dict(state_dict)
  Loads the optimizer state.

  Parameters:
     **state_dict** (*dict*) -- optimizer state. Should be an
     object returned from a call to "state_dict()".

register_step_post_hook(hook)
  Register an optimizer step post hook which will be called after
  optimizer step. It should have the following signature:

     hook(optimizer, args, kwargs) -> None

  The "optimizer" argument is the optimizer instance being used.

  Parameters:
     **hook** (*Callable*) -- The user defined hook to be
     registered.

  Returns:
     a handle that can be used to remove the added hook by calling
     "handle.remove()"

  Return type:
     "torch.utils.hooks.RemoveableHandle"

register_step_pre_hook(hook)
register_step_pre_hook(hook)
  Register an optimizer step pre hook which will be called before
  optimizer step. It should have the following signature:

     hook(optimizer, args, kwargs) -> None or modified args and kwargs

  The "optimizer" argument is the optimizer instance being used.
  If args and kwargs are modified by the pre-hook, then the
  transformed values are returned as a tuple containing the
  new_args and new_kwargs.

  Parameters:
     **hook** (*Callable*) -- The user defined hook to be
     registered.

  Returns:
     a handle that can be used to remove the added hook by calling
     "handle.remove()"

  Return type:
     "torch.utils.hooks.RemoveableHandle"

state_dict()
  Returns the state of the optimizer as a "dict".

  It contains two entries:

  * state - a dict holding current optimization state. Its content
       differs between optimizer classes.
torch.squeeze
torch.squeeze(input, dim=None) -> Tensor
Returns a tensor with all specified dimensions of "input" of size
   1 removed.
For example, if input is of shape: (A \times 1 \times B \times C
   \times 1 \times D) then the input.squeeze() will be of shape: (A
   \times B \times C \times D).
When "dim" is given, a squeeze operation is done only in the given
   dimension(s). If input is of shape: (A \times 1 \times B),
   "squeeze(input, 0)" leaves the tensor unchanged, but
   "squeeze(input, 1)" will squeeze the tensor to the shape (A \times
   B).
Note:
 The returned tensor shares the storage with the input tensor, so
 changing the contents of one will change the contents of the
 other.

Warning:
 If the tensor has a batch dimension of size 1, then
 *squeeze(input)* will also remove the batch dimension, which can
 lead to unexpected errors. Consider specifying only the dims you
 wish to be squeezed.

Parameters:
wish to be squeezed.
Parameters:
      * input (Tensor) -- the input tensor.
  * **dim** (*int** or **tuple of ints**, **optional*) --

    if given, the input will be squeezed
       only in the specified dimensions.

    Changed in version 2.0: "dim" now accepts tuples of
    dimensions.

Example:
  >>> x = torch.zeros(2, 1, 2, 1, 2)
  >>> x.size()
  torch.Size([2, 1, 2, 1, 2])
  >>> y = torch.squeeze(x)
  >>> y.size()
  torch.Size([2, 2, 2])
  >>> y = torch.squeeze(x, 0)
  >>> y.size()
  torch.Size([2, 1, 2, 1, 2])
  >>> y = torch.squeeze(x, 1)
  >>> y.size()
  torch.Size([2, 2, 1, 2])
  >>> y = torch.squeeze(x, (1, 2, 3))
  torch.Size([2, 2, 2])
torch.cuda.empty_cache
torch.cuda.empty_cache()
Releases all unoccupied cached memory currently held by the caching
   allocator so that those can be used in other GPU application and
   visible in nvidia-smi.
Note:
 "empty_cache()" doesn't increase the amount of GPU memory
 available for PyTorch. However, it may help reduce fragmentation
 of GPU memory in certain cases. See Memory management for more
 details about GPU memory management.
torch.nn.utils.rnn.pack_sequence
torch.nn.utils.rnn.pack_sequence(sequences, enforce_sorted=True)
Packs a list of variable length Tensors
Consecutive call of the next functions: "pad_sequence",
   "pack_padded_sequence".
"sequences" should be a list of Tensors of size "L x ", where L*
   is the length of a sequence and *** is any number of trailing
   dimensions, including zero.
For unsorted sequences, use enforce_sorted = False. If
   "enforce_sorted" is "True", the sequences should be sorted in the
   order of decreasing length. "enforce_sorted = True" is only
   necessary for ONNX export.
-[ Example ]-



from torch.nn.utils.rnn import pack_sequence
a = torch.tensor([1, 2, 3])
b = torch.tensor([4, 5])
c = torch.tensor([6])
pack_sequence([a, b, c])
   PackedSequence(data=tensor([1, 4, 6, 2, 5, 3]), batch_sizes=tensor([3, 2, 1]), sorted_indices=None, unsorted_indices=None)



Parameters:
Parameters:
      * sequences (list[Tensor]) -- A list of sequences of
        decreasing length.
  * **enforce_sorted** (*bool**, **optional*) -- if "True", checks
    that the input contains sequences sorted by length in a
    decreasing order. If "False", this condition is not checked.
    Default: "True".

Returns:
      a "PackedSequence" object
Return type:
      PackedSequence
torch.Tensor.resize_as_
Tensor.resize_as_(tensor, memory_format=torch.contiguous_format) -> Tensor
Resizes the "self" tensor to be the same size as the specified
   "tensor". This is equivalent to "self.resize_(tensor.size())".
Parameters:
      memory_format ("torch.memory_format", optional) -- the
      desired memory format of Tensor. Default:
      "torch.contiguous_format". Note that memory format of "self" is
      going to be unaffected if "self.size()" matches "tensor.size()".
torch.empty_strided
torch.empty_strided(size, stride, *, dtype=None, layout=None, device=None, requires_grad=False, pin_memory=False) -> Tensor
Creates a tensor with the specified "size" and "stride" and filled
   with undefined data.
Warning:
 If the constructed tensor is "overlapped" (with multiple indices
 referring to the same element in memory) its behavior is
 undefined.

Parameters:
      * size (tuple of python:int) -- the shape of the output
        tensor
  * **stride** (*tuple of python:int*) -- the strides of the
    output tensor

Keyword Arguments:
      * dtype ("torch.dtype", optional) -- the desired data type
        of returned tensor. Default: if "None", uses a global default
        (see "torch.set_default_tensor_type()").
  * **layout** ("torch.layout", optional) -- the desired layout of
    returned Tensor. Default: "torch.strided".

  * **device** ("torch.device", optional) -- the desired device of
returned tensor. Default: if "None", uses the current device
        for the default tensor type (see
        "torch.set_default_tensor_type()"). "device" will be the CPU
        for CPU tensor types and the current CUDA device for CUDA
        tensor types.
  * **requires_grad** (*bool**, **optional*) -- If autograd should
    record operations on the returned tensor. Default: "False".

  * **pin_memory** (*bool**, **optional*) -- If set, returned
    tensor would be allocated in the pinned memory. Works only for
    CPU tensors. Default: "False".

Example:
  >>> a = torch.empty_strided((2, 3), (1, 2))
  >>> a
  tensor([[8.9683e-44, 4.4842e-44, 5.1239e+07],
          [0.0000e+00, 0.0000e+00, 3.0705e-41]])
  >>> a.stride()
  (1, 2)
  >>> a.size()
  torch.Size([2, 3])
torch.nn.functional.ctc_loss
torch.nn.functional.ctc_loss(log_probs, targets, input_lengths, target_lengths, blank=0, reduction='mean', zero_infinity=False)
The Connectionist Temporal Classification loss.
See "CTCLoss" for details.
Note:
 In some circumstances when given tensors on a CUDA device and
 using CuDNN, this operator may select a nondeterministic
 algorithm to increase performance. If this is undesirable, you
 can try to make the operation deterministic (potentially at a
 performance cost) by setting "torch.backends.cudnn.deterministic
 = True". See Reproducibility for more information.

Note:
 This operation may produce nondeterministic gradients when given
 tensors on a CUDA device. See Reproducibility for more
 information.

Parameters:
      * log_probs (Tensor) -- (T, N, C) or (T, C) where C =
        number of characters in alphabet including blank, *T = input
length, and N = batch size*. The logarithmized probabilities
        of the outputs (e.g. obtained with
        "torch.nn.functional.log_softmax()").
  * **targets** (*Tensor*) -- (N, S) or *(sum(target_lengths))*.
    Targets cannot be blank. In the second form, the targets are
    assumed to be concatenated.

  * **input_lengths** (*Tensor*) -- (N) or (). Lengths of the
    inputs (must each be \leq T)

  * **target_lengths** (*Tensor*) -- (N) or (). Lengths of the
    targets

  * **blank** (*int**, **optional*) -- Blank label. Default 0.

  * **reduction** (*str**, **optional*) -- Specifies the reduction
    to apply to the output: "'none'" | "'mean'" | "'sum'".
    "'none'": no reduction will be applied, "'mean'": the output
    losses will be divided by the target lengths and then the mean
    over the batch is taken, "'sum'": the output will be summed.
    Default: "'mean'"
Default: "'mean'"
  * **zero_infinity** (*bool**, **optional*) -- Whether to zero
    infinite losses and the associated gradients. Default: "False"
    Infinite losses mainly occur when the inputs are too short to
    be aligned to the targets.

Return type:
      Tensor
Example:
  >>> log_probs = torch.randn(50, 16, 20).log_softmax(2).detach().requires_grad_()
  >>> targets = torch.randint(1, 20, (16, 30), dtype=torch.long)
  >>> input_lengths = torch.full((16,), 50, dtype=torch.long)
  >>> target_lengths = torch.randint(10, 30, (16,), dtype=torch.long)
  >>> loss = F.ctc_loss(log_probs, targets, input_lengths, target_lengths)
  >>> loss.backward()
torch.mm
torch.mm(input, mat2, *, out=None) -> Tensor
Performs a matrix multiplication of the matrices "input" and
   "mat2".
If "input" is a (n \times m) tensor, "mat2" is a (m \times p)
   tensor, "out" will be a (n \times p) tensor.
Note:
 This function does not broadcast. For broadcasting matrix
 products, see "torch.matmul()".

Supports strided and sparse 2-D tensors as inputs, autograd with
   respect to strided inputs.
This operation has support for arguments with sparse layouts. If
   "out" is provided it's layout will be used. Otherwise, the result
   layout will be deduced from that of "input".
Warning:
 Sparse support is a beta feature and some layout(s)/dtype/device
 combinations may not be supported, or may not have autograd
 support. If you notice missing functionality please open a
 feature request.

This operator supports TensorFloat32.
On certain ROCm devices, when using float16 inputs this module will
use different precision for backward.
Parameters:
      * input (Tensor) -- the first matrix to be matrix
        multiplied
  * **mat2** (*Tensor*) -- the second matrix to be matrix
    multiplied

Keyword Arguments:
      out (Tensor, optional) -- the output tensor.
Example:
  >>> mat1 = torch.randn(2, 3)
  >>> mat2 = torch.randn(3, 3)
  >>> torch.mm(mat1, mat2)
  tensor([[ 0.4851,  0.5037, -0.3633],
          [-0.0760, -3.6705,  2.4784]])
torch.le
torch.le(input, other, *, out=None) -> Tensor
Computes \text{input} \leq \text{other} element-wise.
The second argument can be a number or a tensor whose shape is
   broadcastable with the first argument.
Parameters:
      * input (Tensor) -- the tensor to compare
  * **other** (*Tensor** or **Scalar*) -- the tensor or value to
    compare

Keyword Arguments:
      out (Tensor, optional) -- the output tensor.
Returns:
      A boolean tensor that is True where "input" is less than or
      equal to "other" and False elsewhere
Example:
  >>> torch.le(torch.tensor([[1, 2], [3, 4]]), torch.tensor([[1, 1], [4, 4]]))
  tensor([[True, False], [True, True]])
torch.Tensor.imag
Tensor.imag
Returns a new tensor containing imaginary values of the "self"
   tensor. The returned tensor and "self" share the same underlying
   storage.
Warning:
 "imag()" is only supported for tensors with complex dtypes.

Example::
      >>> x=torch.randn(4, dtype=torch.cfloat)
      >>> x
      tensor([(0.3100+0.3553j), (-0.5445-0.7896j), (-1.6492-0.0633j), (-0.0638-0.8119j)])
      >>> x.imag
      tensor([ 0.3553, -0.7896, -0.0633, -0.8119])
torch.nn.utils.prune.identity
torch.nn.utils.prune.identity(module, name)
Applies pruning reparametrization to the tensor corresponding to
   the parameter called "name" in "module" without actually pruning
   any units. Modifies module in place (and also return the modified
   module) by:


adding a named buffer called "name+'_mask'" corresponding to the
      binary mask applied to the parameter "name" by the pruning
      method.


replacing the parameter "name" by its pruned version, while the
      original (unpruned) parameter is stored in a new parameter named
      "name+'_orig'".


Note:
 The mask is a tensor of ones.

Parameters:
      * module (nn.Module) -- module containing the tensor to
        prune.
  * **name** (*str*) -- parameter name within "module" on which
    pruning will act.

Returns:
      modified (i.e. pruned) version of the input module
Return type:
      module (nn.Module)
-[ Examples ]-
Mish
class torch.nn.Mish(inplace=False)
Applies the Mish function, element-wise. Mish: A Self Regularized
   Non-Monotonic Neural Activation Function.
  \text{Mish}(x) = x * \text{Tanh}(\text{Softplus}(x))

Note:
 See Mish: A Self Regularized Non-Monotonic Neural Activation
 Function

Shape:
      * Input: (*), where * means any number of dimensions.
  * Output: (*), same shape as the input.

[image]
Examples:
  >>> m = nn.Mish()
  >>> input = torch.randn(2)
  >>> output = m(input)
torch.cos
torch.cos(input, *, out=None) -> Tensor
Returns a new tensor with the cosine  of the elements of "input".
  \text{out}_{i} = \cos(\text{input}_{i})

Parameters:
      input (Tensor) -- the input tensor.
Keyword Arguments:
      out (Tensor, optional) -- the output tensor.
Example:
  >>> a = torch.randn(4)
  >>> a
  tensor([ 1.4309,  1.2706, -0.8562,  0.9796])
  >>> torch.cos(a)
  tensor([ 0.1395,  0.2957,  0.6553,  0.5574])
torch.cuda.jiterator._create_multi_output_jit_fn
torch.cuda.jiterator._create_multi_output_jit_fn(code_string, num_outputs, **kwargs)
Create a jiterator-generated cuda kernel for an elementwise op that
   supports returning one or more outputs.
Parameters:
      * code_string (str) -- CUDA code string to be compiled by
        jiterator. The entry functor must return value by reference.
  * **num_outputs** (*int*) -- number of outputs return by the
    kernel

  * **kwargs** (*Dict**, **optional*) -- Keyword arguments for
    generated function

Return type:
      Callable
Example:
  code_string = "template <typename T> void my_kernel(T x, T y, T alpha, T& out) { out = -x + alpha * y; }"
  jitted_fn = create_jit_fn(code_string, alpha=1.0)
  a = torch.rand(3, device='cuda')
  b = torch.rand(3, device='cuda')
  # invoke jitted function like a regular python function
LazyConv3d
class torch.nn.LazyConv3d(out_channels, kernel_size, stride=1, padding=0, dilation=1, groups=1, bias=True, padding_mode='zeros', device=None, dtype=None)
A "torch.nn.Conv3d" module with lazy initialization of the
   "in_channels" argument of the "Conv3d" that is inferred from the
   "input.size(1)". The attributes that will be lazily initialized are
   weight and bias.
Check the "torch.nn.modules.lazy.LazyModuleMixin" for further
   documentation on lazy modules and their limitations.
Parameters:
      * out_channels (int) -- Number of channels produced by the
        convolution
  * **kernel_size** (*int** or **tuple*) -- Size of the convolving
    kernel

  * **stride** (*int** or **tuple**, **optional*) -- Stride of the
    convolution. Default: 1

  * **padding** (*int** or **tuple**, **optional*) -- Zero-padding
    added to both sides of the input. Default: 0

  * **padding_mode** (*str**, **optional*) -- "'zeros'",
"'reflect'", "'replicate'" or "'circular'". Default: "'zeros'"
  * **dilation** (*int** or **tuple**, **optional*) -- Spacing
    between kernel elements. Default: 1

  * **groups** (*int**, **optional*) -- Number of blocked
    connections from input channels to output channels. Default: 1

  * **bias** (*bool**, **optional*) -- If "True", adds a learnable
    bias to the output. Default: "True"

See also:
 "torch.nn.Conv3d" and "torch.nn.modules.lazy.LazyModuleMixin"

cls_to_become
  alias of "Conv3d"
torch.nn.functional.conv_transpose2d
torch.nn.functional.conv_transpose2d(input, weight, bias=None, stride=1, padding=0, output_padding=0, groups=1, dilation=1) -> Tensor
Applies a 2D transposed convolution operator over an input image
   composed of several input planes, sometimes also called
   "deconvolution".
This operator supports TensorFloat32.
See "ConvTranspose2d" for details and output shape.
Note:
 In some circumstances when given tensors on a CUDA device and
 using CuDNN, this operator may select a nondeterministic
 algorithm to increase performance. If this is undesirable, you
 can try to make the operation deterministic (potentially at a
 performance cost) by setting "torch.backends.cudnn.deterministic
 = True". See Reproducibility for more information.

Parameters:
      * input -- input tensor of shape (\text{minibatch} ,
        \text{in_channels} , iH , iW)
\text{in_channels} , iH , iW)
  * **weight** -- filters of shape (\text{in\_channels} ,
    \frac{\text{out\_channels}}{\text{groups}} , kH , kW)

  * **bias** -- optional bias of shape (\text{out\_channels}).
    Default: None

  * **stride** -- the stride of the convolving kernel. Can be a
    single number or a tuple "(sH, sW)". Default: 1

  * **padding** -- "dilation * (kernel_size - 1) - padding" zero-
    padding will be added to both sides of each dimension in the
    input. Can be a single number or a tuple "(padH, padW)".
    Default: 0

  * **output_padding** -- additional size added to one side of
    each dimension in the output shape. Can be a single number or
    a tuple "(out_padH, out_padW)". Default: 0

  * **groups** -- split input into groups, \text{in\_channels}
    should be divisible by the number of groups. Default: 1

  * **dilation** -- the spacing between kernel elements. Can be a
torch.nn.utils.parametrize.remove_parametrizations
torch.nn.utils.parametrize.remove_parametrizations(module, tensor_name, leave_parametrized=True)
Removes the parametrizations on a tensor in a module.


If "leave_parametrized=True", "module[tensor_name]" will be set
     to its current output. In this case, the parametrization shall
     not change the "dtype" of the tensor.


If "leave_parametrized=False", "module[tensor_name]" will be set
     to the unparametrised tensor in
     "module.parametrizations[tensor_name].original". This is only
     possible when the parametrization depends on just one tensor.


Parameters:
      * module (nn.Module) -- module from which remove the
        parametrization
  * **tensor_name** (*str*) -- name of the parametrization to be
    removed

  * **leave_parametrized** (*bool**, **optional*) -- leave the
    attribute "tensor_name" parametrized. Default: "True"

Returns:
RandomUnstructured
class torch.nn.utils.prune.RandomUnstructured(amount)
Prune (currently unpruned) units in a tensor at random.
Parameters:
      * name (str) -- parameter name within "module" on which
        pruning will act.
  * **amount** (*int** or **float*) -- quantity of parameters to
    prune. If "float", should be between 0.0 and 1.0 and represent
    the fraction of parameters to prune. If "int", it represents
    the absolute number of parameters to prune.

classmethod apply(module, name, amount)
  Adds the forward pre-hook that enables pruning on the fly and
  the reparametrization of a tensor in terms of the original
  tensor and the pruning mask.

  Parameters:
     * **module** (*nn.Module*) -- module containing the tensor to
       prune

     * **name** (*str*) -- parameter name within "module" on which
       pruning will act.

     * **amount** (*int** or **float*) -- quantity of parameters
to prune. If "float", should be between 0.0 and 1.0 and
           represent the fraction of parameters to prune. If "int", it
           represents the absolute number of parameters to prune.
apply_mask(module)
  Simply handles the multiplication between the parameter being
  pruned and the generated mask. Fetches the mask and the original
  tensor from the module and returns the pruned version of the
  tensor.

  Parameters:
     **module** (*nn.Module*) -- module containing the tensor to
     prune

  Returns:
     pruned version of the input tensor

  Return type:
     pruned_tensor (torch.Tensor)

prune(t, default_mask=None, importance_scores=None)
  Computes and returns a pruned version of input tensor "t"
  according to the pruning rule specified in "compute_mask()".

  Parameters:
     * **t** (*torch.Tensor*) -- tensor to prune (of same
       dimensions as "default_mask").
RNNCell
class torch.nn.RNNCell(input_size, hidden_size, bias=True, nonlinearity='tanh', device=None, dtype=None)
An Elman RNN cell with tanh or ReLU non-linearity.
  h' = \tanh(W_{ih} x + b_{ih}  +  W_{hh} h + b_{hh})

If "nonlinearity" is 'relu', then ReLU is used in place of tanh.
Parameters:
      * input_size (int) -- The number of expected features in
        the input x
  * **hidden_size** (*int*) -- The number of features in the
    hidden state *h*

  * **bias** (*bool*) -- If "False", then the layer does not use
    bias weights *b_ih* and *b_hh*. Default: "True"

  * **nonlinearity** (*str*) -- The non-linearity to use. Can be
    either "'tanh'" or "'relu'". Default: "'tanh'"

Inputs: input, hidden
      * input: tensor containing input features
  * **hidden**: tensor containing the initial hidden state
    Defaults to zero if not provided.

Outputs: h'
Outputs: h'
      * h' of shape (batch, hidden_size): tensor containing the
        next hidden state for each element in the batch
Shape:
      * input: (N, H_{in}) or (H_{in}) tensor containing input
        features where H_{in} = input_size.
  * hidden: (N, H_{out}) or (H_{out}) tensor containing the
    initial hidden state where H_{out} = *hidden_size*. Defaults
    to zero if not provided.

  * output: (N, H_{out}) or (H_{out}) tensor containing the next
    hidden state.

Variables:
      * weight_ih (torch.Tensor) -- the learnable input-hidden
        weights, of shape (hidden_size, input_size)
  * **weight_hh** (*torch.Tensor*) -- the learnable hidden-hidden
    weights, of shape *(hidden_size, hidden_size)*

  * **bias_ih** -- the learnable input-hidden bias, of shape
    *(hidden_size)*

  * **bias_hh** -- the learnable hidden-hidden bias, of shape
    *(hidden_size)*

Note:
(hidden_size)
Note:
 All the weights and biases are initialized from
 \mathcal{U}(-\sqrt{k}, \sqrt{k}) where k =
 \frac{1}{\text{hidden\_size}}

Examples:
  >>> rnn = nn.RNNCell(10, 20)
  >>> input = torch.randn(6, 3, 10)
  >>> hx = torch.randn(3, 20)
  >>> output = []
  >>> for i in range(6):
  ...     hx = rnn(input[i], hx)
  ...     output.append(hx)
torch.diagonal_scatter
torch.diagonal_scatter(input, src, offset=0, dim1=0, dim2=1) -> Tensor
Embeds the values of the "src" tensor into "input" along the
   diagonal elements of "input", with respect to "dim1" and "dim2".
This function returns a tensor with fresh storage; it does not
   return a view.
The argument "offset" controls which diagonal to consider:


If "offset" = 0, it is the main diagonal.


If "offset" > 0, it is above the main diagonal.


If "offset" < 0, it is below the main diagonal.


Parameters:
      * input (Tensor) -- the input tensor. Must be at least
        2-dimensional.
  * **src** (*Tensor*) -- the tensor to embed into "input".

  * **offset** (*int**, **optional*) -- which diagonal to
    consider. Default: 0 (main diagonal).

  * **dim1** (*int**, **optional*) -- first dimension with respect
    to which to take diagonal. Default: 0.

  * **dim2** (*int**, **optional*) -- second dimension with