shrinath-suresh/blogs-docs-splitted huggingface dataset:
torch.sparse.softmax
torch.sparse.softmax(input, dim, *, dtype=None) -> Tensor
Applies a softmax function.
Softmax is defined as:
\text{Softmax}(x_{i}) = \frac{exp(x_i)}{\sum_j exp(x_j)}
where i, j run over sparse tensor indices and unspecified entries
   are ignores. This is equivalent to defining unspecified entries as
   negative infinity so that exp(x_k) = 0 when the entry with index k
   has not specified.
It is applied to all slices along dim, and will re-scale them so
   that the elements lie in the range [0, 1] and sum to 1.
Parameters:
      * input (Tensor) -- input
  * **dim** (*int*) -- A dimension along which softmax will be
    computed.

  * **dtype** ("torch.dtype", optional) -- the desired data type
    of returned tensor.  If specified, the input tensor is casted
    to "dtype" before the operation is performed. This is useful
    for preventing data type overflows. Default: None
L1Loss
class torch.nn.L1Loss(size_average=None, reduce=None, reduction='mean')
Creates a criterion that measures the mean absolute error (MAE)
   between each element in the input x and target y.
The unreduced (i.e. with "reduction" set to "'none'") loss can be
   described as:
  \ell(x, y) = L = \{l_1,\dots,l_N\}^\top, \quad l_n = \left| x_n
  - y_n \right|,

where N is the batch size. If "reduction" is not "'none'" (default
   "'mean'"), then:
  \ell(x, y) = \begin{cases}     \operatorname{mean}(L), &
  \text{if reduction} = \text{`mean';}\\
  \operatorname{sum}(L),  & \text{if reduction} = \text{`sum'.}
  \end{cases}

x and y are tensors of arbitrary shapes with a total of n elements
   each.
The sum operation still operates over all the elements, and divides
   by n.
The division by n can be avoided if one sets "reduction = 'sum'".
Supports real-valued and complex-valued inputs.
Parameters:
Parameters:
      * size_average (bool, optional) -- Deprecated (see
        "reduction"). By default, the losses are averaged over each
        loss element in the batch. Note that for some losses, there
        are multiple elements per sample. If the field "size_average"
        is set to "False", the losses are instead summed for each
        minibatch. Ignored when "reduce" is "False". Default: "True"
  * **reduce** (*bool**, **optional*) -- Deprecated (see
    "reduction"). By default, the losses are averaged or summed
    over observations for each minibatch depending on
    "size_average". When "reduce" is "False", returns a loss per
    batch element instead and ignores "size_average". Default:
    "True"

  * **reduction** (*str**, **optional*) -- Specifies the reduction
    to apply to the output: "'none'" | "'mean'" | "'sum'".
    "'none'": no reduction will be applied, "'mean'": the sum of
the output will be divided by the number of elements in the
        output, "'sum'": the output will be summed. Note:
        "size_average" and "reduce" are in the process of being
        deprecated, and in the meantime, specifying either of those
        two args will override "reduction". Default: "'mean'"
Shape:
      * Input: (*), where * means any number of dimensions.
  * Target: (*), same shape as the input.

  * Output: scalar. If "reduction" is "'none'", then (*), same
    shape as the input.

Examples:
  >>> loss = nn.L1Loss()
  >>> input = torch.randn(3, 5, requires_grad=True)
  >>> target = torch.randn(3, 5)
  >>> output = loss(input, target)
  >>> output.backward()
torch.cuda.mem_get_info
torch.cuda.mem_get_info(device=None)
Returns the global free and total GPU memory occupied for a given
   device using cudaMemGetInfo.
Parameters:
      device (torch.device or int, optional) -- selected
      device. Returns statistic for the current device, given by
      "current_device()", if "device" is "None" (default).
Return type:
      Tuple[int, int]
Note:
 See Memory management for more details about GPU memory
 management.
torch.isneginf
torch.isneginf(input, *, out=None) -> Tensor
Tests if each element of "input" is negative infinity or not.
Parameters:
      input (Tensor) -- the input tensor.
Keyword Arguments:
      out (Tensor, optional) -- the output tensor.
Example:
  >>> a = torch.tensor([-float('inf'), float('inf'), 1.2])
  >>> torch.isneginf(a)
  tensor([ True, False, False])
MaxPool1d
class torch.nn.MaxPool1d(kernel_size, stride=None, padding=0, dilation=1, return_indices=False, ceil_mode=False)
Applies a 1D max pooling over an input signal composed of several
   input planes.
In the simplest case, the output value of the layer with input size
   (N, C, L) and output (N, C, L_{out}) can be precisely described as:
  out(N_i, C_j, k) = \max_{m=0, \ldots, \text{kernel\_size} - 1}
  input(N_i, C_j, stride \times k + m)

If "padding" is non-zero, then the input is implicitly padded with
   negative infinity on both sides for "padding" number of points.
   "dilation" is the stride between the elements within the sliding
   window. This link has a nice visualization of the pooling
   parameters.
Note:
 When ceil_mode=True, sliding windows are allowed to go off-bounds
 if they start within the left padding or the input. Sliding
 windows that would start in the right padded region are ignored.

Parameters:
Parameters:
      * kernel_size (Union[int, Tuple[int]]) --
        The size of the sliding window, must be > 0.
  * **stride** (*Union**[**int**, **Tuple**[**int**]**]*) -- The
    stride of the sliding window, must be > 0. Default value is
    "kernel_size".

  * **padding** (*Union**[**int**, **Tuple**[**int**]**]*) --
    Implicit negative infinity padding to be added on both sides,
    must be >= 0 and <= kernel_size / 2.

  * **dilation** (*Union**[**int**, **Tuple**[**int**]**]*) -- The
    stride between elements within a sliding window, must be > 0.

  * **return_indices** (*bool*) -- If "True", will return the
    argmax along with the max values. Useful for
    "torch.nn.MaxUnpool1d" later

  * **ceil_mode** (*bool*) -- If "True", will use *ceil* instead
    of *floor* to compute the output shape. This ensures that
    every element in the input tensor is covered by a sliding
    window.

Shape:
window.
Shape:
      * Input: (N, C, L_{in}) or (C, L_{in}).
  * Output: (N, C, L_{out}) or (C, L_{out}), where

       L_{out} = \left\lfloor \frac{L_{in} + 2 \times
       \text{padding} - \text{dilation}       \times
       (\text{kernel\_size} - 1) - 1}{\text{stride}} +
       1\right\rfloor

Examples:
  >>> # pool of size=3, stride=2
  >>> m = nn.MaxPool1d(3, stride=2)
  >>> input = torch.randn(20, 16, 50)
  >>> output = m(input)
torch.atan2
torch.atan2(input, other, *, out=None) -> Tensor
Element-wise arctangent of \text{input}{i} / \text{other} with
   consideration of the quadrant. Returns a new tensor with the signed
   angles in radians between vector (\text{other}{i},
   \text{input}) and vector (1, 0). (Note that \text{other}{i},
   the second parameter, is the x-coordinate, while \text{input},
   the first parameter, is the y-coordinate.)
The shapes of "input" and "other" must be broadcastable.
Parameters:
      * input (Tensor) -- the first input tensor
  * **other** (*Tensor*) -- the second input tensor

Keyword Arguments:
      out (Tensor, optional) -- the output tensor.
Example:
  >>> a = torch.randn(4)
  >>> a
  tensor([ 0.9041,  0.0196, -0.3108, -2.4423])
  >>> torch.atan2(a, torch.randn(4))
  tensor([ 0.9833,  0.0811, -1.9743, -1.4151])
torch.autograd.Function.vmap
static Function.vmap(info, in_dims, *args)
Defines a rule for the behavior of this autograd.Function
   underneath "torch.vmap()". For a "torch.autograd.Function()" to
   support "torch.vmap()", you must either override this staticmethod,
   or set "generate_vmap_rule" to "True" (you may not do both).
If you choose to override this staticmethod: it must accept


an "info" object as the first argument. "info.batch_size"
     specifies the size of the dimension being vmapped over, while
     "info.randomness" is the randomness option passed to
     "torch.vmap()".


an "in_dims" tuple as the second argument. For each arg in
     "args", "in_dims" has a corresponding "Optional[int]". It is
     "None" if the arg is not a Tensor or if the arg is not being
     vmapped over, otherwise, it is an integer specifying what
     dimension of the Tensor is being vmapped over.


"*args", which is the same as the args to "forward()".
TripletMarginLoss
class torch.nn.TripletMarginLoss(margin=1.0, p=2.0, eps=1e-06, swap=False, size_average=None, reduce=None, reduction='mean')
Creates a criterion that measures the triplet loss given an input
   tensors x1, x2, x3 and a margin with a value greater than 0. This
   is used for measuring a relative similarity between samples. A
   triplet is composed by a, p and n (i.e., anchor, positive
   examples and negative examples respectively). The shapes of all
   input tensors should be (N, D).
The distance swap is described in detail in the paper Learning
   shallow convolutional feature descriptors with triplet losses by V.
   Balntas, E. Riba et al.
The loss function for each sample in the mini-batch is:
  L(a, p, n) = \max \{d(a_i, p_i) - d(a_i, n_i) + {\rm margin},
  0\}

where
  d(x_i, y_i) = \left\lVert {\bf x}_i - {\bf y}_i \right\rVert_p

See also "TripletMarginWithDistanceLoss", which computes the
triplet margin loss for input tensors using a custom distance
   function.
Parameters:
      * margin (float, optional) -- Default: 1.
  * **p** (*int**, **optional*) -- The norm degree for pairwise
    distance. Default: 2.

  * **swap** (*bool**, **optional*) -- The distance swap is
    described in detail in the paper *Learning shallow
    convolutional feature descriptors with triplet losses* by V.
    Balntas, E. Riba et al. Default: "False".

  * **size_average** (*bool**, **optional*) -- Deprecated (see
    "reduction"). By default, the losses are averaged over each
    loss element in the batch. Note that for some losses, there
    are multiple elements per sample. If the field "size_average"
    is set to "False", the losses are instead summed for each
    minibatch. Ignored when "reduce" is "False". Default: "True"

  * **reduce** (*bool**, **optional*) -- Deprecated (see
"reduction"). By default, the losses are averaged or summed
        over observations for each minibatch depending on
        "size_average". When "reduce" is "False", returns a loss per
        batch element instead and ignores "size_average". Default:
        "True"
  * **reduction** (*str**, **optional*) -- Specifies the reduction
    to apply to the output: "'none'" | "'mean'" | "'sum'".
    "'none'": no reduction will be applied, "'mean'": the sum of
    the output will be divided by the number of elements in the
    output, "'sum'": the output will be summed. Note:
    "size_average" and "reduce" are in the process of being
    deprecated, and in the meantime, specifying either of those
    two args will override "reduction". Default: "'mean'"

Shape:
      * Input: (N, D) or (D) where D is the vector dimension.
  * Output: A Tensor of shape (N) if "reduction" is "'none'" and
    input shape is (N, D); a scalar otherwise.

Examples:
Examples:
  >>> triplet_loss = nn.TripletMarginLoss(margin=1.0, p=2)
  >>> anchor = torch.randn(100, 128, requires_grad=True)
  >>> positive = torch.randn(100, 128, requires_grad=True)
  >>> negative = torch.randn(100, 128, requires_grad=True)
  >>> output = triplet_loss(anchor, positive, negative)
  >>> output.backward()
torch.broadcast_shapes
torch.broadcast_shapes(*shapes) -> Size
Similar to "broadcast_tensors()" but for shapes.
This is equivalent to "torch.broadcast_tensors(*map(torch.empty,
   shapes))[0].shape" but avoids the need create to intermediate
   tensors. This is useful for broadcasting tensors of common batch
   shape but different rightmost shape, e.g. to broadcast mean vectors
   with covariance matrices.
Example:
  >>> torch.broadcast_shapes((2,), (3, 1), (1, 1, 1))
  torch.Size([1, 3, 2])

Parameters:
      shapes (torch.Size*) -- Shapes of tensors.
Returns:
      A shape compatible with all input shapes.
Return type:
      shape (torch.Size)
Raises:
      RuntimeError -- If shapes are incompatible.
torch.nn.functional.conv_transpose1d
torch.nn.functional.conv_transpose1d(input, weight, bias=None, stride=1, padding=0, output_padding=0, groups=1, dilation=1) -> Tensor
Applies a 1D transposed convolution operator over an input signal
   composed of several input planes, sometimes also called
   "deconvolution".
This operator supports TensorFloat32.
See "ConvTranspose1d" for details and output shape.
Note:
 In some circumstances when given tensors on a CUDA device and
 using CuDNN, this operator may select a nondeterministic
 algorithm to increase performance. If this is undesirable, you
 can try to make the operation deterministic (potentially at a
 performance cost) by setting "torch.backends.cudnn.deterministic
 = True". See Reproducibility for more information.

Parameters:
      * input -- input tensor of shape (\text{minibatch} ,
        \text{in_channels} , iW)
\text{in_channels} , iW)
  * **weight** -- filters of shape (\text{in\_channels} ,
    \frac{\text{out\_channels}}{\text{groups}} , kW)

  * **bias** -- optional bias of shape (\text{out\_channels}).
    Default: None

  * **stride** -- the stride of the convolving kernel. Can be a
    single number or a tuple "(sW,)". Default: 1

  * **padding** -- "dilation * (kernel_size - 1) - padding" zero-
    padding will be added to both sides of each dimension in the
    input. Can be a single number or a tuple "(padW,)". Default: 0

  * **output_padding** -- additional size added to one side of
    each dimension in the output shape. Can be a single number or
    a tuple "(out_padW)". Default: 0

  * **groups** -- split input into groups, \text{in\_channels}
    should be divisible by the number of groups. Default: 1

  * **dilation** -- the spacing between kernel elements. Can be a
    single number or a tuple "(dW,)". Default: 1
Hardshrink
class torch.nn.Hardshrink(lambd=0.5)
Applies the Hard Shrinkage (Hardshrink) function element-wise.
Hardshrink is defined as:
  \text{HardShrink}(x) = \begin{cases} x, & \text{ if } x >
  \lambda \\ x, & \text{ if } x < -\lambda \\ 0, & \text{
  otherwise } \end{cases}

Parameters:
      lambd (float) -- the \lambda value for the Hardshrink
      formulation. Default: 0.5
Shape:
      * Input: (*), where * means any number of dimensions.
  * Output: (*), same shape as the input.

[image]
Examples:
  >>> m = nn.Hardshrink()
  >>> input = torch.randn(2)
  >>> output = m(input)
torch.dot
torch.dot(input, other, *, out=None) -> Tensor
Computes the dot product of two 1D tensors.
Note:
 Unlike NumPy's dot, torch.dot intentionally only supports
 computing the dot product of two 1D tensors with the same number
 of elements.

Parameters:
      * input (Tensor) -- first tensor in the dot product, must
        be 1D.
  * **other** (*Tensor*) -- second tensor in the dot product, must
    be 1D.

Keyword Arguments:
      out (Tensor, optional) -- the output tensor.
Example:
  >>> torch.dot(torch.tensor([2, 3]), torch.tensor([2, 1]))
  tensor(7)
AdaptiveMaxPool1d
class torch.nn.AdaptiveMaxPool1d(output_size, return_indices=False)
Applies a 1D adaptive max pooling over an input signal composed of
   several input planes.
The output size is L_{out}, for any input size. The number of
   output features is equal to the number of input planes.
Parameters:
      * output_size (Union[int, Tuple[int]]) --
        the target output size L_{out}.
  * **return_indices** (*bool*) -- if "True", will return the
    indices along with the outputs. Useful to pass to
    nn.MaxUnpool1d. Default: "False"

Shape:
      * Input: (N, C, L_{in}) or (C, L_{in}).
  * Output: (N, C, L_{out}) or (C, L_{out}), where
    L_{out}=\text{output\_size}.

-[ Examples ]-



target output size of 5
m = nn.AdaptiveMaxPool1d(5)
input = torch.randn(1, 64, 8)
output = m(input)
ScriptFunction
class torch.jit.ScriptFunction
Functionally equivalent to a "ScriptModule", but represents a
   single function and does not have any attributes or Parameters.
get_debug_state(self: torch._C.ScriptFunction) -> torch._C.GraphExecutorState
save(self: torch._C.ScriptFunction, filename: str, _extra_files: Dict[str, str] = {}) -> None
save_to_buffer(self: torch._C.ScriptFunction, _extra_files: Dict[str, str] = {}) -> bytes
torch.nn.utils.parametrize.cached
torch.nn.utils.parametrize.cached()
Context manager that enables the caching system within
   parametrizations registered with "register_parametrization()".
The value of the parametrized objects is computed and cached the
   first time they are required when this context manager is active.
   The cached values are discarded when leaving the context manager.
This is useful when using a parametrized parameter more than once
   in the forward pass. An example of this is when parametrizing the
   recurrent kernel of an RNN or when sharing weights.
The simplest way to activate the cache is by wrapping the forward
   pass of the neural network
  import torch.nn.utils.parametrize as P
  ...
  with P.cached():
      output = model(inputs)

in training and evaluation. One may also wrap the parts of the
   modules that use several times the parametrized tensors. For
LeakyReLU
class torch.ao.nn.quantized.LeakyReLU(scale, zero_point, negative_slope=0.01, inplace=False, device=None, dtype=None)
This is the quantized equivalent of "LeakyReLU".
Parameters:
      * scale (float) -- quantization scale of the output tensor
  * **zero_point** (*int*) -- quantization zero point of the
    output tensor

  * **negative_slope** (*float*) -- Controls the angle of the
    negative slope. Default: 1e-2
torch.reshape
torch.reshape(input, shape) -> Tensor
Returns a tensor with the same data and number of elements as
   "input", but with the specified shape. When possible, the returned
   tensor will be a view of "input". Otherwise, it will be a copy.
   Contiguous inputs and inputs with compatible strides can be
   reshaped without copying, but you should not depend on the copying
   vs. viewing behavior.
See "torch.Tensor.view()" on when it is possible to return a view.
A single dimension may be -1, in which case it's inferred from the
   remaining dimensions and the number of elements in "input".
Parameters:
      * input (Tensor) -- the tensor to be reshaped
  * **shape** (*tuple of python:int*) -- the new shape

Example:
  >>> a = torch.arange(4.)
  >>> torch.reshape(a, (2, 2))
  tensor([[ 0.,  1.],
          [ 2.,  3.]])
  >>> b = torch.tensor([[0, 1], [2, 3]])
  >>> torch.reshape(b, (-1,))
  tensor([ 0,  1,  2,  3])
torch.acos
torch.acos(input, *, out=None) -> Tensor
Computes the inverse cosine of each element in "input".
  \text{out}_{i} = \cos^{-1}(\text{input}_{i})

Parameters:
      input (Tensor) -- the input tensor.
Keyword Arguments:
      out (Tensor, optional) -- the output tensor.
Example:
  >>> a = torch.randn(4)
  >>> a
  tensor([ 0.3348, -0.5889,  0.2005, -0.1584])
  >>> torch.acos(a)
  tensor([ 1.2294,  2.2004,  1.3690,  1.7298])
torch.resolve_conj
torch.resolve_conj(input) -> Tensor
Returns a new tensor with materialized conjugation if "input"'s
   conjugate bit is set to True, else returns "input". The output
   tensor will always have its conjugate bit set to False.
Parameters:
      input (Tensor) -- the input tensor.
Example:
  >>> x = torch.tensor([-1 + 1j, -2 + 2j, 3 - 3j])
  >>> y = x.conj()
  >>> y.is_conj()
  True
  >>> z = y.resolve_conj()
  >>> z
  tensor([-1 - 1j, -2 - 2j, 3 + 3j])
  >>> z.is_conj()
  False
Dropout1d
class torch.nn.Dropout1d(p=0.5, inplace=False)
Randomly zero out entire channels (a channel is a 1D feature map,
   e.g., the j-th channel of the i-th sample in the batched input is a
   1D tensor \text{input}[i, j]). Each channel will be zeroed out
   independently on every forward call with probability "p" using
   samples from a Bernoulli distribution.
Usually the input comes from "nn.Conv1d" modules.
As described in the paper Efficient Object Localization Using
   Convolutional Networks , if adjacent pixels within feature maps are
   strongly correlated (as is normally the case in early convolution
   layers) then i.i.d. dropout will not regularize the activations and
   will otherwise just result in an effective learning rate decrease.
In this case, "nn.Dropout1d()" will help promote independence
   between feature maps and should be used instead.
Parameters:
      * p (float, optional) -- probability of an element to
        be zero-ed.
be zero-ed.
  * **inplace** (*bool**, **optional*) -- If set to "True", will
    do this operation in-place

Shape:
      * Input: (N, C, L) or (C, L).
  * Output: (N, C, L) or (C, L) (same shape as input).

Examples:
  >>> m = nn.Dropout1d(p=0.2)
  >>> input = torch.randn(20, 16, 32)
  >>> output = m(input)
MultiplicativeLR
class torch.optim.lr_scheduler.MultiplicativeLR(optimizer, lr_lambda, last_epoch=- 1, verbose=False)
Multiply the learning rate of each parameter group by the factor
   given in the specified function. When last_epoch=-1, sets initial
   lr as lr.
Parameters:
      * optimizer (Optimizer) -- Wrapped optimizer.
  * **lr_lambda** (*function** or **list*) -- A function which
    computes a multiplicative factor given an integer parameter
    epoch, or a list of such functions, one for each group in
    optimizer.param_groups.

  * **last_epoch** (*int*) -- The index of last epoch. Default:
    -1.

  * **verbose** (*bool*) -- If "True", prints a message to stdout
    for each update. Default: "False".

-[ Example ]-



lmbda = lambda epoch: 0.95
scheduler = MultiplicativeLR(optimizer, lr_lambda=lmbda)
for epoch in range(100):
    train(...)
    validate(...)
    scheduler.step()
scheduler.step()




get_last_lr()
  Return last computed learning rate by current scheduler.

load_state_dict(state_dict)
  Loads the schedulers state.

  Parameters:
     **state_dict** (*dict*) -- scheduler state. Should be an
     object returned from a call to "state_dict()".

print_lr(is_verbose, group, lr, epoch=None)
  Display the current learning rate.

state_dict()
  Returns the state of the scheduler as a "dict".

  It contains an entry for every variable in self.__dict__ which
  is not the optimizer. The learning rate lambda functions will
  only be saved if they are callable objects and not if they are
  functions or lambdas.
torch.cuda.reset_peak_memory_stats
torch.cuda.reset_peak_memory_stats(device=None)
Resets the "peak" stats tracked by the CUDA memory allocator.
See "memory_stats()" for details. Peak stats correspond to the
   "peak" key in each individual stat dict.
Parameters:
      device (torch.device or int, optional) -- selected
      device. Returns statistic for the current device, given by
      "current_device()", if "device" is "None" (default).
Note:
 See Memory management for more details about GPU memory
 management.
torch.nn.functional.embedding
torch.nn.functional.embedding(input, weight, padding_idx=None, max_norm=None, norm_type=2.0, scale_grad_by_freq=False, sparse=False)
A simple lookup table that looks up embeddings in a fixed
   dictionary and size.
This module is often used to retrieve word embeddings using
   indices. The input to the module is a list of indices, and the
   embedding matrix, and the output is the corresponding word
   embeddings.
See "torch.nn.Embedding" for more details.
Parameters:
      * input (LongTensor) -- Tensor containing indices into the
        embedding matrix
  * **weight** (*Tensor*) -- The embedding matrix with number of
    rows equal to the maximum possible index + 1, and number of
    columns equal to the embedding size

  * **padding_idx** (*int**, **optional*) -- If specified, the
    entries at "padding_idx" do not contribute to the gradient;