shrinath-suresh/blogs-docs-splitted huggingface dataset:
Keyword Arguments:
      out (Tensor, optional) -- the output tensor.
Example:
  >>> a = torch.randn(3, 3)
  >>> a
  tensor([[-1.0813, -0.8619,  0.7105],
          [ 0.0935,  0.1380,  2.2112],
          [-0.3409, -0.9828,  0.0289]])
  >>> torch.tril(a)
  tensor([[-1.0813,  0.0000,  0.0000],
          [ 0.0935,  0.1380,  0.0000],
          [-0.3409, -0.9828,  0.0289]])

  >>> b = torch.randn(4, 6)
  >>> b
  tensor([[ 1.2219,  0.5653, -0.2521, -0.2345,  1.2544,  0.3461],
          [ 0.4785, -0.4477,  0.6049,  0.6368,  0.8775,  0.7145],
          [ 1.1502,  3.2716, -1.1243, -0.5413,  0.3615,  0.6864],
          [-0.0614, -0.7344, -1.3164, -0.7648, -1.4024,  0.0978]])
  >>> torch.tril(b, diagonal=1)
  tensor([[ 1.2219,  0.5653,  0.0000,  0.0000,  0.0000,  0.0000],
          [ 0.4785, -0.4477,  0.6049,  0.0000,  0.0000,  0.0000],
          [ 1.1502,  3.2716, -1.1243, -0.5413,  0.0000,  0.0000],
[-0.0614, -0.7344, -1.3164, -0.7648, -1.4024,  0.0000]])
      >>> torch.tril(b, diagonal=-1)
      tensor([[ 0.0000,  0.0000,  0.0000,  0.0000,  0.0000,  0.0000],
              [ 0.4785,  0.0000,  0.0000,  0.0000,  0.0000,  0.0000],
              [ 1.1502,  3.2716,  0.0000,  0.0000,  0.0000,  0.0000],
              [-0.0614, -0.7344, -1.3164,  0.0000,  0.0000,  0.0000]])
torch.cuda.max_memory_allocated
torch.cuda.max_memory_allocated(device=None)
Returns the maximum GPU memory occupied by tensors in bytes for a
   given device.
By default, this returns the peak allocated memory since the
   beginning of this program. "reset_peak_memory_stats()" can be used
   to reset the starting point in tracking this metric. For example,
   these two functions can measure the peak allocated memory usage of
   each iteration in a training loop.
Parameters:
      device (torch.device or int, optional) -- selected
      device. Returns statistic for the current device, given by
      "current_device()", if "device" is "None" (default).
Return type:
      int
Note:
 See Memory management for more details about GPU memory
 management.
torch.cuda.caching_allocator_delete
torch.cuda.caching_allocator_delete(mem_ptr)
Deletes memory allocated using the CUDA memory allocator.
Memory allocated with "caching_allocator_alloc()". is freed here.
   The associated device and stream are tracked inside the allocator.
Parameters:
      mem_ptr (int) -- memory address to be freed by the
      allocator.
Note:
 See Memory management for more details about GPU memory
 management.
LazyConvTranspose1d
class torch.nn.LazyConvTranspose1d(out_channels, kernel_size, stride=1, padding=0, output_padding=0, groups=1, bias=True, dilation=1, padding_mode='zeros', device=None, dtype=None)
A "torch.nn.ConvTranspose1d" module with lazy initialization of the
   "in_channels" argument of the "ConvTranspose1d" that is inferred
   from the "input.size(1)". The attributes that will be lazily
   initialized are weight and bias.
Check the "torch.nn.modules.lazy.LazyModuleMixin" for further
   documentation on lazy modules and their limitations.
Parameters:
      * out_channels (int) -- Number of channels produced by the
        convolution
  * **kernel_size** (*int** or **tuple*) -- Size of the convolving
    kernel

  * **stride** (*int** or **tuple**, **optional*) -- Stride of the
    convolution. Default: 1

  * **padding** (*int** or **tuple**, **optional*) -- "dilation *
    (kernel_size - 1) - padding" zero-padding will be added to
both sides of the input. Default: 0
  * **output_padding** (*int** or **tuple**, **optional*) --
    Additional size added to one side of the output shape.
    Default: 0

  * **groups** (*int**, **optional*) -- Number of blocked
    connections from input channels to output channels. Default: 1

  * **bias** (*bool**, **optional*) -- If "True", adds a learnable
    bias to the output. Default: "True"

  * **dilation** (*int** or **tuple**, **optional*) -- Spacing
    between kernel elements. Default: 1

See also:
 "torch.nn.ConvTranspose1d" and
 "torch.nn.modules.lazy.LazyModuleMixin"

cls_to_become
  alias of "ConvTranspose1d"
Embedding
class torch.nn.Embedding(num_embeddings, embedding_dim, padding_idx=None, max_norm=None, norm_type=2.0, scale_grad_by_freq=False, sparse=False, _weight=None, _freeze=False, device=None, dtype=None)
A simple lookup table that stores embeddings of a fixed dictionary
   and size.
This module is often used to store word embeddings and retrieve
   them using indices. The input to the module is a list of indices,
   and the output is the corresponding word embeddings.
Parameters:
      * num_embeddings (int) -- size of the dictionary of
        embeddings
  * **embedding_dim** (*int*) -- the size of each embedding vector

  * **padding_idx** (*int**, **optional*) -- If specified, the
    entries at "padding_idx" do not contribute to the gradient;
    therefore, the embedding vector at "padding_idx" is not
    updated during training, i.e. it remains as a fixed "pad". For
    a newly constructed Embedding, the embedding vector at
"padding_idx" will default to all zeros, but can be updated to
        another value to be used as the padding vector.
  * **max_norm** (*float**, **optional*) -- If given, each
    embedding vector with norm larger than "max_norm" is
    renormalized to have norm "max_norm".

  * **norm_type** (*float**, **optional*) -- The p of the p-norm
    to compute for the "max_norm" option. Default "2".

  * **scale_grad_by_freq** (*bool**, **optional*) -- If given,
    this will scale gradients by the inverse of frequency of the
    words in the mini-batch. Default "False".

  * **sparse** (*bool**, **optional*) -- If "True", gradient
    w.r.t. "weight" matrix will be a sparse tensor. See Notes for
    more details regarding sparse gradients.

Variables:
      weight (Tensor) -- the learnable weights of the module of
      shape (num_embeddings, embedding_dim) initialized from
      \mathcal{N}(0, 1)
Shape:
\mathcal{N}(0, 1)
Shape:
      * Input: (*), IntTensor or LongTensor of arbitrary shape
        containing the indices to extract
  * Output: (*, H), where *** is the input shape and
    H=\text{embedding\_dim}

Note:
 Keep in mind that only a limited number of optimizers support
 sparse gradients: currently it's "optim.SGD" (*CUDA* and *CPU*),
 "optim.SparseAdam" (*CUDA* and *CPU*) and "optim.Adagrad" (*CPU*)

Note:
 When "max_norm" is not "None", "Embedding"'s forward method will
 modify the "weight" tensor in-place. Since tensors needed for
 gradient computations cannot be modified in-place, performing a
 differentiable operation on "Embedding.weight" before calling
 "Embedding"'s forward method requires cloning "Embedding.weight"
 when "max_norm" is not "None". For example:

    n, d, m = 3, 5, 7
    embedding = nn.Embedding(n, d, max_norm=True)
    W = torch.randn((m, d), requires_grad=True)
    idx = torch.tensor([1, 2])
idx = torch.tensor([1, 2])
        a = embedding.weight.clone() @ W.t()  # weight must be cloned for this to be differentiable
        b = embedding(idx) @ W.t()  # modifies weight in-place
        out = (a.unsqueeze(0) + b.unsqueeze(1))
        loss = out.sigmoid().prod()
        loss.backward()
Examples:
  >>> # an Embedding module containing 10 tensors of size 3
  >>> embedding = nn.Embedding(10, 3)
  >>> # a batch of 2 samples of 4 indices each
  >>> input = torch.LongTensor([[1, 2, 4, 5], [4, 3, 2, 9]])
  >>> embedding(input)
  tensor([[[-0.0251, -1.6902,  0.7172],
           [-0.6431,  0.0748,  0.6969],
           [ 1.4970,  1.3448, -0.9685],
           [-0.3677, -2.7265, -0.1685]],

          [[ 1.4970,  1.3448, -0.9685],
           [ 0.4362, -0.4004,  0.9400],
           [-0.6431,  0.0748,  0.6969],
           [ 0.9124, -2.3616,  1.1151]]])


  >>> # example with padding_idx
  >>> embedding = nn.Embedding(10, 3, padding_idx=0)
input = torch.LongTensor([[0, 2, 0, 5]])
      >>> embedding(input)
      tensor([[[ 0.0000,  0.0000,  0.0000],
               [ 0.1535, -2.0309,  0.9315],
               [ 0.0000,  0.0000,  0.0000],
               [-0.1655,  0.9897,  0.0635]]])



  >>> # example of changing `pad` vector
  >>> padding_idx = 0
  >>> embedding = nn.Embedding(3, 3, padding_idx=padding_idx)
  >>> embedding.weight
  Parameter containing:
  tensor([[ 0.0000,  0.0000,  0.0000],
          [-0.7895, -0.7089, -0.0364],
          [ 0.6778,  0.5803,  0.2678]], requires_grad=True)
  >>> with torch.no_grad():
  ...     embedding.weight[padding_idx] = torch.ones(3)
  >>> embedding.weight
  Parameter containing:
  tensor([[ 1.0000,  1.0000,  1.0000],
          [-0.7895, -0.7089, -0.0364],
          [ 0.6778,  0.5803,  0.2678]], requires_grad=True)
classmethod from_pretrained(embeddings, freeze=True, padding_idx=None, max_norm=None, norm_type=2.0, scale_grad_by_freq=False, sparse=False)
  Creates Embedding instance from given 2-dimensional FloatTensor.

  Parameters:
     * **embeddings** (*Tensor*) -- FloatTensor containing weights
       for the Embedding. First dimension is being passed to
       Embedding as "num_embeddings", second as "embedding_dim".

     * **freeze** (*bool**, **optional*) -- If "True", the tensor
       does not get updated in the learning process. Equivalent to
       "embedding.weight.requires_grad = False". Default: "True"

     * **padding_idx** (*int**, **optional*) -- If specified, the
       entries at "padding_idx" do not contribute to the gradient;
       therefore, the embedding vector at "padding_idx" is not
       updated during training, i.e. it remains as a fixed "pad".

     * **max_norm** (*float**, **optional*) -- See module
initialization documentation.
     * **norm_type** (*float**, **optional*) -- See module
       initialization documentation. Default "2".

     * **scale_grad_by_freq** (*bool**, **optional*) -- See module
       initialization documentation. Default "False".

     * **sparse** (*bool**, **optional*) -- See module
       initialization documentation.

  Examples:

     >>> # FloatTensor containing pretrained weights
     >>> weight = torch.FloatTensor([[1, 2.3, 3], [4, 5.1, 6.3]])
     >>> embedding = nn.Embedding.from_pretrained(weight)
     >>> # Get embeddings for index 1
     >>> input = torch.LongTensor([1])
     >>> embedding(input)
     tensor([[ 4.0000,  5.1000,  6.3000]])
torch.sparse_csc_tensor
torch.sparse_csc_tensor(ccol_indices, row_indices, values, size=None, *, dtype=None, device=None, requires_grad=False, check_invariants=None) -> Tensor
Constructs a sparse tensor in CSC (Compressed Sparse Column) with
   specified values at the given "ccol_indices" and "row_indices".
   Sparse matrix multiplication operations in CSC format are typically
   faster than that for sparse tensors in COO format. Make you have a
   look at the note on the data type of the indices.
Note:
 If the "device" argument is not specified the device of the given
 "values" and indices tensor(s) must match. If, however, the
 argument is specified the input Tensors will be converted to the
 given device and in turn determine the device of the constructed
 sparse tensor.

Parameters:
      * ccol_indices (array_like) -- (B+1)-dimensional array of
        size "(*batchsize, ncols + 1)".  The last element of each
batch is the number of non-zeros. This tensor encodes the
        index in values and row_indices depending on where the given
        column starts. Each successive number in the tensor subtracted
        by the number before it denotes the number of elements in a
        given column.
  * **row_indices** (*array_like*) -- Row co-ordinates of each
    element in values. (B+1)-dimensional tensor with the same
    length as values.

  * **values** (*array_list*) -- Initial values for the tensor.
    Can be a list, tuple, NumPy "ndarray", scalar, and other types
    that represents a (1+K)-dimensional tensor where "K" is the
    number of dense dimensions.

  * **size** (list, tuple, "torch.Size", optional) -- Size of the
    sparse tensor: "(*batchsize, nrows, ncols, *densesize)". If
    not provided, the size will be inferred as the minimum size
    big enough to hold all non-zero elements.

Keyword Arguments:
Keyword Arguments:
      * dtype ("torch.dtype", optional) -- the desired data type
        of returned tensor.  Default: if None, infers data type from
        "values".
  * **device** ("torch.device", optional) -- the desired device of
    returned tensor.  Default: if None, uses the current device
    for the default tensor type (see
    "torch.set_default_tensor_type()"). "device" will be the CPU
    for CPU tensor types and the current CUDA device for CUDA
    tensor types.

  * **requires_grad** (*bool**, **optional*) -- If autograd should
    record operations on the returned tensor. Default: "False".

  * **check_invariants** (*bool**, **optional*) -- If sparse
    tensor invariants are checked. Default: as returned by
    "torch.sparse.check_sparse_tensor_invariants.is_enabled()",
    initially False.

Example::
      >>> ccol_indices = [0, 2, 4]
      >>> row_indices = [0, 1, 0, 1]
      >>> values = [1, 2, 3, 4]
values = [1, 2, 3, 4]
      >>> torch.sparse_csc_tensor(torch.tensor(ccol_indices, dtype=torch.int64),
      ...                         torch.tensor(row_indices, dtype=torch.int64),
      ...                         torch.tensor(values), dtype=torch.double)
      tensor(ccol_indices=tensor([0, 2, 4]),
             row_indices=tensor([0, 1, 0, 1]),
             values=tensor([1., 2., 3., 4.]), size=(2, 2), nnz=4,
             dtype=torch.float64, layout=torch.sparse_csc)
torch.cuda.memory_summary
torch.cuda.memory_summary(device=None, abbreviated=False)
Returns a human-readable printout of the current memory allocator
   statistics for a given device.
This can be useful to display periodically during training, or when
   handling out-of-memory exceptions.
Parameters:
      * device (torch.device or int, optional) --
        selected device. Returns printout for the current device,
        given by "current_device()", if "device" is "None" (default).
  * **abbreviated** (*bool**, **optional*) -- whether to return an
    abbreviated summary (default: False).

Return type:
      str
Note:
 See Memory management for more details about GPU memory
 management.
torch.diff
torch.diff(input, n=1, dim=- 1, prepend=None, append=None) -> Tensor
Computes the n-th forward difference along the given dimension.
The first-order differences are given by out[i] = input[i + 1] -
   input[i]. Higher-order differences are calculated by using
   "torch.diff()" recursively.
Parameters:
      * input (Tensor) -- the tensor to compute the differences
        on
  * **n** (*int**, **optional*) -- the number of times to
    recursively compute the difference

  * **dim** (*int**, **optional*) -- the dimension to compute the
    difference along. Default is the last dimension.

  * **prepend** (*Tensor**, **optional*) -- values to prepend or
    append to "input" along "dim" before computing the difference.
    Their dimensions must be equivalent to that of input, and
    their shapes must match input's shape except on "dim".

  * **append** (*Tensor**, **optional*) -- values to prepend or
append to "input" along "dim" before computing the difference.
        Their dimensions must be equivalent to that of input, and
        their shapes must match input's shape except on "dim".
Keyword Arguments:
      out (Tensor, optional) -- the output tensor.
Example:
  >>> a = torch.tensor([1, 3, 2])
  >>> torch.diff(a)
  tensor([ 2, -1])
  >>> b = torch.tensor([4, 5])
  >>> torch.diff(a, append=b)
  tensor([ 2, -1,  2,  1])
  >>> c = torch.tensor([[1, 2, 3], [3, 4, 5]])
  >>> torch.diff(c, dim=0)
  tensor([[2, 2, 2]])
  >>> torch.diff(c, dim=1)
  tensor([[1, 1],
          [1, 1]])
torch.addbmm
torch.addbmm(input, batch1, batch2, *, beta=1, alpha=1, out=None) -> Tensor
Performs a batch matrix-matrix product of matrices stored in
   "batch1" and "batch2", with a reduced add step (all matrix
   multiplications get accumulated along the first dimension). "input"
   is added to the final result.
"batch1" and "batch2" must be 3-D tensors each containing the same
   number of matrices.
If "batch1" is a (b \times n \times m) tensor, "batch2" is a (b
   \times m \times p) tensor, "input" must be broadcastable with a (n
   \times p) tensor and "out" will be a (n \times p) tensor.
  out = \beta\ \text{input} + \alpha\ (\sum_{i=0}^{b-1}
  \text{batch1}_i \mathbin{@} \text{batch2}_i)

If "beta" is 0, then "input" will be ignored, and nan and inf
   in it will not be propagated.
For inputs of type FloatTensor or DoubleTensor, arguments
   "beta" and "alpha" must be real numbers, otherwise they should be
   integers.
integers.
This operator supports TensorFloat32.
On certain ROCm devices, when using float16 inputs this module will
   use different precision for backward.
Parameters:
      * batch1 (Tensor) -- the first batch of matrices to be
        multiplied
  * **batch2** (*Tensor*) -- the second batch of matrices to be
    multiplied

Keyword Arguments:
      * beta (Number, optional) -- multiplier for "input"
        (\beta)
  * **input** (*Tensor*) -- matrix to be added

  * **alpha** (*Number**, **optional*) -- multiplier for *batch1 @
    batch2* (\alpha)

  * **out** (*Tensor**, **optional*) -- the output tensor.

Example:
  >>> M = torch.randn(3, 5)
  >>> batch1 = torch.randn(10, 3, 4)
  >>> batch2 = torch.randn(10, 4, 5)
  >>> torch.addbmm(M, batch1, batch2)
  tensor([[  6.6311,   0.0503,   6.9768, -12.0362,  -2.1653],
          [ -4.8185,  -1.4255,  -6.6760,   8.9453,   2.5743],
ScriptModule
class torch.jit.ScriptModule
A wrapper around C++ "torch::jit::Module". "ScriptModule"s contain
   methods, attributes, parameters, and constants. These can be
   accessed the same way as on a normal "nn.Module".
add_module(name, module)
  Adds a child module to the current module.

  The module can be accessed as an attribute using the given name.

  Parameters:
     * **name** (*str*) -- name of the child module. The child
       module can be accessed from this module using the given
       name

     * **module** (*Module*) -- child module to be added to the
       module.

apply(fn)
  Applies "fn" recursively to every submodule (as returned by
  ".children()") as well as self. Typical use includes
  initializing the parameters of a model (see also torch.nn.init).

  Parameters:
     **fn** ("Module" -> None) -- function to be applied to each
     submodule

  Returns:
     self

  Return type:
Returns:
         self
  Return type:
     Module

  Example:

     >>> @torch.no_grad()
     >>> def init_weights(m):
     >>>     print(m)
     >>>     if type(m) == nn.Linear:
     >>>         m.weight.fill_(1.0)
     >>>         print(m.weight)
     >>> net = nn.Sequential(nn.Linear(2, 2), nn.Linear(2, 2))
     >>> net.apply(init_weights)
     Linear(in_features=2, out_features=2, bias=True)
     Parameter containing:
     tensor([[1., 1.],
             [1., 1.]], requires_grad=True)
     Linear(in_features=2, out_features=2, bias=True)
     Parameter containing:
     tensor([[1., 1.],
             [1., 1.]], requires_grad=True)
     Sequential(
       (0): Linear(in_features=2, out_features=2, bias=True)
       (1): Linear(in_features=2, out_features=2, bias=True)
     )

bfloat16()
  Casts all floating point parameters and buffers to "bfloat16"
  datatype.

  Note:
datatype.
  Note:

    This method modifies the module in-place.

  Returns:
     self

  Return type:
     Module

buffers(recurse=True)
  Returns an iterator over module buffers.

  Parameters:
     **recurse** (*bool*) -- if True, then yields buffers of this
     module and all submodules. Otherwise, yields only buffers
     that are direct members of this module.

  Yields:
     *torch.Tensor* -- module buffer

  Return type:
     *Iterator*[*Tensor*]

  Example:

     >>> for buf in model.buffers():
     >>>     print(type(buf), buf.size())
     <class 'torch.Tensor'> (20L,)
     <class 'torch.Tensor'> (20L, 1L, 5L, 5L)

children()
  Returns an iterator over immediate children modules.

  Yields:
     *Module* -- a child module

  Return type:
     *Iterator*[*Module*]

property code
  Returns a pretty-printed representation (as valid Python syntax)
of the internal graph for the "forward" method. See Inspecting
      Code for details.
property code_with_constants
  Returns a tuple of:

  [0] a pretty-printed representation (as valid Python syntax) of
  the internal graph for the "forward" method. See *code*. [1] a
  ConstMap following the CONSTANT.cN format of the output in [0].
  The indices in the [0] output are keys to the underlying
  constant's values.

  See Inspecting Code for details.

cpu()
  Moves all model parameters and buffers to the CPU.

  Note:

    This method modifies the module in-place.

  Returns:
     self

  Return type:
     Module

cuda(device=None)
  Moves all model parameters and buffers to the GPU.

  This also makes associated parameters and buffers different
  objects. So it should be called before constructing optimizer if
  the module will live on GPU while being optimized.

  Note:
Note:
    This method modifies the module in-place.

  Parameters:
     **device** (*int**, **optional*) -- if specified, all
     parameters will be copied to that device

  Returns:
     self

  Return type:
     Module

double()
  Casts all floating point parameters and buffers to "double"
  datatype.

  Note:

    This method modifies the module in-place.

  Returns:
     self

  Return type:
     Module

eval()
  Sets the module in evaluation mode.

  This has any effect only on certain modules. See documentations
  of particular modules for details of their behaviors in
  training/evaluation mode, if they are affected, e.g. "Dropout",
  "BatchNorm", etc.

  This is equivalent with "self.train(False)".

  See Locally disabling gradient computation for a comparison
  between *.eval()* and several similar mechanisms that may be
  confused with it.

  Returns:
     self
Returns:
         self
  Return type:
     Module

extra_repr()
  Set the extra representation of the module

  To print customized extra information, you should re-implement
  this method in your own modules. Both single-line and multi-line
  strings are acceptable.

  Return type:
     str

float()
  Casts all floating point parameters and buffers to "float"
  datatype.

  Note:

    This method modifies the module in-place.

  Returns:
     self

  Return type:
     Module

get_buffer(target)
  Returns the buffer given by "target" if it exists, otherwise
  throws an error.

  See the docstring for "get_submodule" for a more detailed
  explanation of this method's functionality as well as how to
  correctly specify "target".

  Parameters:
     **target** (*str*) -- The fully-qualified string name of the
     buffer to look for. (See "get_submodule" for how to specify a
fully-qualified string.)
  Returns:
     The buffer referenced by "target"

  Return type:
     torch.Tensor

  Raises:
     **AttributeError** -- If the target string references an
     invalid     path or resolves to something that is not a
     buffer

get_extra_state()
  Returns any extra state to include in the module's state_dict.
  Implement this and a corresponding "set_extra_state()" for your
  module if you need to store extra state. This function is called
  when building the module's *state_dict()*.

  Note that extra state should be picklable to ensure working
  serialization of the state_dict. We only provide provide
  backwards compatibility guarantees for serializing Tensors;
  other objects may break backwards compatibility if their
  serialized pickled form changes.

  Returns:
     Any extra state to store in the module's state_dict

  Return type:
     object

get_parameter(target)