shrinath-suresh/blogs-docs-splitted huggingface dataset:
AdaptiveLogSoftmaxWithLoss
class torch.nn.AdaptiveLogSoftmaxWithLoss(in_features, n_classes, cutoffs, div_value=4.0, head_bias=False, device=None, dtype=None)
Efficient softmax approximation as described in Efficient softmax
   approximation for GPUs by Edouard Grave, Armand Joulin, Moustapha
   CissÃ©, David Grangier, and HervÃ© JÃ©gou.
Adaptive softmax is an approximate strategy for training models
   with large output spaces. It is most effective when the label
   distribution is highly imbalanced, for example in natural language
   modelling, where the word frequency distribution approximately
   follows the Zipf's law.
Adaptive softmax partitions the labels into several clusters,
   according to their frequency. These clusters may contain different
   number of targets each. Additionally, clusters containing less
   frequent labels assign lower dimensional embeddings to those
   labels, which speeds up the computation. For each minibatch, only
clusters for which at least one target is present are evaluated.
The idea is that the clusters which are accessed frequently (like
   the first one, containing most frequent labels), should also be
   cheap to compute -- that is, contain a small number of assigned
   labels.
We highly recommend taking a look at the original paper for more
   details.


"cutoffs" should be an ordered Sequence of integers sorted in the
     increasing order. It controls number of clusters and the
     partitioning of targets into clusters. For example setting
     "cutoffs = [10, 100, 1000]" means that first 10 targets will be
     assigned to the 'head' of the adaptive softmax, targets 11, 12,
     ..., 100 will be assigned to the first cluster, and targets
     101, 102, ..., 1000 will be assigned to the second cluster,
     while targets 1001, 1002, ..., n_classes - 1 will be assigned
     to the last, third cluster.


"div_value" is used to compute the size of each additional
cluster, which is given as \left\lfloor\frac{\texttt{in_feature
     s}}{\texttt{div_value}^{idx}}\right\rfloor, where idx is the
     cluster index (with clusters for less frequent words having
     larger indices, and indices starting from 1).

"head_bias" if set to True, adds a bias term to the 'head' of the
     adaptive softmax. See paper for details. Set to False in the
     official implementation.

Warning:
 Labels passed as inputs to this module should be sorted according
 to their frequency. This means that the most frequent label
 should be represented by the index *0*, and the least frequent
 label should be represented by the index *n_classes - 1*.

Note:
 This module returns a "NamedTuple" with "output" and "loss"
 fields. See further documentation for details.

Note:
 To compute log-probabilities for all classes, the "log_prob"
 method can be used.

Parameters:
      * in_features (int) -- Number of features in the input
tensor
  * **n_classes** (*int*) -- Number of classes in the dataset

  * **cutoffs** (*Sequence*) -- Cutoffs used to assign targets to
    their buckets

  * **div_value** (*float**, **optional*) -- value used as an
    exponent to compute sizes of the clusters. Default: 4.0

  * **head_bias** (*bool**, **optional*) -- If "True", adds a bias
    term to the 'head' of the adaptive softmax. Default: "False"

Returns:
      * output is a Tensor of size "N" containing computed target
        log probabilities for each example
  * **loss** is a Scalar representing the computed negative log
    likelihood loss

Return type:
      "NamedTuple" with "output" and "loss" fields
Shape:
      * input: (N, \texttt{in_features}) or (\texttt{in_features})
  * target: (N) or () where each value satisfies 0 <=
    \texttt{target[i]} <= \texttt{n\_classes}

  * output1: (N) or ()

  * output2: "Scalar"

log_prob(input)
output2: "Scalar"

log_prob(input)
  Computes log probabilities for all \texttt{n\_classes}

  Parameters:
     **input** (*Tensor*) -- a minibatch of examples

  Returns:
     log-probabilities of for each class c in range 0 <= c <=
     \texttt{n\_classes}, where \texttt{n\_classes} is a parameter
     passed to "AdaptiveLogSoftmaxWithLoss" constructor.

  Return type:
     *Tensor*

  Shape:
     * Input: (N, \texttt{in\_features})

     * Output: (N, \texttt{n\_classes})

predict(input)
  This is equivalent to *self.log_prob(input).argmax(dim=1)*, but
  is more efficient in some cases.

  Parameters:
     **input** (*Tensor*) -- a minibatch of examples

  Returns:
     a class with the highest probability for each example

  Return type:
     output (Tensor)

  Shape:
     * Input: (N, \texttt{in\_features})

     * Output: (N)
torch.atleast_1d
torch.atleast_1d(*tensors)
Returns a 1-dimensional view of each input tensor with zero
   dimensions. Input tensors with one or more dimensions are returned
   as-is.
Parameters:
      input (Tensor or list of Tensors) --
Returns:
      output (Tensor or tuple of Tensors)
Example:
  >>> x = torch.arange(2)
  >>> x
  tensor([0, 1])
  >>> torch.atleast_1d(x)
  tensor([0, 1])
  >>> x = torch.tensor(1.)
  >>> x
  tensor(1.)
  >>> torch.atleast_1d(x)
  tensor([1.])
  >>> x = torch.tensor(0.5)
  >>> y = torch.tensor(1.)
  >>> torch.atleast_1d((x, y))
  (tensor([0.5000]), tensor([1.]))
torch.set_flush_denormal
torch.set_flush_denormal(mode) -> bool
Disables denormal floating numbers on CPU.
Returns "True" if your system supports flushing denormal numbers
   and it successfully configures flush denormal mode.
   "set_flush_denormal()" is only supported on x86 architectures
   supporting SSE3.
Parameters:
      mode (bool) -- Controls whether to enable flush denormal
      mode or not
Example:
  >>> torch.set_flush_denormal(True)
  True
  >>> torch.tensor([1e-323], dtype=torch.float64)
  tensor([ 0.], dtype=torch.float64)
  >>> torch.set_flush_denormal(False)
  True
  >>> torch.tensor([1e-323], dtype=torch.float64)
  tensor(9.88131e-324 *
         [ 1.0000], dtype=torch.float64)
SiLU
class torch.nn.SiLU(inplace=False)
Applies the Sigmoid Linear Unit (SiLU) function, element-wise. The
   SiLU function is also known as the swish function.
  \text{silu}(x) = x * \sigma(x), \text{where } \sigma(x) \text{
  is the logistic sigmoid.}

Note:
 See Gaussian Error Linear Units (GELUs) where the SiLU (Sigmoid
 Linear Unit) was originally coined, and see Sigmoid-Weighted
 Linear Units for Neural Network Function Approximation in
 Reinforcement Learning and Swish: a Self-Gated Activation
 Function where the SiLU was experimented with later.

Shape:
      * Input: (*), where * means any number of dimensions.
  * Output: (*), same shape as the input.

[image]
Examples:
  >>> m = nn.SiLU()
  >>> input = torch.randn(2)
  >>> output = m(input)
torch.nn.functional.torch.nn.parallel.data_parallel
torch.nn.parallel.data_parallel(module, inputs, device_ids=None, output_device=None, dim=0, module_kwargs=None)
Evaluates module(input) in parallel across the GPUs given in
   device_ids.
This is the functional version of the DataParallel module.
Parameters:
      * module (Module) -- the module to evaluate in parallel
  * **inputs** (*Tensor*) -- inputs to the module

  * **device_ids** (*list of python:int** or **torch.device*) --
    GPU ids on which to replicate module

  * **output_device** (*list of python:int** or **torch.device*)
    -- GPU location of the output  Use -1 to indicate the CPU.
    (default: device_ids[0])

Returns:
      a Tensor containing the result of module(input) located on
      output_device
torch.cuda.init
torch.cuda.init()
Initialize PyTorch's CUDA state.  You may need to call this
   explicitly if you are interacting with PyTorch via its C API, as
   Python bindings for CUDA functionality will not be available until
   this initialization takes place.  Ordinary users should not need
   this, as all of PyTorch's CUDA methods automatically initialize
   CUDA state on-demand.
Does nothing if the CUDA state is already initialized.
torch.Tensor.real
Tensor.real
Returns a new tensor containing real values of the "self" tensor
   for a complex-valued input tensor. The returned tensor and "self"
   share the same underlying storage.
Returns "self" if "self" is a real-valued tensor tensor.
Example::
      >>> x=torch.randn(4, dtype=torch.cfloat)
      >>> x
      tensor([(0.3100+0.3553j), (-0.5445-0.7896j), (-1.6492-0.0633j), (-0.0638-0.8119j)])
      >>> x.real
      tensor([ 0.3100, -0.5445, -1.6492, -0.0638])
torch.signal.windows.bartlett
torch.signal.windows.bartlett(M, *, sym=True, dtype=None, layout=torch.strided, device=None, requires_grad=False)
Computes the Bartlett window.
The Bartlett window is defined as follows:
  w_n = 1 - \left| \frac{2n}{M - 1} - 1 \right| = \begin{cases}
  \frac{2n}{M - 1} & \text{if } 0 \leq n \leq \frac{M - 1}{2} \\
  2 - \frac{2n}{M - 1} & \text{if } \frac{M - 1}{2} < n < M \\
  \end{cases}

The window is normalized to 1 (maximum value is 1). However, the 1
   doesn't appear if "M" is even and "sym" is True.
Parameters:
      M (int) -- the length of the window. In other words, the
      number of points of the returned window.
Keyword Arguments:
      * sym (bool, optional) -- If False, returns a
        periodic window suitable for use in spectral analysis. If
        True, returns a symmetric window suitable for use in filter
        design. Default: True.
design. Default: True.
  * **dtype** ("torch.dtype", optional) -- the desired data type
    of returned tensor. Default: if "None", uses a global default
    (see "torch.set_default_tensor_type()").

  * **layout** ("torch.layout", optional) -- the desired layout of
    returned Tensor. Default: "torch.strided".

  * **device** ("torch.device", optional) -- the desired device of
    returned tensor. Default: if "None", uses the current device
    for the default tensor type (see
    "torch.set_default_tensor_type()"). "device" will be the CPU
    for CPU tensor types and the current CUDA device for CUDA
    tensor types.

  * **requires_grad** (*bool**, **optional*) -- If autograd should
    record operations on the returned tensor. Default: "False".

Return type:
      Tensor
Examples:
  >>> # Generates a symmetric Bartlett window.
  >>> torch.signal.windows.bartlett(10)
torch.signal.windows.bartlett(10)
      tensor([0.0000, 0.2222, 0.4444, 0.6667, 0.8889, 0.8889, 0.6667, 0.4444, 0.2222, 0.0000])



  >>> # Generates a periodic Bartlett window.
  >>> torch.signal.windows.bartlett(10, sym=False)
  tensor([0.0000, 0.2000, 0.4000, 0.6000, 0.8000, 1.0000, 0.8000, 0.6000, 0.4000, 0.2000])
torch.poisson
torch.poisson(input, generator=None) -> Tensor
Returns a tensor of the same size as "input" with each element
   sampled from a Poisson distribution with rate parameter given by
   the corresponding element in "input" i.e.,
  \text{out}_i \sim \text{Poisson}(\text{input}_i)

"input" must be non-negative.
Parameters:
      input (Tensor) -- the input tensor containing the rates of
      the Poisson distribution
Keyword Arguments:
      generator ("torch.Generator", optional) -- a pseudorandom
      number generator for sampling
Example:
  >>> rates = torch.rand(4, 4) * 5  # rate parameter between 0 and 5
  >>> torch.poisson(rates)
  tensor([[9., 1., 3., 5.],
          [8., 6., 6., 0.],
          [0., 4., 5., 3.],
          [2., 1., 4., 2.]])
torch.asin
torch.asin(input, *, out=None) -> Tensor
Returns a new tensor with the arcsine of the elements of "input".
  \text{out}_{i} = \sin^{-1}(\text{input}_{i})

Parameters:
      input (Tensor) -- the input tensor.
Keyword Arguments:
      out (Tensor, optional) -- the output tensor.
Example:
  >>> a = torch.randn(4)
  >>> a
  tensor([-0.5962,  1.4985, -0.4396,  1.4525])
  >>> torch.asin(a)
  tensor([-0.6387,     nan, -0.4552,     nan])
HuberLoss
class torch.nn.HuberLoss(reduction='mean', delta=1.0)
Creates a criterion that uses a squared term if the absolute
   element-wise error falls below delta and a delta-scaled L1 term
   otherwise. This loss combines advantages of both "L1Loss" and
   "MSELoss"; the delta-scaled L1 region makes the loss less sensitive
   to outliers than "MSELoss", while the L2 region provides smoothness
   over "L1Loss" near 0. See Huber loss for more information.
For a batch of size N, the unreduced loss can be described as:
  \ell(x, y) = L = \{l_1, ..., l_N\}^T

with
  l_n = \begin{cases} 0.5 (x_n - y_n)^2, & \text{if } |x_n - y_n|
  < delta \\ delta * (|x_n - y_n| - 0.5 * delta), &
  \text{otherwise } \end{cases}

If reduction is not none, then:
  \ell(x, y) = \begin{cases}     \operatorname{mean}(L), &
  \text{if reduction} = \text{`mean';}\\
  \operatorname{sum}(L),  &  \text{if reduction} = \text{`sum'.}
  \end{cases}

Note:
\end{cases}
Note:
 When delta is set to 1, this loss is equivalent to
 "SmoothL1Loss". In general, this loss differs from "SmoothL1Loss"
 by a factor of delta (AKA beta in Smooth L1). See "SmoothL1Loss"
 for additional discussion on the differences in behavior between
 the two losses.

Parameters:
      * reduction (str, optional) -- Specifies the reduction
        to apply to the output: "'none'" | "'mean'" | "'sum'".
        "'none'": no reduction will be applied, "'mean'": the sum of
        the output will be divided by the number of elements in the
        output, "'sum'": the output will be summed. Default: "'mean'"
  * **delta** (*float**, **optional*) -- Specifies the threshold
    at which to change between delta-scaled L1 and L2 loss. The
    value must be positive.  Default: 1.0

Shape:
      * Input: (*) where * means any number of dimensions.
  * Target: (*), same shape as the input.
Module
class torch.nn.Module
Base class for all neural network modules.
Your models should also subclass this class.
Modules can also contain other Modules, allowing to nest them in a
   tree structure. You can assign the submodules as regular
   attributes:
  import torch.nn as nn
  import torch.nn.functional as F

  class Model(nn.Module):
      def __init__(self):
          super().__init__()
          self.conv1 = nn.Conv2d(1, 20, 5)
          self.conv2 = nn.Conv2d(20, 20, 5)

      def forward(self, x):
          x = F.relu(self.conv1(x))
          return F.relu(self.conv2(x))

Submodules assigned in this way will be registered, and will have
   their parameters converted too when you call "to()", etc.
Note:
 As per the example above, an "__init__()" call to the parent
 class must be made before assignment on the child.

Variables:
      training (bool) -- Boolean represents whether this module
is in training or evaluation mode.
add_module(name, module)
  Adds a child module to the current module.

  The module can be accessed as an attribute using the given name.

  Parameters:
     * **name** (*str*) -- name of the child module. The child
       module can be accessed from this module using the given
       name

     * **module** (*Module*) -- child module to be added to the
       module.

apply(fn)
  Applies "fn" recursively to every submodule (as returned by
  ".children()") as well as self. Typical use includes
  initializing the parameters of a model (see also torch.nn.init).

  Parameters:
     **fn** ("Module" -> None) -- function to be applied to each
     submodule

  Returns:
     self

  Return type:
     Module

  Example:

     >>> @torch.no_grad()
     >>> def init_weights(m):
     >>>     print(m)
     >>>     if type(m) == nn.Linear:
     >>>         m.weight.fill_(1.0)
m.weight.fill_(1.0)
     >>>         print(m.weight)
     >>> net = nn.Sequential(nn.Linear(2, 2), nn.Linear(2, 2))
     >>> net.apply(init_weights)
     Linear(in_features=2, out_features=2, bias=True)
     Parameter containing:
     tensor([[1., 1.],
             [1., 1.]], requires_grad=True)
     Linear(in_features=2, out_features=2, bias=True)
     Parameter containing:
     tensor([[1., 1.],
             [1., 1.]], requires_grad=True)
     Sequential(
       (0): Linear(in_features=2, out_features=2, bias=True)
       (1): Linear(in_features=2, out_features=2, bias=True)
     )




bfloat16()
  Casts all floating point parameters and buffers to "bfloat16"
  datatype.

  Note:

    This method modifies the module in-place.

  Returns:
     self

  Return type:
     Module

buffers(recurse=True)
  Returns an iterator over module buffers.

  Parameters:
Parameters:
         recurse (bool) -- if True, then yields buffers of this
         module and all submodules. Otherwise, yields only buffers
         that are direct members of this module.
  Yields:
     *torch.Tensor* -- module buffer

  Return type:
     *Iterator*[*Tensor*]

  Example:

     >>> for buf in model.buffers():
     >>>     print(type(buf), buf.size())
     <class 'torch.Tensor'> (20L,)
     <class 'torch.Tensor'> (20L, 1L, 5L, 5L)

children()
  Returns an iterator over immediate children modules.

  Yields:
     *Module* -- a child module

  Return type:
     *Iterator*[*Module*]

cpu()
  Moves all model parameters and buffers to the CPU.

  Note:

    This method modifies the module in-place.

  Returns:
     self

  Return type:
     Module

cuda(device=None)
  Moves all model parameters and buffers to the GPU.

  This also makes associated parameters and buffers different
objects. So it should be called before constructing optimizer if
      the module will live on GPU while being optimized.
  Note:

    This method modifies the module in-place.

  Parameters:
     **device** (*int**, **optional*) -- if specified, all
     parameters will be copied to that device

  Returns:
     self

  Return type:
     Module

double()
  Casts all floating point parameters and buffers to "double"
  datatype.

  Note:

    This method modifies the module in-place.

  Returns:
     self

  Return type:
     Module

eval()
  Sets the module in evaluation mode.

  This has any effect only on certain modules. See documentations
  of particular modules for details of their behaviors in
  training/evaluation mode, if they are affected, e.g. "Dropout",
  "BatchNorm", etc.

  This is equivalent with "self.train(False)".

  See Locally disabling gradient computation for a comparison
between .eval() and several similar mechanisms that may be
      confused with it.
  Returns:
     self

  Return type:
     Module

extra_repr()
  Set the extra representation of the module

  To print customized extra information, you should re-implement
  this method in your own modules. Both single-line and multi-line
  strings are acceptable.

  Return type:
     str

float()
  Casts all floating point parameters and buffers to "float"
  datatype.

  Note:

    This method modifies the module in-place.

  Returns:
     self

  Return type:
     Module

forward(*input)
  Defines the computation performed at every call.

  Should be overridden by all subclasses.

  Note:

    Although the recipe for forward pass needs to be defined
    within this function, one should call the "Module" instance
    afterwards instead of this since the former takes care of
running the registered hooks while the latter silently ignores
        them.
get_buffer(target)
  Returns the buffer given by "target" if it exists, otherwise
  throws an error.

  See the docstring for "get_submodule" for a more detailed
  explanation of this method's functionality as well as how to
  correctly specify "target".

  Parameters:
     **target** (*str*) -- The fully-qualified string name of the
     buffer to look for. (See "get_submodule" for how to specify a
     fully-qualified string.)

  Returns:
     The buffer referenced by "target"

  Return type:
     torch.Tensor

  Raises:
     **AttributeError** -- If the target string references an
     invalid     path or resolves to something that is not a
     buffer

get_extra_state()
  Returns any extra state to include in the module's state_dict.
  Implement this and a corresponding "set_extra_state()" for your
module if you need to store extra state. This function is called
      when building the module's state_dict().
  Note that extra state should be picklable to ensure working
  serialization of the state_dict. We only provide provide
  backwards compatibility guarantees for serializing Tensors;
  other objects may break backwards compatibility if their
  serialized pickled form changes.

  Returns:
     Any extra state to store in the module's state_dict

  Return type:
     object

get_parameter(target)
  Returns the parameter given by "target" if it exists, otherwise
  throws an error.

  See the docstring for "get_submodule" for a more detailed
  explanation of this method's functionality as well as how to
  correctly specify "target".

  Parameters:
     **target** (*str*) -- The fully-qualified string name of the
     Parameter to look for. (See "get_submodule" for how to
     specify a fully-qualified string.)
specify a fully-qualified string.)
  Returns:
     The Parameter referenced by "target"

  Return type:
     torch.nn.Parameter

  Raises:
     **AttributeError** -- If the target string references an
     invalid     path or resolves to something that is not an
     "nn.Parameter"

get_submodule(target)
  Returns the submodule given by "target" if it exists, otherwise
  throws an error.

  For example, let's say you have an "nn.Module" "A" that looks
  like this:

     A(
         (net_b): Module(
             (net_c): Module(
                 (conv): Conv2d(16, 33, kernel_size=(3, 3), stride=(2, 2))
             )
             (linear): Linear(in_features=100, out_features=200, bias=True)
         )
     )

  (The diagram shows an "nn.Module" "A". "A" has a nested
  submodule "net_b", which itself has two submodules "net_c" and
  "linear". "net_c" then has a submodule "conv".)
To check whether or not we have the "linear" submodule, we would
      call "get_submodule("net_b.linear")". To check whether we have
      the "conv" submodule, we would call
      "get_submodule("net_b.net_c.conv")".
  The runtime of "get_submodule" is bounded by the degree of
  module nesting in "target". A query against "named_modules"
  achieves the same result, but it is O(N) in the number of
  transitive modules. So, for a simple check to see if some
  submodule exists, "get_submodule" should always be used.

  Parameters:
     **target** (*str*) -- The fully-qualified string name of the
     submodule to look for. (See above example for how to specify
     a fully-qualified string.)

  Returns:
     The submodule referenced by "target"

  Return type:
     torch.nn.Module

  Raises:
     **AttributeError** -- If the target string references an
     invalid     path or resolves to something that is not an
     "nn.Module"
"nn.Module"
half()
  Casts all floating point parameters and buffers to "half"
  datatype.

  Note:

    This method modifies the module in-place.

  Returns:
     self

  Return type:
     Module

ipu(device=None)
  Moves all model parameters and buffers to the IPU.

  This also makes associated parameters and buffers different
  objects. So it should be called before constructing optimizer if
  the module will live on IPU while being optimized.

  Note:

    This method modifies the module in-place.

  Parameters:
     **device** (*int**, **optional*) -- if specified, all
     parameters will be copied to that device

  Returns:
     self

  Return type:
     Module

load_state_dict(state_dict, strict=True)
  Copies parameters and buffers from "state_dict" into this module
  and its descendants. If "strict" is "True", then the keys of
"state_dict" must exactly match the keys returned by this
      module's "state_dict()" function.
  Parameters:
     * **state_dict** (*dict*) -- a dict containing parameters and
       persistent buffers.

     * **strict** (*bool**, **optional*) -- whether to strictly
       enforce that the keys in "state_dict" match the keys
       returned by this module's "state_dict()" function. Default:
       "True"

  Returns:
     * **missing_keys** is a list of str containing the missing
       keys

     * **unexpected_keys** is a list of str containing the
       unexpected keys

  Return type:
     "NamedTuple" with "missing_keys" and "unexpected_keys" fields

  Note:

    If a parameter or buffer is registered as "None" and its
    corresponding key exists in "state_dict", "load_state_dict()"
    will raise a "RuntimeError".

modules()
  Returns an iterator over all modules in the network.

  Yields: