shrinath-suresh/blogs-docs-splitted huggingface dataset:
design. Default: True.
  * **dtype** ("torch.dtype", optional) -- the desired data type
    of returned tensor. Default: if "None", uses a global default
    (see "torch.set_default_tensor_type()").

  * **layout** ("torch.layout", optional) -- the desired layout of
    returned Tensor. Default: "torch.strided".

  * **device** ("torch.device", optional) -- the desired device of
    returned tensor. Default: if "None", uses the current device
    for the default tensor type (see
    "torch.set_default_tensor_type()"). "device" will be the CPU
    for CPU tensor types and the current CUDA device for CUDA
    tensor types.

  * **requires_grad** (*bool**, **optional*) -- If autograd should
    record operations on the returned tensor. Default: "False".

Return type:
      Tensor
Examples:
  >>> # Generates a symmetric Hamming window with the general Hamming window.
  >>> torch.signal.windows.general_hamming(10, sym=True)
tensor([0.0800, 0.1876, 0.4601, 0.7700, 0.9723, 0.9723, 0.7700, 0.4601, 0.1876, 0.0800])
  >>> # Generates a periodic Hann window with the general Hamming window.
  >>> torch.signal.windows.general_hamming(10, alpha=0.5, sym=False)
  tensor([0.0000, 0.0955, 0.3455, 0.6545, 0.9045, 1.0000, 0.9045, 0.6545, 0.3455, 0.0955])
torch.addcdiv
torch.addcdiv(input, tensor1, tensor2, *, value=1, out=None) -> Tensor
Performs the element-wise division of "tensor1" by "tensor2",
   multiplies the result by the scalar "value" and adds it to "input".
Warning:
 Integer division with addcdiv is no longer supported, and in a
 future release addcdiv will perform a true division of tensor1
 and tensor2. The historic addcdiv behavior can be implemented as
 (input + value * torch.trunc(tensor1 / tensor2)).to(input.dtype)
 for integer inputs and as (input + value * tensor1 / tensor2) for
 float inputs. The future addcdiv behavior is just the latter
 implementation: (input + value * tensor1 / tensor2), for all
 dtypes.

  \text{out}_i = \text{input}_i + \text{value} \times
  \frac{\text{tensor1}_i}{\text{tensor2}_i}

The shapes of "input", "tensor1", and "tensor2" must be
   broadcastable.
For inputs of type FloatTensor or DoubleTensor, "value" must be
a real number, otherwise an integer.
Parameters:
      * input (Tensor) -- the tensor to be added
  * **tensor1** (*Tensor*) -- the numerator tensor

  * **tensor2** (*Tensor*) -- the denominator tensor

Keyword Arguments:
      * value (Number, optional) -- multiplier for
        \text{tensor1} / \text{tensor2}
  * **out** (*Tensor**, **optional*) -- the output tensor.

Example:
  >>> t = torch.randn(1, 3)
  >>> t1 = torch.randn(3, 1)
  >>> t2 = torch.randn(1, 3)
  >>> torch.addcdiv(t, t1, t2, value=0.1)
  tensor([[-0.2312, -3.6496,  0.1312],
          [-1.0428,  3.4292, -0.1030],
          [-0.5369, -0.9829,  0.0430]])
VerificationOptions
class torch.onnx.verification.VerificationOptions(flatten=True, ignore_none=True, check_shape=True, check_dtype=True, backend=OnnxBackend.ONNX_RUNTIME_CPU, rtol=0.001, atol=1e-07, remained_onnx_input_idx=None, acceptable_error_percentage=None)
Options for ONNX export verification.
Variables:
      * flatten (bool) -- If True, unpack nested list/tuple/dict
        inputs into a flattened list of Tensors for ONNX. Set this to
        False if nested structures are to be preserved for ONNX, which
        is usually the case with exporting ScriptModules. Default
        True.
  * **ignore_none** (*bool*) -- Whether to ignore None type in
    torch output, which is usually the case with tracing. Set this
    to False, if torch output should keep None type, which is
    usually the case with exporting ScriptModules. Default to
    True.

  * **check_shape** (*bool*) -- Whether to check the shapes
between PyTorch and ONNX Runtime outputs are exactly the same.
        Set this to False to allow output shape broadcasting. Default
        to True.
  * **check_dtype** (*bool*) -- Whether to check the dtypes
    between PyTorch and ONNX Runtime outputs are consistent.
    Default to True.

  * **backend** (*torch.onnx.verification.OnnxBackend*) -- ONNX
    backend for verification. Default to
    OnnxBackend.ONNX_RUNTIME_CPU.

  * **rtol** (*float*) -- relative tolerance in comparison between
    ONNX and PyTorch outputs.

  * **atol** (*float*) -- absolute tolerance in comparison between
    ONNX and PyTorch outputs.

  * **remained_onnx_input_idx**
    (*Optional**[**Sequence**[**int**]**]*) -- If provided, only
    the specified inputs will be passed to the ONNX model. Supply
    a list when there are unused inputs in the model. Since unused
    inputs will be removed in the exported ONNX model, supplying
torch.sqrt
torch.sqrt(input, *, out=None) -> Tensor
Returns a new tensor with the square-root of the elements of
   "input".
  \text{out}_{i} = \sqrt{\text{input}_{i}}

Parameters:
      input (Tensor) -- the input tensor.
Keyword Arguments:
      out (Tensor, optional) -- the output tensor.
Example:
  >>> a = torch.randn(4)
  >>> a
  tensor([-2.0755,  1.0226,  0.0831,  0.4806])
  >>> torch.sqrt(a)
  tensor([    nan,  1.0112,  0.2883,  0.6933])
torch.func.stack_module_state
torch.func.stack_module_state(models) -> params, buffers
Prepares a list of torch.nn.Modules for ensembling with "vmap()".
Given a list of "M" "nn.Modules" of the same class, returns two
   dictionaries that stack all of their parameters and buffers
   together, indexed by name. The stacked parameters are optimizable
   (i.e. they are new leaf nodes in the autograd history that are
   unrelated to the original parameters and can be passed directly to
   an optimizer).
Here's an example of how to ensemble over a very simple model:
  num_models = 5
  batch_size = 64
  in_features, out_features = 3, 3
  models = [torch.nn.Linear(in_features, out_features) for i in range(num_models)]
  data = torch.randn(batch_size, 3)

  def wrapper(params, buffers, data):
      return torch.func.functional_call(model[0], (params, buffers), data)

  params, buffers = stack_module_state(models)
params, buffers = stack_module_state(models)
      output = vmap(wrapper, (0, 0, None))(params, buffers, data)
  assert output.shape == (num_models, batch_size, out_features)

When there's submodules, this follows state dict naming conventions
  import torch.nn as nn
  class Foo(nn.Module):
      def __init__(self, in_features, out_features):
          super().__init__()
          hidden = 4
          self.l1 = nn.Linear(in_features, hidden)
          self.l2 = nn.Linear(hidden, out_features)

      def forward(self, x):
          return self.l2(self.l1(x))

  num_models = 5
  in_features, out_features = 3, 3
  models = [Foo(in_features, out_features) for i in range(num_models)]
  params, buffers = stack_module_state(models)
  print(list(params.keys()))  # "l1.weight", "l1.bias", "l2.weight", "l2.bias"

Warning:
 All of the modules being stacked together must be the same
ConvReLU2d
class torch.ao.nn.intrinsic.quantized.ConvReLU2d(in_channels, out_channels, kernel_size, stride=1, padding=0, dilation=1, groups=1, bias=True, padding_mode='zeros', device=None, dtype=None)
A ConvReLU2d module is a fused module of Conv2d and ReLU
We adopt the same interface as "torch.ao.nn.quantized.Conv2d".
Variables:
      torch.ao.nn.quantized.Conv2d (Same as) --
torch.narrow
torch.narrow(input, dim, start, length) -> Tensor
Returns a new tensor that is a narrowed version of "input" tensor.
   The dimension "dim" is input from "start" to "start + length". The
   returned tensor and "input" tensor share the same underlying
   storage.
Parameters:
      * input (Tensor) -- the tensor to narrow
  * **dim** (*int*) -- the dimension along which to narrow

  * **start** (*int** or **Tensor*) -- index of the element to
    start the narrowed dimension from. Can be negative, which
    means indexing from the end of *dim*. If *Tensor*, it must be
    an 0-dim integral *Tensor* (bools not allowed)

  * **length** (*int*) -- length of the narrowed dimension, must
    be weakly positive

Example:
  >>> x = torch.tensor([[1, 2, 3], [4, 5, 6], [7, 8, 9]])
  >>> torch.narrow(x, 0, 0, 2)
  tensor([[ 1,  2,  3],
          [ 4,  5,  6]])
  >>> torch.narrow(x, 1, 1, 2)
  tensor([[ 2,  3],
torch.nn.functional.dropout2d
torch.nn.functional.dropout2d(input, p=0.5, training=True, inplace=False)
Randomly zero out entire channels (a channel is a 2D feature map,
   e.g., the j-th channel of the i-th sample in the batched input is a
   2D tensor \text{input}[i, j]) of the input tensor). Each channel
   will be zeroed out independently on every forward call with
   probability "p" using samples from a Bernoulli distribution.
See "Dropout2d" for details.
Parameters:
      * p (float) -- probability of a channel to be zeroed.
        Default: 0.5
  * **training** (*bool*) -- apply dropout if is "True". Default:
    "True"

  * **inplace** (*bool*) -- If set to "True", will do this
    operation in-place. Default: "False"

Return type:
      Tensor
torch.nn.functional.upsample_bilinear
torch.nn.functional.upsample_bilinear(input, size=None, scale_factor=None)
Upsamples the input, using bilinear upsampling.
Warning:
 This function is deprecated in favor of
 "torch.nn.functional.interpolate()". This is equivalent with
 "nn.functional.interpolate(..., mode='bilinear',
 align_corners=True)".

Expected inputs are spatial (4 dimensional). Use
   upsample_trilinear fo volumetric (5 dimensional) inputs.
Parameters:
      * input (Tensor) -- input
  * **size** (*int** or **Tuple**[**int**, **int**]*) -- output
    spatial size.

  * **scale_factor** (*int** or **Tuple**[**int**, **int**]*) --
    multiplier for spatial size

Note:
 This operation may produce nondeterministic gradients when given
 tensors on a CUDA device. See Reproducibility for more
 information.
torch.reciprocal
torch.reciprocal(input, *, out=None) -> Tensor
Returns a new tensor with the reciprocal of the elements of "input"
  \text{out}_{i} = \frac{1}{\text{input}_{i}}

Note:
 Unlike NumPy's reciprocal, torch.reciprocal supports integral
 inputs. Integral inputs to reciprocal are automatically promoted
 to the default scalar type.

Parameters:
      input (Tensor) -- the input tensor.
Keyword Arguments:
      out (Tensor, optional) -- the output tensor.
Example:
  >>> a = torch.randn(4)
  >>> a
  tensor([-0.4595, -2.1219, -1.4314,  0.7298])
  >>> torch.reciprocal(a)
  tensor([-2.1763, -0.4713, -0.6986,  1.3702])
torch.cuda.reset_max_memory_cached
torch.cuda.reset_max_memory_cached(device=None)
Resets the starting point in tracking maximum GPU memory managed by
   the caching allocator for a given device.
See "max_memory_cached()" for details.
Parameters:
      device (torch.device or int, optional) -- selected
      device. Returns statistic for the current device, given by
      "current_device()", if "device" is "None" (default).
Warning:
 This function now calls "reset_peak_memory_stats()", which resets
 /all/ peak memory stats.

Note:
 See Memory management for more details about GPU memory
 management.
SparseAdam
class torch.optim.SparseAdam(params, lr=0.001, betas=(0.9, 0.999), eps=1e-08, maximize=False)
Implements lazy version of Adam algorithm suitable for sparse
   tensors.
In this variant, only moments that show up in the gradient get
   updated, and only those portions of the gradient get applied to the
   parameters.
Parameters:
      * params (iterable) -- iterable of parameters to optimize
        or dicts defining parameter groups
  * **lr** (*float**, **optional*) -- learning rate (default:
    1e-3)

  * **betas** (*Tuple**[**float**, **float**]**, **optional*) --
    coefficients used for computing running averages of gradient
    and its square (default: (0.9, 0.999))

  * **eps** (*float**, **optional*) -- term added to the
    denominator to improve numerical stability (default: 1e-8)

  * **maximize** (*bool**, **optional*) -- maximize the params
    based on the objective, instead of minimizing (default: False)
state_dict()
  Returns the state of the optimizer as a "dict".

  It contains two entries:

  * state - a dict holding current optimization state. Its content
       differs between optimizer classes.

  * param_groups - a list containing all parameter groups where
    each
       parameter group is a dict

step(closure=None)
  Performs a single optimization step.

  Parameters:
     **closure** (*Callable**, **optional*) -- A closure that
     reevaluates the model and returns the loss.

zero_grad(set_to_none=False)
  Sets the gradients of all optimized "torch.Tensor" s to zero.

  Parameters:
     **set_to_none** (*bool*) -- instead of setting to zero, set
     the grads to None. This will in general have lower memory
     footprint, and can modestly improve performance. However, it
     changes certain behaviors. For example: 1. When the user
     tries to access a gradient and perform manual ops on it, a
torch.log
torch.log(input, *, out=None) -> Tensor
Returns a new tensor with the natural logarithm of the elements of
   "input".
  y_{i} = \log_{e} (x_{i})

Parameters:
      input (Tensor) -- the input tensor.
Keyword Arguments:
      out (Tensor, optional) -- the output tensor.
Example:
  >>> a = torch.rand(5) * 5
  >>> a
  tensor([4.7767, 4.3234, 1.2156, 0.2411, 4.5739])
  >>> torch.log(a)
  tensor([ 1.5637,  1.4640,  0.1952, -1.4226,  1.5204])
torch.count_nonzero
torch.count_nonzero(input, dim=None) -> Tensor
Counts the number of non-zero values in the tensor "input" along
   the given "dim". If no dim is specified then all non-zeros in the
   tensor are counted.
Parameters:
      * input (Tensor) -- the input tensor.
  * **dim** (*int** or **tuple of ints**, **optional*) -- Dim or
    tuple of dims along which to count non-zeros.

Example:
  >>> x = torch.zeros(3,3)
  >>> x[torch.randn(3,3) > 0.5] = 1
  >>> x
  tensor([[0., 1., 1.],
          [0., 0., 0.],
          [0., 0., 1.]])
  >>> torch.count_nonzero(x)
  tensor(3)
  >>> torch.count_nonzero(x, dim=0)
  tensor([0, 1, 2])
NAdam
class torch.optim.NAdam(params, lr=0.002, betas=(0.9, 0.999), eps=1e-08, weight_decay=0, momentum_decay=0.004, *, foreach=None, differentiable=False)
Implements NAdam algorithm.
  \begin{aligned}      &\rule{110mm}{0.4pt}
  \\      &\textbf{input}      : \gamma_t \text{ (lr)}, \:
  \beta_1,\beta_2 \text{ (betas)},          \: \theta_0 \text{
  (params)}, \: f(\theta) \text{ (objective)}                   \\
  &\hspace{13mm} \: \lambda \text{ (weight decay)}, \:\psi \text{
  (momentum decay)}    \\      &\textbf{initialize} :  m_0
  \leftarrow 0 \text{ ( first moment)},          v_0 \leftarrow 0
  \text{ ( second moment)}
  \\[-1.ex]      &\rule{110mm}{0.4pt}
  \\      &\textbf{for} \: t=1 \: \textbf{to} \: \ldots \:
  \textbf{do}                         \\      &\hspace{5mm}g_t
  \leftarrow   \nabla_{\theta} f_t (\theta_{t-1})           \\
  &\hspace{5mm}if \: \lambda \neq 0
&\hspace{5mm}if \: \lambda \neq 0
      \      &\hspace{10mm} g_t \leftarrow g_t + \lambda \theta_{t-1}
      \      &\hspace{5mm} \mu_t \leftarrow \beta_1 \big(1 -
      \frac{1}{2}  0.96^{t \psi} \big)     \      &\hspace{5mm}
      \mu_{t+1} \leftarrow \beta_1 \big(1 - \frac{1}{2}
      0.96^{(t+1)\psi}\big)\      &\hspace{5mm}m_t
      \leftarrow   \beta_1 m_{t-1} + (1 - \beta_1) g_t          \
      &\hspace{5mm}v_t           \leftarrow   \beta_2 v_{t-1} +
      (1-\beta_2) g^2_t          \      &\hspace{5mm}\widehat{m_t}
      \leftarrow \mu_{t+1} m_t/(1-\prod_{i=1}^{t+1}\mu_i)\[-1.ex]
      & \hspace{11mm} + (1-\mu_t) g_t /(1-\prod_{i=1}^{t} \mu_{i})
      \      &\hspace{5mm}\widehat{v_t} \leftarrow
      v_t/\big(1-\beta_2^t \big)                   \
      &\hspace{5mm}\theta_t \leftarrow \theta_{t-1} - \gamma
      \widehat{m_t}/          \big(\sqrt{\widehat{v_t}} + \epsilon
      \big)                                       \
      &\rule{110mm}{0.4pt}
&\rule{110mm}{0.4pt}
      \[-1.ex]      &\bf{return} \:  \theta_t
      \[-1.ex]      &\rule{110mm}{0.4pt}
      \[-1.ex] \end{aligned}
For further details regarding the algorithm we refer to
   Incorporating Nesterov Momentum into Adam.
Parameters:
      * params (iterable) -- iterable of parameters to optimize
        or dicts defining parameter groups
  * **lr** (*float**, **optional*) -- learning rate (default:
    2e-3)

  * **betas** (*Tuple**[**float**, **float**]**, **optional*) --
    coefficients used for computing running averages of gradient
    and its square (default: (0.9, 0.999))

  * **eps** (*float**, **optional*) -- term added to the
    denominator to improve numerical stability (default: 1e-8)

  * **weight_decay** (*float**, **optional*) -- weight decay (L2
    penalty) (default: 0)

  * **momentum_decay** (*float**, **optional*) -- momentum
    momentum_decay (default: 4e-3)
momentum_decay (default: 4e-3)
  * **foreach** (*bool**, **optional*) -- whether foreach
    implementation of optimizer is used. If unspecified by the
    user (so foreach is None), we will try to use foreach over the
    for-loop implementation on CUDA, since it is usually
    significantly more performant. (default: None)

  * **differentiable** (*bool**, **optional*) -- whether autograd
    should occur through the optimizer step in training.
    Otherwise, the step() function runs in a torch.no_grad()
    context. Setting to True can impair performance, so leave it
    False if you don't intend to run autograd through this
    instance (default: False)

add_param_group(param_group)
  Add a param group to the "Optimizer" s *param_groups*.

  This can be useful when fine tuning a pre-trained network as
  frozen layers can be made trainable and added to the "Optimizer"
  as training progresses.

  Parameters:
torch.rad2deg
torch.rad2deg(input, *, out=None) -> Tensor
Returns a new tensor with each of the elements of "input" converted
   from angles in radians to degrees.
Parameters:
      input (Tensor) -- the input tensor.
Keyword Arguments:
      out (Tensor, optional) -- the output tensor.
Example:
  >>> a = torch.tensor([[3.142, -3.142], [6.283, -6.283], [1.570, -1.570]])
  >>> torch.rad2deg(a)
  tensor([[ 180.0233, -180.0233],
          [ 359.9894, -359.9894],
          [  89.9544,  -89.9544]])
LSTM
class torch.ao.nn.quantized.dynamic.LSTM(args, *kwargs)
A dynamic quantized LSTM module with floating point tensor as
   inputs and outputs. We adopt the same interface as torch.nn.LSTM,
   please see https://pytorch.org/docs/stable/nn.html#torch.nn.LSTM
   for documentation.
Examples:
  >>> rnn = nn.LSTM(10, 20, 2)
  >>> input = torch.randn(5, 3, 10)
  >>> h0 = torch.randn(2, 3, 20)
  >>> c0 = torch.randn(2, 3, 20)
  >>> output, (hn, cn) = rnn(input, (h0, c0))
torch.can_cast
torch.can_cast(from, to) -> bool
Determines if a type conversion is allowed under PyTorch casting
   rules described in the type promotion documentation.
Parameters:
      * from (dtype) -- The original "torch.dtype".
  * **to** (*dtype*) -- The target "torch.dtype".

Example:
  >>> torch.can_cast(torch.double, torch.float)
  True
  >>> torch.can_cast(torch.float, torch.int)
  False
torch.autograd.function.FunctionCtx.set_materialize_grads
FunctionCtx.set_materialize_grads(value)
Sets whether to materialize grad tensors. Default is "True".
This should be called only from inside the "forward()"
   method
If "True", undefined grad tensors will be expanded to tensors full
   of zeros prior to calling the "backward()" and "jvp()" methods.
Example::
      >>> class SimpleFunc(Function):
      >>>     @staticmethod
      >>>     def forward(ctx, x):
      >>>         return x.clone(), x.clone()
      >>>
      >>>     @staticmethod
      >>>     @once_differentiable
      >>>     def backward(ctx, g1, g2):
      >>>         return g1 + g2  # No check for None necessary
      >>>
      >>> # We modify SimpleFunc to handle non-materialized grad outputs
      >>> class Func(Function):
      >>>     @staticmethod
      >>>     def forward(ctx, x):
      >>>         ctx.set_materialize_grads(False)
ctx.set_materialize_grads(False)
  >>>         ctx.save_for_backward(x)
  >>>         return x.clone(), x.clone()
  >>>
  >>>     @staticmethod
  >>>     @once_differentiable
  >>>     def backward(ctx, g1, g2):
  >>>         x, = ctx.saved_tensors
  >>>         grad_input = torch.zeros_like(x)
  >>>         if g1 is not None:  # We must check for None now
  >>>             grad_input += g1
  >>>         if g2 is not None:
  >>>             grad_input += g2
  >>>         return grad_input
  >>>
  >>> a = torch.tensor(1., requires_grad=True)
  >>> b, _ = Func.apply(a)  # induces g2 to be undefined
torch.bmm
torch.bmm(input, mat2, *, out=None) -> Tensor
Performs a batch matrix-matrix product of matrices stored in
   "input" and "mat2".
"input" and "mat2" must be 3-D tensors each containing the same
   number of matrices.
If "input" is a (b \times n \times m) tensor, "mat2" is a (b \times
   m \times p) tensor, "out" will be a (b \times n \times p) tensor.
  \text{out}_i = \text{input}_i \mathbin{@} \text{mat2}_i

This operator supports TensorFloat32.
On certain ROCm devices, when using float16 inputs this module will
   use different precision for backward.
Note:
 This function does not broadcast. For broadcasting matrix
 products, see "torch.matmul()".

Parameters:
      * input (Tensor) -- the first batch of matrices to be
        multiplied
  * **mat2** (*Tensor*) -- the second batch of matrices to be
    multiplied

Keyword Arguments:
      out (Tensor, optional) -- the output tensor.
Example:
torch.cuda.memory_stats
torch.cuda.memory_stats(device=None)
Returns a dictionary of CUDA memory allocator statistics for a
   given device.
The return value of this function is a dictionary of statistics,
   each of which is a non-negative integer.
Core statistics:


""allocated.{all,large_pool,small_pool}.{current,peak,allocated,
     freed}"": number of allocation requests received by the memory
     allocator.


""allocated_bytes.{all,large_pool,small_pool}.{current,peak,allo
     cated,freed}"": amount of allocated memory.


""segment.{all,large_pool,small_pool}.{current,peak,allocated,fr
     eed}"": number of reserved segments from "cudaMalloc()".


""reserved_bytes.{all,large_pool,small_pool}.{current,peak,alloc
     ated,freed}"": amount of reserved memory.


""active.{all,large_pool,small_pool}.{current,peak,allocated,fre
     ed}"": number of active memory blocks.


""active_bytes.{all,large_pool,small_pool}.{current,peak,allocat
ed,freed}"": amount of active memory.


""inactive_split.{all,large_pool,small_pool}.{current,peak,alloc
     ated,freed}"": number of inactive, non-releasable memory blocks.


""inactive_split_bytes.{all,large_pool,small_pool}.{current,peak
     ,allocated,freed}"": amount of inactive, non-releasable memory.


For these core statistics, values are broken down as follows.
Pool type:


"all": combined statistics across all memory pools.


"large_pool": statistics for the large allocation pool (as of
     October 2019, for size >= 1MB allocations).


"small_pool": statistics for the small allocation pool (as of
     October 2019, for size < 1MB allocations).


Metric type:


"current": current value of this metric.


"peak": maximum value of this metric.


"allocated": historical total increase in this metric.


"freed": historical total decrease in this metric.


In addition to the core statistics, we also provide some simple
   event counters:
event counters:


""num_alloc_retries"": number of failed "cudaMalloc" calls that
     result in a cache flush and retry.


""num_ooms"": number of out-of-memory errors thrown.


The caching allocator can be configured via ENV to not split blocks
   larger than a defined size (see Memory Management section of the
   Cuda Semantics documentation). This helps avoid memory
   fragmentation but may have a performance penalty. Additional
   outputs to assist with tuning and evaluating impact:


""max_split_size"": blocks above this size will not be split.


""oversize_allocations.{current,peak,allocated,freed}"": number
     of over-size allocation requests received by the memory
     allocator.


""oversize_segments.{current,peak,allocated,freed}"": number of
     over-size reserved segments from "cudaMalloc()".


Parameters:
      device (torch.device or int, optional) -- selected
      device. Returns statistics for the current device, given by