shrinath-suresh/blogs-docs-splitted huggingface dataset:
the output tensor having 1 fewer dimension than "input".
Parameters:
      * input (Tensor) -- the input tensor.
  * **dim** (*int*) -- the dimension to reduce.

  * **keepdim** (*bool*) -- whether the output tensor has "dim"
    retained or not.

Keyword Arguments:
      dtype ("torch.dtype", optional) -- the desired data type of
      returned tensor. If specified, the input tensor is casted to
      "dtype" before the operation is performed. This is useful for
      preventing data type overflows. Default: None.
Example:
  >>> a = torch.randn(4, 2)
  >>> a
  tensor([[ 0.5261, -0.3837],
          [ 1.1857, -0.2498],
          [-1.1646,  0.0705],
          [ 1.1131, -1.0629]])
  >>> torch.prod(a, 1)
  tensor([-0.2018, -0.2962, -0.0821, -1.1831])
torch.Tensor.stft
Tensor.stft(n_fft, hop_length=None, win_length=None, window=None, center=True, pad_mode='reflect', normalized=False, onesided=None, return_complex=None)
See "torch.stft()"
Warning:
 This function changed signature at version 0.4.1. Calling with
 the previous signature may cause error or return incorrect
 result.
torch.fft.hfftn
torch.fft.hfftn(input, s=None, dim=None, norm=None, *, out=None) -> Tensor
Computes the n-dimensional discrete Fourier transform of a
   Hermitian symmetric "input" signal.
"input" is interpreted as a one-sided Hermitian signal in the time
   domain. By the Hermitian property, the Fourier transform will be
   real-valued.
Note:
 "hfftn()"/"ihfftn()" are analogous to "rfftn()"/"irfftn()". The
 real FFT expects a real signal in the time-domain and gives
 Hermitian symmetry in the frequency-domain. The Hermitian FFT is
 the opposite; Hermitian symmetric in the time-domain and real-
 valued in the frequency-domain. For this reason, special care
 needs to be taken with the shape argument "s", in the same way as
 with "irfftn()".

Note:
 Some input frequencies must be real-valued to satisfy the
 Hermitian property. In these cases the imaginary component will
 be ignored. For example, any imaginary component in the zero-
frequency term cannot be represented in a real output and so will
     always be ignored.
Note:
 The correct interpretation of the Hermitian input depends on the
 length of the original data, as given by "s". This is because
 each input shape could correspond to either an odd or even length
 signal. By default, the signal is assumed to be even length and
 odd signals will not round-trip properly. It is recommended to
 always pass the signal shape "s".

Note:
 Supports torch.half and torch.chalf on CUDA with GPU Architecture
 SM53 or greater. However it only supports powers of 2 signal
 length in every transformed dimensions. With default arguments,
 the size of last dimension should be (2^n + 1) as argument *s*
 defaults to even output size = 2 * (last_dim_size - 1)

Parameters:
      * input (Tensor) -- the input tensor
  * **s** (*Tuple**[**int**]**, **optional*) -- Signal size in the
transformed dimensions. If given, each dimension "dim[i]" will
        either be zero-padded or trimmed to the length "s[i]" before
        computing the real FFT. If a length "-1" is specified, no
        padding is done in that dimension. Defaults to even output in
        the last dimension: "s[-1] = 2*(input.size(dim[-1]) - 1)".
  * **dim** (*Tuple**[**int**]**, **optional*) -- Dimensions to be
    transformed. The last dimension must be the half-Hermitian
    compressed dimension. Default: all dimensions, or the last
    "len(s)" dimensions if "s" is given.

  * **norm** (*str**, **optional*) --

    Normalization mode. For the forward transform ("hfftn()"),
    these correspond to:

    * ""forward"" - normalize by "1/n"

    * ""backward"" - no normalization

    * ""ortho"" - normalize by "1/sqrt(n)" (making the Hermitian
      FFT orthonormal)

    Where "n = prod(s)" is the logical FFT size. Calling the
backward transform ("ihfftn()") with the same normalization
        mode will apply an overall normalization of "1/n" between the
        two transforms. This is required to make "ihfftn()" the exact
        inverse.
    Default is ""backward"" (no normalization).

Keyword Arguments:
      out (Tensor, optional) -- the output tensor.
-[ Example ]-
Starting from a real frequency-space signal, we can generate a
   Hermitian-symmetric time-domain signal: >>> T = torch.rand(10, 9)



t = torch.fft.ihfftn(T)



Without specifying the output length to "hfftn()", the output will
   not round-trip properly because the input is odd-length in the last
   dimension:



torch.fft.hfftn(t).size()
   torch.Size([10, 10])



So, it is recommended to always pass the signal shape "s".



roundtrip = torch.fft.hfftn(t, T.size())
roundtrip.size()
   torch.Size([10, 9])
torch.allclose(roundtrip, T)
   True
MultiMarginLoss
class torch.nn.MultiMarginLoss(p=1, margin=1.0, weight=None, size_average=None, reduce=None, reduction='mean')
Creates a criterion that optimizes a multi-class classification
   hinge loss (margin-based loss) between input x (a 2D mini-batch
   Tensor) and output y (which is a 1D tensor of target class
   indices, 0 \leq y \leq \text{x.size}(1)-1):
For each mini-batch sample, the loss in terms of the 1D input x and
   scalar output y is:
  \text{loss}(x, y) = \frac{\sum_i \max(0, \text{margin} - x[y] +
  x[i])^p}{\text{x.size}(0)}

where i \in \left{0, \; \cdots , \; \text{x.size}(0) - 1\right}
   and i \neq y.
Optionally, you can give non-equal weighting on the classes by
   passing a 1D "weight" tensor into the constructor.
The loss function then becomes:
  \text{loss}(x, y) = \frac{\sum_i \max(0, w[y] * (\text{margin} -
  x[y] + x[i]))^p}{\text{x.size}(0)}

Parameters:
Parameters:
      * p (int, optional) -- Has a default value of 1. 1 and
        2 are the only supported values.
  * **margin** (*float**, **optional*) -- Has a default value of
    1.

  * **weight** (*Tensor**, **optional*) -- a manual rescaling
    weight given to each class. If given, it has to be a Tensor of
    size *C*. Otherwise, it is treated as if having all ones.

  * **size_average** (*bool**, **optional*) -- Deprecated (see
    "reduction"). By default, the losses are averaged over each
    loss element in the batch. Note that for some losses, there
    are multiple elements per sample. If the field "size_average"
    is set to "False", the losses are instead summed for each
    minibatch. Ignored when "reduce" is "False". Default: "True"

  * **reduce** (*bool**, **optional*) -- Deprecated (see
    "reduction"). By default, the losses are averaged or summed
    over observations for each minibatch depending on
"size_average". When "reduce" is "False", returns a loss per
        batch element instead and ignores "size_average". Default:
        "True"
  * **reduction** (*str**, **optional*) -- Specifies the reduction
    to apply to the output: "'none'" | "'mean'" | "'sum'".
    "'none'": no reduction will be applied, "'mean'": the sum of
    the output will be divided by the number of elements in the
    output, "'sum'": the output will be summed. Note:
    "size_average" and "reduce" are in the process of being
    deprecated, and in the meantime, specifying either of those
    two args will override "reduction". Default: "'mean'"

Shape:
      * Input: (N, C) or (C), where N is the batch size and C is the
        number of classes.
  * Target: (N) or (), where each value is 0 \leq
    \text{targets}[i] \leq C-1.

  * Output: scalar. If "reduction" is "'none'", then same shape as
    the target.

Examples:
  >>> loss = nn.MultiMarginLoss()
loss = nn.MultiMarginLoss()
      >>> x = torch.tensor([[0.1, 0.2, 0.4, 0.8]])
      >>> y = torch.tensor([3])
      >>> # 0.25 * ((1-(0.8-0.1)) + (1-(0.8-0.2)) + (1-(0.8-0.4)))
      >>> loss(x, y)
      tensor(0.32...)
torch.slice_scatter
torch.slice_scatter(input, src, dim=0, start=None, end=None, step=1) -> Tensor
Embeds the values of the "src" tensor into "input" at the given
   dimension. This function returns a tensor with fresh storage; it
   does not create a view.
Parameters:
      * input (Tensor) -- the input tensor.
  * **src** (*Tensor*) -- The tensor to embed into "input"

  * **dim** (*int*) -- the dimension to insert the slice into

  * **start** (*Optional**[**int**]*) -- the start index of where
    to insert the slice

  * **end** (*Optional**[**int**]*) -- the end index of where to
    insert the slice

  * **step** (*int*) -- the how many elements to skip in

Example:
  >>> a = torch.zeros(8, 8)
  >>> b = torch.ones(8)
  >>> a.slice_scatter(b, start=6)
  tensor([[0., 0., 0., 0., 0., 0., 0., 0.],
          [0., 0., 0., 0., 0., 0., 0., 0.],
          [0., 0., 0., 0., 0., 0., 0., 0.],
[0., 0., 0., 0., 0., 0., 0., 0.],
              [0., 0., 0., 0., 0., 0., 0., 0.],
              [0., 0., 0., 0., 0., 0., 0., 0.],
              [0., 0., 0., 0., 0., 0., 0., 0.],
              [1., 1., 1., 1., 1., 1., 1., 1.],
              [1., 1., 1., 1., 1., 1., 1., 1.]])
  >>> b = torch.ones(2)
  >>> a.slice_scatter(b, dim=1, start=2, end=6, step=2)
  tensor([[0., 0., 1., 0., 1., 0., 0., 0.],
          [0., 0., 1., 0., 1., 0., 0., 0.],
          [0., 0., 1., 0., 1., 0., 0., 0.],
          [0., 0., 1., 0., 1., 0., 0., 0.],
          [0., 0., 1., 0., 1., 0., 0., 0.],
          [0., 0., 1., 0., 1., 0., 0., 0.],
          [0., 0., 1., 0., 1., 0., 0., 0.],
          [0., 0., 1., 0., 1., 0., 0., 0.]])
torch.linalg.matrix_power
torch.linalg.matrix_power(A, n, *, out=None) -> Tensor
Computes the n-th power of a square matrix for an integer n.
Supports input of float, double, cfloat and cdouble dtypes. Also
   supports batches of matrices, and if "A" is a batch of matrices
   then the output has the same batch dimensions.
If "n"= 0, it returns the identity matrix (or batch) of the same
   shape as "A". If "n" is negative, it returns the inverse of each
   matrix (if invertible) raised to the power of abs(n).
Note:
 Consider using "torch.linalg.solve()" if possible for multiplying
 a matrix on the left by a negative power as, if "n"*> 0*:

    matrix_power(torch.linalg.solve(A, B), n) == matrix_power(A, -n)  @ B

 It is always preferred to use "solve()" when possible, as it is
 faster and more numerically stable than computing A^{-n}
 explicitly.

See also:
 "torch.linalg.solve()" computes "A"*.inverse() @ *"B" with a
numerically stable algorithm.
Parameters:
      * A (Tensor) -- tensor of shape (, m, m)* where *** is
        zero or more batch dimensions.
  * **n** (*int*) -- the exponent.

Keyword Arguments:
      out (Tensor, optional) -- output tensor. Ignored if
      None. Default: None.
Raises:
      RuntimeError -- if "n"< 0 and the matrix "A" or any matrix
      in the     batch of matrices "A" is not invertible.
Examples:
  >>> A = torch.randn(3, 3)
  >>> torch.linalg.matrix_power(A, 0)
  tensor([[1., 0., 0.],
          [0., 1., 0.],
          [0., 0., 1.]])
  >>> torch.linalg.matrix_power(A, 3)
  tensor([[ 1.0756,  0.4980,  0.0100],
          [-1.6617,  1.4994, -1.9980],
          [-0.4509,  0.2731,  0.8001]])
  >>> torch.linalg.matrix_power(A.expand(2, -1, -1), -2)
  tensor([[[ 0.2640,  0.4571, -0.5511],
          [-1.0163,  0.3491, -1.5292],
          [-0.4899,  0.0822,  0.2773]],
Adadelta
class torch.optim.Adadelta(params, lr=1.0, rho=0.9, eps=1e-06, weight_decay=0, foreach=None, *, maximize=False, differentiable=False)
Implements Adadelta algorithm.
  \begin{aligned}      &\rule{110mm}{0.4pt}
  \\      &\textbf{input}      : \gamma \text{ (lr)}, \: \theta_0
  \text{ (params)},          \: f(\theta) \text{ (objective)}, \:
  \rho \text{ (decay)},          \: \lambda \text{ (weight decay)}
  \\      &\textbf{initialize} :  v_0  \leftarrow 0 \: \text{
  (square avg)},          \: u_0 \leftarrow 0 \: \text{
  (accumulate variables)}                     \\[-1.ex]
  &\rule{110mm}{0.4pt}
  \\      &\textbf{for} \: t=1 \: \textbf{to} \: \ldots \:
  \textbf{do}                         \\      &\hspace{5mm}g_t
  \leftarrow   \nabla_{\theta} f_t (\theta_{t-1})           \\
  &\hspace{5mm}if \: \lambda \neq 0
  \\      &\hspace{10mm} g_t \leftarrow g_t + \lambda
  \theta_{t-1}                            \\      &\hspace{5mm}
v_t      \leftarrow v_{t-1} \rho + g^2_t (1 - \rho)
      \      &\hspace{5mm}\Delta x_t    \leftarrow
      \frac{\sqrt{u_{t-1} +          \epsilon }}{ \sqrt{v_t +
      \epsilon}  }g_t \hspace{21mm}                           \
      &\hspace{5mm} u_t  \leftarrow   u_{t-1}  \rho +           \Delta
      x^2_t  (1 - \rho)
      \      &\hspace{5mm}\theta_t      \leftarrow   \theta_{t-1} -
      \gamma  \Delta x_t            \      &\rule{110mm}{0.4pt}
      \[-1.ex]      &\bf{return} \:  \theta_t
      \[-1.ex]      &\rule{110mm}{0.4pt}
      \[-1.ex] \end{aligned}
For further details regarding the algorithm we refer to ADADELTA:
   An Adaptive Learning Rate Method.
Parameters:
      * params (iterable) -- iterable of parameters to optimize
        or dicts defining parameter groups
  * **rho** (*float**, **optional*) -- coefficient used for
    computing a running average of squared gradients (default:
    0.9)
0.9)
  * **eps** (*float**, **optional*) -- term added to the
    denominator to improve numerical stability (default: 1e-6)

  * **lr** (*float**, **optional*) -- coefficient that scale delta
    before it is applied to the parameters (default: 1.0)

  * **weight_decay** (*float**, **optional*) -- weight decay (L2
    penalty) (default: 0)

  * **foreach** (*bool**, **optional*) -- whether foreach
    implementation of optimizer is used. If unspecified by the
    user (so foreach is None), we will try to use foreach over the
    for-loop implementation on CUDA, since it is usually
    significantly more performant. (default: None)

  * **maximize** (*bool**, **optional*) -- maximize the params
    based on the objective, instead of minimizing (default: False)

  * **differentiable** (*bool**, **optional*) -- whether autograd
    should occur through the optimizer step in training.
Otherwise, the step() function runs in a torch.no_grad()
        context. Setting to True can impair performance, so leave it
        False if you don't intend to run autograd through this
        instance (default: False)
add_param_group(param_group)
  Add a param group to the "Optimizer" s *param_groups*.

  This can be useful when fine tuning a pre-trained network as
  frozen layers can be made trainable and added to the "Optimizer"
  as training progresses.

  Parameters:
     **param_group** (*dict*) -- Specifies what Tensors should be
     optimized along with group specific optimization options.

load_state_dict(state_dict)
  Loads the optimizer state.

  Parameters:
     **state_dict** (*dict*) -- optimizer state. Should be an
     object returned from a call to "state_dict()".

register_step_post_hook(hook)
  Register an optimizer step post hook which will be called after
  optimizer step. It should have the following signature:
torch.autograd.function.FunctionCtx.mark_non_differentiable
FunctionCtx.mark_non_differentiable(*args)
Marks outputs as non-differentiable.
This should be called at most once, only from inside the
   "forward()" method, and all arguments should be tensor outputs.
This will mark outputs as not requiring gradients, increasing the
   efficiency of backward computation. You still need to accept a
   gradient for each output in "backward()", but it's always going to
   be a zero tensor with the same shape as the shape of a
   corresponding output.
This is used e.g. for indices returned from a sort. See example::
      >>> class Func(Function):
      >>>     @staticmethod
      >>>     def forward(ctx, x):
      >>>         sorted, idx = x.sort()
      >>>         ctx.mark_non_differentiable(idx)
      >>>         ctx.save_for_backward(x, idx)
      >>>         return sorted, idx
      >>>
      >>>     @staticmethod
>>>     @staticmethod
  >>>     @once_differentiable
  >>>     def backward(ctx, g1, g2):  # still need to accept g2
  >>>         x, idx = ctx.saved_tensors
  >>>         grad_input = torch.zeros_like(x)
  >>>         grad_input.index_add_(0, idx, g1)
  >>>         return grad_input
torch.sparse.mm
torch.sparse.mm()
  Performs a matrix multiplication of the sparse matrix "mat1" and
  the (sparse or strided) matrix "mat2". Similar to "torch.mm()",
  if "mat1" is a (n \times m) tensor, "mat2" is a (m \times p)
  tensor, out will be a (n \times p) tensor. When "mat1" is a COO
  tensor it must have *sparse_dim = 2*. When inputs are COO
  tensors, this function also supports backward for both inputs.

  Supports both CSR and COO storage formats.

Note:
 This function doesn't support computing derivaties with respect
 to CSR matrices.

 Args:
    mat1 (Tensor): the first sparse matrix to be multiplied mat2
    (Tensor): the second matrix to be multiplied, which could be
    sparse or dense

 Shape:
    The format of the output tensor of this function follows: -
    sparse x sparse -> sparse - sparse x dense -> dense

 Example:

    >>> a = torch.randn(2, 3).to_sparse().requires_grad_(True)
a
        tensor(indices=tensor([[0, 0, 0, 1, 1, 1],
                               [0, 1, 2, 0, 1, 2]]),
               values=tensor([ 1.5901,  0.0183, -0.6146,  1.8061, -0.0112,  0.6302]),
               size=(2, 3), nnz=6, layout=torch.sparse_coo, requires_grad=True)



    >>> b = torch.randn(3, 2, requires_grad=True)
    >>> b
    tensor([[-0.6479,  0.7874],
            [-1.2056,  0.5641],
            [-1.1716, -0.9923]], requires_grad=True)

    >>> y = torch.sparse.mm(a, b)
    >>> y
    tensor([[-0.3323,  1.8723],
            [-1.8951,  0.7904]], grad_fn=<SparseAddmmBackward>)
    >>> y.sum().backward()
    >>> a.grad
    tensor(indices=tensor([[0, 0, 0, 1, 1, 1],
                           [0, 1, 2, 0, 1, 2]]),
           values=tensor([ 0.1394, -0.6415, -2.1639,  0.1394, -0.6415, -2.1639]),
           size=(2, 3), nnz=6, layout=torch.sparse_coo)
torch.Tensor.get_device
Tensor.get_device() -> Device ordinal (Integer)
For CUDA tensors, this function returns the device ordinal of the
   GPU on which the tensor resides. For CPU tensors, this function
   returns -1.
Example:
  >>> x = torch.randn(3, 4, 5, device='cuda:0')
  >>> x.get_device()
  0
  >>> x.cpu().get_device()
  -1
torch.cuda.set_per_process_memory_fraction
torch.cuda.set_per_process_memory_fraction(fraction, device=None)
Set memory fraction for a process. The fraction is used to limit an
   caching allocator to allocated memory on a CUDA device. The allowed
   value equals the total visible memory multiplied fraction. If
   trying to allocate more than the allowed value in a process, will
   raise an out of memory error in allocator.
Parameters:
      * fraction (float) -- Range: 0~1. Allowed memory equals
        total_memory * fraction.
  * **device** (*torch.device** or **int**, **optional*) --
    selected device. If it is "None" the default CUDA device is
    used.

Note:
 In general, the total available free memory is less than the
 total capacity.
torch.func.grad
torch.func.grad(func, argnums=0, has_aux=False)
"grad" operator helps computing gradients of "func" with respect to
   the input(s) specified by "argnums". This operator can be nested to
   compute higher-order gradients.
Parameters:
      * func (Callable) -- A Python function that takes one or
        more arguments. Must return a single-element Tensor. If
        specified "has_aux" equals "True", function can return a tuple
        of single-element Tensor and other auxiliary objects:
        "(output, aux)".
  * **argnums** (*int** or **Tuple**[**int**]*) -- Specifies
    arguments to compute gradients with respect to. "argnums" can
    be single integer or tuple of integers. Default: 0.

  * **has_aux** (*bool*) -- Flag indicating that "func" returns a
    tensor and other auxiliary objects: "(output, aux)". Default:
    False.

Returns:
      Function to compute gradients with respect to its inputs. By
default, the output of the function is the gradient tensor(s)
      with respect to the first argument. If specified "has_aux"
      equals "True", tuple of gradients and output auxiliary objects
      is returned. If "argnums" is a tuple of integers, a tuple of
      output gradients with respect to each "argnums" value is
      returned.
Return type:
      Callable
Example of using "grad":



from torch.func import grad
x = torch.randn([])
cos_x = grad(lambda x: torch.sin(x))(x)
assert torch.allclose(cos_x, x.cos())
Second-order gradients
neg_sin_x = grad(grad(lambda x: torch.sin(x)))(x)
assert torch.allclose(neg_sin_x, -x.sin())



When composed with "vmap", "grad" can be used to compute per-
   sample-gradients:



from torch.func import grad, vmap
batch_size, feature_size = 3, 5
def model(weights, feature_vec):
    # Very simple linear model with activation
    assert feature_vec.dim() == 1
assert feature_vec.dim() == 1
return feature_vec.dot(weights).relu()

def compute_loss(weights, example, target):
    y = model(weights, example)
    return ((y - target) ** 2).mean()  # MSELoss
weights = torch.randn(feature_size, requires_grad=True)
examples = torch.randn(batch_size, feature_size)
targets = torch.randn(batch_size)
inputs = (weights, examples, targets)
grad_weight_per_example = vmap(grad(compute_loss), in_dims=(None, 0, 0))(*inputs)



Example of using "grad" with "has_aux" and "argnums":



from torch.func import grad
def my_loss_func(y, y_pred):
   loss_per_sample = (0.5 * y_pred - y) ** 2
   loss = loss_per_sample.mean()
   return loss, (y_pred, loss_per_sample)
fn = grad(my_loss_func, argnums=(0, 1), has_aux=True)
y_true = torch.rand(4)
y_preds = torch.rand(4, requires_grad=True)
out = fn(y_true, y_preds)
out = fn(y_true, y_preds)
> output is ((grads w.r.t y_true, grads w.r.t y_preds), (y_pred, loss_per_sample))



Note:
 Using PyTorch "torch.no_grad" together with "grad".Case 1: Using
 "torch.no_grad" inside a function:

 >>> def f(x):
 >>>     with torch.no_grad():
 >>>         c = x ** 2
 >>>     return x - c

 In this case, "grad(f)(x)" will respect the inner
 "torch.no_grad".Case 2: Using "grad" inside "torch.no_grad"
 context manager:

 >>> with torch.no_grad():
 >>>     grad(f)(x)

 In this case, "grad" will respect the inner "torch.no_grad", but
 not the outer one. This is because "grad" is a "function
 transform": its result should not depend on the result of a
 context manager outside of "f".
torch.sparse_coo_tensor
torch.sparse_coo_tensor(indices, values, size=None, *, dtype=None, device=None, requires_grad=False, check_invariants=None) -> Tensor
Constructs a sparse tensor in COO(rdinate) format with specified
   values at the given "indices".
Note:
 This function returns an uncoalesced tensor.

Note:
 If the "device" argument is not specified the device of the given
 "values" and indices tensor(s) must match. If, however, the
 argument is specified the input Tensors will be converted to the
 given device and in turn determine the device of the constructed
 sparse tensor.

Parameters:
      * indices (array_like) -- Initial data for the tensor. Can
        be a list, tuple, NumPy "ndarray", scalar, and other types.
        Will be cast to a "torch.LongTensor" internally. The indices
        are the coordinates of the non-zero values in the matrix, and
        thus should be two-dimensional where the first dimension is
the number of tensor dimensions and the second dimension is
        the number of non-zero values.
  * **values** (*array_like*) -- Initial values for the tensor.
    Can be a list, tuple, NumPy "ndarray", scalar, and other
    types.

  * **size** (list, tuple, or "torch.Size", optional) -- Size of
    the sparse tensor. If not provided the size will be inferred
    as the minimum size big enough to hold all non-zero elements.

Keyword Arguments:
      * dtype ("torch.dtype", optional) -- the desired data type
        of returned tensor. Default: if None, infers data type from
        "values".
  * **device** ("torch.device", optional) -- the desired device of
    returned tensor. Default: if None, uses the current device for
    the default tensor type (see
    "torch.set_default_tensor_type()"). "device" will be the CPU
    for CPU tensor types and the current CUDA device for CUDA
    tensor types.