shrinath-suresh/blogs-docs-splitted huggingface dataset:
torch.backends.cudnn.allow_tf32
A "bool" that controls where TensorFloat-32 tensor cores may be
   used in cuDNN convolutions on Ampere or newer GPUs. See
   TensorFloat-32(TF32) on Ampere devices.
torch.backends.cudnn.deterministic
A "bool" that, if True, causes cuDNN to only use deterministic
   convolution algorithms. See also
   "torch.are_deterministic_algorithms_enabled()" and
   "torch.use_deterministic_algorithms()".
torch.backends.cudnn.benchmark
A "bool" that, if True, causes cuDNN to benchmark multiple
   convolution algorithms and select the fastest.
torch.backends.cudnn.benchmark_limit
A "int" that specifies the maximum number of cuDNN convolution
   algorithms to try when torch.backends.cudnn.benchmark is True.
   Set benchmark_limit to zero to try every available algorithm.
   Note that this setting only affects convolutions dispatched via the
   cuDNN v8 API.
torch.backends.mps
torch.backends.mps.is_available()
torch.backends.mps.is_available()
Returns a bool indicating if MPS is currently available.
Return type:
      bool
torch.backends.mps.is_built()
Returns whether PyTorch is built with MPS support. Note that this
   doesn't necessarily mean MPS is available; just that if this
   PyTorch binary were run a machine with working MPS drivers and
   devices, we would be able to use it.
Return type:
      bool
torch.backends.mkl
torch.backends.mkl.is_available()
Returns whether PyTorch is built with MKL support.
class torch.backends.mkl.verbose(enable)
On-demand oneMKL verbosing functionality To make it easier to debug
   performance issues, oneMKL can dump verbose messages containing
   execution information like duration while executing the kernel. The
   verbosing functionality can be invoked via an environment variable
   named MKL_VERBOSE. However, this methodology dumps messages in
   all steps. Those are a large amount of verbose messages. Moreover,
for investigating the performance issues, generally taking verbose
   messages for one single iteration is enough. This on-demand
   verbosing functionality makes it possible to control scope for
   verbose message dumping. In the following example, verbose messages
   will be dumped out for the second inference only.
  import torch
  model(data)
  with torch.backends.mkl.verbose(torch.backends.mkl.VERBOSE_ON):
      model(data)

Parameters:
      level -- Verbose level - "VERBOSE_OFF": Disable verbosing -
      "VERBOSE_ON":  Enable verbosing
torch.backends.mkldnn
torch.backends.mkldnn.is_available()
Returns whether PyTorch is built with MKL-DNN support.
class torch.backends.mkldnn.verbose(level)
On-demand oneDNN (former MKL-DNN) verbosing functionality To make
   it easier to debug performance issues, oneDNN can dump verbose
   messages containing information like kernel size, input data size
and execution duration while executing the kernel. The verbosing
   functionality can be invoked via an environment variable named
   DNNL_VERBOSE. However, this methodology dumps messages in all
   steps. Those are a large amount of verbose messages. Moreover, for
   investigating the performance issues, generally taking verbose
   messages for one single iteration is enough. This on-demand
   verbosing functionality makes it possible to control scope for
   verbose message dumping. In the following example, verbose messages
   will be dumped out for the second inference only.
  import torch
  model(data)
  with torch.backends.mkldnn.verbose(torch.backends.mkldnn.VERBOSE_ON):
      model(data)

Parameters:
      level -- Verbose level - "VERBOSE_OFF": Disable verbosing -
      "VERBOSE_ON":  Enable verbosing - "VERBOSE_ON_CREATION": Enable
      verbosing, including oneDNN kernel creation
torch.backends.openmp
torch.backends.openmp.is_available()
torch.backends.openmp.is_available()
Returns whether PyTorch is built with OpenMP support.
torch.backends.opt_einsum
torch.backends.opt_einsum.is_available()
Returns a bool indicating if opt_einsum is currently available.
Return type:
      bool
torch.backends.opt_einsum.get_opt_einsum()
Returns the opt_einsum package if opt_einsum is currently
   available, else None.
Return type:
      Any
torch.backends.opt_einsum.enabled
A :class:"bool" that controls whether opt_einsum is enabled ("True"
   by default). If so, torch.einsum will use opt_einsum (https
   ://optimized-einsum.readthedocs.io/en/stable/path_finding.html) if
   available to calculate an optimal path of contraction for faster
   performance.
If opt_einsum is not available, torch.einsum will fall back to the
   default contraction path of left to right.
torch.backends.opt_einsum.strategy
A :class:"str" that specifies which strategies to try when
"torch.backends.opt_einsum.enabled" is "True". By default,
   torch.einsum will try the "auto" strategy, but the "greedy" and
   "optimal" strategies are also supported. Note that the "optimal"
   strategy is factorial on the number of inputs as it tries all
   possible paths. See more details in opt_einsum's docs (https
   ://optimized-einsum.readthedocs.io/en/stable/path_finding.html).
torch.backends.xeon
torch.utils.dlpack
torch.utils.dlpack.from_dlpack(ext_tensor) -> Tensor
Converts a tensor from an external library into a "torch.Tensor".
The returned PyTorch tensor will share the memory with the input
   tensor (which may have come from another library). Note that in-
   place operations will therefore also affect the data of the input
   tensor. This may lead to unexpected issues (e.g., other libraries
   may have read-only flags or immutable data structures), so the user
   should only do this if they know for sure that this is fine.
Parameters:
      ext_tensor (object with "dlpack" attribute, or a DLPack
      capsule) --
  The tensor or DLPack capsule to convert.

  If "ext_tensor" is a tensor (or ndarray) object, it must support
  the "__dlpack__" protocol (i.e., have a "ext_tensor.__dlpack__"
  method). Otherwise "ext_tensor" may be a DLPack capsule, which
  is an opaque "PyCapsule" instance, typically produced by a
"to_dlpack" function or method.
Return type:
      Tensor
Examples:
  >>> import torch.utils.dlpack
  >>> t = torch.arange(4)

  # Convert a tensor directly (supported in PyTorch >= 1.10)
  >>> t2 = torch.from_dlpack(t)
  >>> t2[:2] = -1  # show that memory is shared
  >>> t2
  tensor([-1, -1,  2,  3])
  >>> t
  tensor([-1, -1,  2,  3])

  # The old-style DLPack usage, with an intermediate capsule object
  >>> capsule = torch.utils.dlpack.to_dlpack(t)
  >>> capsule
  <capsule object "dltensor" at ...>
  >>> t3 = torch.from_dlpack(capsule)
  >>> t3
  tensor([-1, -1,  2,  3])
  >>> t3[0] = -9  # now we're sharing memory between 3 tensors
  >>> t3
  tensor([-9, -1,  2,  3])
  >>> t2
  tensor([-9, -1,  2,  3])
  >>> t
  tensor([-9, -1,  2,  3])

torch.utils.dlpack.to_dlpack(tensor) -> PyCapsule
Returns an opaque object (a "DLPack capsule") representing the
   tensor.
Note:
tensor.
Note:
 "to_dlpack" is a legacy DLPack interface. The capsule it returns
 cannot be used for anything in Python other than use it as input
 to "from_dlpack". The more idiomatic use of DLPack is to call
 "from_dlpack" directly on the tensor object - this works when
 that object has a "__dlpack__" method, which PyTorch and most
 other libraries indeed have now.

Warning:
 Only call "from_dlpack" once per capsule produced with
 "to_dlpack". Behavior when a capsule is consumed multiple times
 is undefined.

Parameters:
      tensor -- a tensor to be exported
The DLPack capsule shares the tensor's memory.
PyTorch Governance | Build + CI
How to Add a New Maintainer
For the person to be a maintainer, a person needs to:


Land at least six commits to the related part of the PyTorch
  repository


At least one of these commits must be submitted in the last six
  months


To add a qualified person to the maintainers' list, please create a PR
that adds a person to the persons of interests page and merge_rules
files. Current maintainers will cast their votes of support. Decision
criteria for approving the PR: * Not earlier than two business days
passed before merging (ensure the majority of the contributors have
seen it) * PR has the correct label (module: ci) * There are no
objections from the current maintainers * There are at least three net
thumbs up from current maintainers (or all maintainers vote thumbs
up when the module has less than 3 maintainers).
Probability distributions - torch.distributions
The "distributions" package contains parameterizable probability
distributions and sampling functions. This allows the construction of
stochastic computation graphs and stochastic gradient estimators for
optimization. This package generally follows the design of the
TensorFlow Distributions package.
It is not possible to directly backpropagate through random samples.
However, there are two main methods for creating surrogate functions
that can be backpropagated through. These are the score function
estimator/likelihood ratio estimator/REINFORCE and the pathwise
derivative estimator. REINFORCE is commonly seen as the basis for
policy gradient methods in reinforcement learning, and the pathwise
derivative estimator is commonly seen in the reparameterization trick
in variational autoencoders. Whilst the score function only requires
the value of samples f(x), the pathwise derivative requires the
derivative f'(x). The next sections discuss these two in a
reinforcement learning example. For more details see Gradient
Estimation Using Stochastic Computation Graphs .
Score function
When the probability density function is differentiable with respect
to its parameters, we only need "sample()" and "log_prob()" to
implement REINFORCE:
\Delta\theta  = \alpha r \frac{\partial\log
   p(a|\pi^\theta(s))}{\partial\theta}
where \theta are the parameters, \alpha is the learning rate, r is the
reward and p(a|\pi^\theta(s)) is the probability of taking action a in
state s given policy \pi^\theta.
In practice we would sample an action from the output of a network,
apply this action in an environment, and then use "log_prob" to
construct an equivalent loss function. Note that we use a negative
because optimizers use gradient descent, whilst the rule above assumes
gradient ascent. With a categorical policy, the code for implementing
REINFORCE would be as follows:
probs = policy_network(state)
probs = policy_network(state)
   # Note that this is equivalent to what used to be called multinomial
   m = Categorical(probs)
   action = m.sample()
   next_state, reward = env.step(action)
   loss = -m.log_prob(action) * reward
   loss.backward()
Pathwise derivative
The other way to implement these stochastic/policy gradients would be
to use the reparameterization trick from the "rsample()" method, where
the parameterized random variable can be constructed via a
parameterized deterministic function of a parameter-free random
variable. The reparameterized sample therefore becomes differentiable.
The code for implementing the pathwise derivative would be as follows:
params = policy_network(state)
   m = Normal(*params)
   # Any distribution with .has_rsample == True could work based on the application
   action = m.rsample()
   next_state, reward = env.step(action)  # Assuming that reward is differentiable
   loss = -reward
   loss.backward()
Distribution
loss.backward()
Distribution
class torch.distributions.distribution.Distribution(batch_shape=torch.Size([]), event_shape=torch.Size([]), validate_args=None)
Bases: "object"
Distribution is the abstract base class for probability
   distributions.
property arg_constraints: Dict[str, Constraint]
  Returns a dictionary from argument names to "Constraint" objects
  that should be satisfied by each argument of this distribution.
  Args that are not tensors need not appear in this dict.

property batch_shape: Size
  Returns the shape over which parameters are batched.

cdf(value)
  Returns the cumulative density/mass function evaluated at
  *value*.

  Parameters:
     **value** (*Tensor*) --

  Return type:
     *Tensor*

entropy()
  Returns entropy of distribution, batched over batch_shape.

  Returns:
     Tensor of shape batch_shape.

  Return type:
     *Tensor*

enumerate_support(expand=True)
enumerate_support(expand=True)
  Returns tensor containing all values supported by a discrete
  distribution. The result will enumerate over dimension 0, so the
  shape of the result will be *(cardinality,) + batch_shape +
  event_shape* (where *event_shape = ()* for univariate
  distributions).

  Note that this enumerates over all batched tensors in lock-step
  *[[0, 0], [1, 1], ...]*. With *expand=False*, enumeration
  happens along dim 0, but with the remaining batch dimensions
  being singleton dimensions, *[[0], [1], ..*.

  To iterate over the full Cartesian product use
  *itertools.product(m.enumerate_support())*.

  Parameters:
     **expand** (*bool*) -- whether to expand the support over the
     batch dims to match the distribution's *batch_shape*.

  Returns:
     Tensor iterating over dimension 0.

  Return type:
     *Tensor*

property event_shape: Size
  Returns the shape of a single sample (without batching).
expand(batch_shape, _instance=None)
  Returns a new distribution instance (or populates an existing
  instance provided by a derived class) with batch dimensions
  expanded to *batch_shape*. This method calls "expand" on the
  distribution's parameters. As such, this does not allocate new
  memory for the expanded distribution instance. Additionally,
  this does not repeat any args checking or parameter broadcasting
  in *__init__.py*, when an instance is first created.

  Parameters:
     * **batch_shape** (*torch.Size*) -- the desired expanded
       size.

     * **_instance** -- new instance provided by subclasses that
       need to override *.expand*.

  Returns:
     New distribution instance with batch dimensions expanded to
     *batch_size*.

icdf(value)
  Returns the inverse cumulative density/mass function evaluated
  at *value*.

  Parameters:
     **value** (*Tensor*) --

  Return type:
     *Tensor*
Return type:
         Tensor
log_prob(value)
  Returns the log of the probability density/mass function
  evaluated at *value*.

  Parameters:
     **value** (*Tensor*) --

  Return type:
     *Tensor*

property mean: Tensor
  Returns the mean of the distribution.

property mode: Tensor
  Returns the mode of the distribution.

perplexity()
  Returns perplexity of distribution, batched over batch_shape.

  Returns:
     Tensor of shape batch_shape.

  Return type:
     *Tensor*

rsample(sample_shape=torch.Size([]))
  Generates a sample_shape shaped reparameterized sample or
  sample_shape shaped batch of reparameterized samples if the
  distribution parameters are batched.

  Return type:
     *Tensor*

sample(sample_shape=torch.Size([]))
  Generates a sample_shape shaped sample or sample_shape shaped
  batch of samples if the distribution parameters are batched.

  Return type:
Return type:
         Tensor
sample_n(n)
  Generates n samples or n batches of samples if the distribution
  parameters are batched.

  Return type:
     *Tensor*

static set_default_validate_args(value)
  Sets whether validation is enabled or disabled.

  The default behavior mimics Python's "assert" statement:
  validation is on by default, but is disabled if Python is run in
  optimized mode (via "python -O"). Validation may be expensive,
  so you may want to disable it once a model is working.

  Parameters:
     **value** (*bool*) -- Whether to enable validation.

property stddev: Tensor
  Returns the standard deviation of the distribution.

property support: Optional[Any]
  Returns a "Constraint" object representing this distribution's
  support.

property variance: Tensor
  Returns the variance of the distribution.

ExponentialFamily
ExponentialFamily
class torch.distributions.exp_family.ExponentialFamily(batch_shape=torch.Size([]), event_shape=torch.Size([]), validate_args=None)
Bases: "Distribution"
ExponentialFamily is the abstract base class for probability
   distributions belonging to an exponential family, whose probability
   mass/density function has the form is defined below
  p_{F}(x; \theta) = \exp(\langle t(x), \theta\rangle - F(\theta)
  + k(x))

where \theta denotes the natural parameters, t(x) denotes the
   sufficient statistic, F(\theta) is the log normalizer function for
   a given family and k(x) is the carrier measure.
Note:
 This class is an intermediary between the *Distribution* class
 and distributions which belong to an exponential family mainly to
 check the correctness of the *.entropy()* and analytic KL
 divergence methods. We use this class to compute the entropy and
 KL divergence using the AD framework and Bregman divergences
(courtesy of: Frank Nielsen and Richard Nock, Entropies and
     Cross-entropies of Exponential Families).
entropy()
  Method to compute the entropy using Bregman divergence of the
  log normalizer.

Bernoulli
class torch.distributions.bernoulli.Bernoulli(probs=None, logits=None, validate_args=None)
Bases: "ExponentialFamily"
Creates a Bernoulli distribution parameterized by "probs" or
   "logits" (but not both).
Samples are binary (0 or 1). They take the value 1 with
   probability p and 0 with probability 1 - p.
Example:
  >>> m = Bernoulli(torch.tensor([0.3]))
  >>> m.sample()  # 30% chance 1; 70% chance 0
  tensor([ 0.])

Parameters:
      * probs (Number, Tensor) -- the probability of
        sampling 1
  * **logits** (*Number**, **Tensor*) -- the log-odds of sampling
    *1*

arg_constraints = {'logits': Real(), 'probs': Interval(lower_bound=0.0, upper_bound=1.0)}
entropy()
entropy()
enumerate_support(expand=True)
expand(batch_shape, _instance=None)
has_enumerate_support = True
log_prob(value)
property logits
property mean
property mode
property param_shape
property probs
sample(sample_shape=torch.Size([]))
support = Boolean()
property variance
Beta
class torch.distributions.beta.Beta(concentration1, concentration0, validate_args=None)
Bases: "ExponentialFamily"
Beta distribution parameterized by "concentration1" and
   "concentration0".
Example:
  >>> m = Beta(torch.tensor([0.5]), torch.tensor([0.5]))
  >>> m.sample()  # Beta distributed with concentration concentration1 and concentration0
  tensor([ 0.1046])

Parameters:
      * concentration1 (float or Tensor) -- 1st
        concentration parameter of the distribution (often referred to
        as alpha)
  * **concentration0** (*float** or **Tensor*) -- 2nd
concentration parameter of the distribution (often referred to
        as beta)
arg_constraints = {'concentration0': GreaterThan(lower_bound=0.0), 'concentration1': GreaterThan(lower_bound=0.0)}
property concentration0
property concentration1
entropy()
expand(batch_shape, _instance=None)
has_rsample = True
log_prob(value)
property mean
property mode
rsample(sample_shape=())
support = Interval(lower_bound=0.0, upper_bound=1.0)
property variance
Binomial
class torch.distributions.binomial.Binomial(total_count=1, probs=None, logits=None, validate_args=None)
Bases: "Distribution"
Creates a Binomial distribution parameterized by "total_count" and
   either "probs" or "logits" (but not both). "total_count" must be
   broadcastable with "probs"/"logits".
Example:
  >>> m = Binomial(100, torch.tensor([0 , .2, .8, 1]))
  >>> x = m.sample()
  tensor([   0.,   22.,   71.,  100.])
tensor([   0.,   22.,   71.,  100.])
  >>> m = Binomial(torch.tensor([[5.], [10.]]), torch.tensor([0.5, 0.8]))
  >>> x = m.sample()
  tensor([[ 4.,  5.],
          [ 7.,  6.]])

Parameters:
      * total_count (int or Tensor) -- number of Bernoulli
        trials
  * **probs** (*Tensor*) -- Event probabilities

  * **logits** (*Tensor*) -- Event log-odds

arg_constraints = {'logits': Real(), 'probs': Interval(lower_bound=0.0, upper_bound=1.0), 'total_count': IntegerGreaterThan(lower_bound=0)}
entropy()
enumerate_support(expand=True)
expand(batch_shape, _instance=None)
has_enumerate_support = True
log_prob(value)
property logits
property mean
property mode
property param_shape
property probs
sample(sample_shape=torch.Size([]))
property support
property variance
Categorical
class torch.distributions.categorical.Categorical(probs=None, logits=None, validate_args=None)
Bases: "Distribution"
Bases: "Distribution"
Creates a categorical distribution parameterized by either "probs"
   or "logits" (but not both).
Note:
 It is equivalent to the distribution that "torch.multinomial()"
 samples from.

Samples are integers from {0, \ldots, K-1} where K is
   "probs.size(-1)".
If probs is 1-dimensional with length-K, each element is the
   relative probability of sampling the class at that index.
If probs is N-dimensional, the first N-1 dimensions are treated
   as a batch of relative probability vectors.
Note:
 The *probs* argument must be non-negative, finite and have a non-
 zero sum, and it will be normalized to sum to 1 along the last
 dimension. "probs" will return this normalized value. The
 *logits* argument will be interpreted as unnormalized log
 probabilities and can therefore be any real number. It will
 likewise be normalized so that the resulting probabilities sum to
1 along the last dimension. "logits" will return this normalized
     value.
See also: "torch.multinomial()"
Example:
  >>> m = Categorical(torch.tensor([ 0.25, 0.25, 0.25, 0.25 ]))
  >>> m.sample()  # equal probability of 0, 1, 2, 3
  tensor(3)

Parameters:
      * probs (Tensor) -- event probabilities
  * **logits** (*Tensor*) -- event log probabilities
    (unnormalized)

arg_constraints = {'logits': IndependentConstraint(Real(), 1), 'probs': Simplex()}
entropy()
enumerate_support(expand=True)
expand(batch_shape, _instance=None)
has_enumerate_support = True
log_prob(value)
property logits
property mean
property mode
property param_shape
property probs
sample(sample_shape=torch.Size([]))
property support
property variance
Cauchy
class torch.distributions.cauchy.Cauchy(loc, scale, validate_args=None)
Bases: "Distribution"
Samples from a Cauchy (Lorentz) distribution. The distribution of
the ratio of independent normally distributed random variables with
   means 0 follows a Cauchy distribution.
Example:
  >>> m = Cauchy(torch.tensor([0.0]), torch.tensor([1.0]))
  >>> m.sample()  # sample from a Cauchy distribution with loc=0 and scale=1
  tensor([ 2.3214])

Parameters:
      * loc (float or Tensor) -- mode or median of the
        distribution.
  * **scale** (*float** or **Tensor*) -- half width at half
    maximum.

arg_constraints = {'loc': Real(), 'scale': GreaterThan(lower_bound=0.0)}
cdf(value)
entropy()
expand(batch_shape, _instance=None)
has_rsample = True
icdf(value)
log_prob(value)
property mean
property mode
rsample(sample_shape=torch.Size([]))
support = Real()
property variance
Chi2
class torch.distributions.chi2.Chi2(df, validate_args=None)
Bases: "Gamma"
Creates a Chi-squared distribution parameterized by shape parameter
"df". This is exactly equivalent to "Gamma(alpha=0.5*df, beta=0.5)"
Example:
  >>> m = Chi2(torch.tensor([1.0]))
  >>> m.sample()  # Chi2 distributed with shape df=1
  tensor([ 0.1046])

Parameters:
      df (float or Tensor) -- shape parameter of the
      distribution
arg_constraints = {'df': GreaterThan(lower_bound=0.0)}
property df
expand(batch_shape, _instance=None)
ContinuousBernoulli
class torch.distributions.continuous_bernoulli.ContinuousBernoulli(probs=None, logits=None, lims=(0.499, 0.501), validate_args=None)
Bases: "ExponentialFamily"
Creates a continuous Bernoulli distribution parameterized by
   "probs" or "logits" (but not both).
The distribution is supported in [0, 1] and parameterized by
   'probs' (in (0,1)) or 'logits' (real-valued). Note that, unlike the
   Bernoulli, 'probs' does not correspond to a probability and
   'logits' does not correspond to log-odds, but the same names are