are interchangeable), you can:
  >>> from torch.distributions.multivariate_normal import MultivariateNormal
  >>> from torch.distributions.normal import Normal
  >>> loc = torch.zeros(3)
  >>> scale = torch.ones(3)
  >>> mvn = MultivariateNormal(loc, scale_tril=torch.diag(scale))
  >>> [mvn.batch_shape, mvn.event_shape]
  [torch.Size([]), torch.Size([3])]
  >>> normal = Normal(loc, scale)
  >>> [normal.batch_shape, normal.event_shape]
  [torch.Size([3]), torch.Size([])]
  >>> diagn = Independent(normal, 1)
  >>> [diagn.batch_shape, diagn.event_shape]
  [torch.Size([]), torch.Size([3])]

Parameters:
      * base_distribution
        (torch.distributions.distribution.Distribution) -- a base
        distribution
  * **reinterpreted_batch_ndims** (*int*) -- the number of batch
    dims to reinterpret as event dims

arg_constraints: Dict[str, Constraint] = {}
entropy()
enumerate_support(expand=True)


entropy()
enumerate_support(expand=True)
expand(batch_shape, _instance=None)
property has_enumerate_support
property has_rsample
log_prob(value)
property mean
property mode
rsample(sample_shape=torch.Size([]))
sample(sample_shape=torch.Size([]))
property support
property variance
Kumaraswamy
class torch.distributions.kumaraswamy.Kumaraswamy(concentration1, concentration0, validate_args=None)
Bases: "TransformedDistribution"
Samples from a Kumaraswamy distribution.
Example:
  >>> m = Kumaraswamy(torch.tensor([1.0]), torch.tensor([1.0]))
  >>> m.sample()  # sample from a Kumaraswamy distribution with concentration alpha=1 and beta=1
  tensor([ 0.1729])

Parameters:
      * concentration1 (float or Tensor) -- 1st
        concentration parameter of the distribution (often referred to
        as alpha)
  * **concentration0** (*float** or **Tensor*) -- 2nd


concentration parameter of the distribution (often referred to
        as beta)
arg_constraints: Dict[str, constraints.Constraint] = {'concentration0': GreaterThan(lower_bound=0.0), 'concentration1': GreaterThan(lower_bound=0.0)}
entropy()
expand(batch_shape, _instance=None)
has_rsample = True
property mean
property mode
support = Interval(lower_bound=0.0, upper_bound=1.0)
property variance
LKJCholesky
class torch.distributions.lkj_cholesky.LKJCholesky(dim, concentration=1.0, validate_args=None)
Bases: "Distribution"
LKJ distribution for lower Cholesky factor of correlation matrices.
   The distribution is controlled by "concentration" parameter \eta to
   make the probability of the correlation matrix M generated from a
   Cholesky factor proportional to \det(M)^{\eta - 1}. Because of
   that, when "concentration == 1", we have a uniform distribution
   over Cholesky factors of correlation matrices:
  L ~ LKJCholesky(dim, concentration)


L ~ LKJCholesky(dim, concentration)
      X = L @ L' ~ LKJCorr(dim, concentration)
Note that this distribution samples the Cholesky factor of
   correlation matrices and not the correlation matrices themselves
   and thereby differs slightly from the derivations in [1] for the
   LKJCorr distribution. For sampling, this uses the Onion method
   from [1] Section 3.
Example:
  >>> l = LKJCholesky(3, 0.5)
  >>> l.sample()  # l @ l.T is a sample of a correlation 3x3 matrix
  tensor([[ 1.0000,  0.0000,  0.0000],
          [ 0.3516,  0.9361,  0.0000],
          [-0.1899,  0.4748,  0.8593]])

Parameters:
      * dimension (dim) -- dimension of the matrices
  * **concentration** (*float** or **Tensor*) --
    concentration/shape parameter of the distribution (often
    referred to as eta)

References
[1] Generating random correlation matrices based on vines and
   extended onion method (2009), Daniel Lewandowski, Dorota


Kurowicka, Harry Joe. Journal of Multivariate Analysis. 100.
   10.1016/j.jmva.2009.04.008
arg_constraints = {'concentration': GreaterThan(lower_bound=0.0)}
expand(batch_shape, _instance=None)
log_prob(value)
sample(sample_shape=torch.Size([]))
support = CorrCholesky()
Laplace
class torch.distributions.laplace.Laplace(loc, scale, validate_args=None)
Bases: "Distribution"
Creates a Laplace distribution parameterized by "loc" and "scale".
Example:
  >>> m = Laplace(torch.tensor([0.0]), torch.tensor([1.0]))
  >>> m.sample()  # Laplace distributed with loc=0, scale=1
  tensor([ 0.1046])

Parameters:
      * loc (float or Tensor) -- mean of the distribution
  * **scale** (*float** or **Tensor*) -- scale of the distribution

arg_constraints = {'loc': Real(), 'scale': GreaterThan(lower_bound=0.0)}
cdf(value)
entropy()
expand(batch_shape, _instance=None)
has_rsample = True
icdf(value)
log_prob(value)


icdf(value)
log_prob(value)
property mean
property mode
rsample(sample_shape=torch.Size([]))
property stddev
support = Real()
property variance
LogNormal
class torch.distributions.log_normal.LogNormal(loc, scale, validate_args=None)
Bases: "TransformedDistribution"
Creates a log-normal distribution parameterized by "loc" and
   "scale" where:
  X ~ Normal(loc, scale)
  Y = exp(X) ~ LogNormal(loc, scale)

Example:
  >>> m = LogNormal(torch.tensor([0.0]), torch.tensor([1.0]))
  >>> m.sample()  # log-normal distributed with mean=0 and stddev=1
  tensor([ 0.1046])

Parameters:
      * loc (float or Tensor) -- mean of log of distribution
  * **scale** (*float** or **Tensor*) -- standard deviation of log
    of the distribution

arg_constraints: Dict[str, constraints.Constraint] = {'loc': Real(), 'scale': GreaterThan(lower_bound=0.0)}
entropy()
expand(batch_shape, _instance=None)
has_rsample = True


has_rsample = True
property loc
property mean
property mode
property scale
support = GreaterThan(lower_bound=0.0)
property variance
LowRankMultivariateNormal
class torch.distributions.lowrank_multivariate_normal.LowRankMultivariateNormal(loc, cov_factor, cov_diag, validate_args=None)
Bases: "Distribution"
Creates a multivariate normal distribution with covariance matrix
   having a low-rank form parameterized by "cov_factor" and
   "cov_diag":
  covariance_matrix = cov_factor @ cov_factor.T + cov_diag

-[ Example ]-

m = LowRankMultivariateNormal(torch.zeros(2), torch.tensor([[1.], [0.]]), torch.ones(2))
m.sample()  # normally distributed with mean=[0,0], cov_factor=[[1],[0]], cov_diag=[1,1]
   tensor([-0.2102, -0.5429])

Parameters:
      * loc (Tensor) -- mean of the distribution with shape
        batch_shape + event_shape
  * **cov_factor** (*Tensor*) -- factor part of low-rank form of


covariance matrix with shape batch_shape + event_shape +
        (rank,)
  * **cov_diag** (*Tensor*) -- diagonal part of low-rank form of
    covariance matrix with shape *batch_shape + event_shape*

Note:
 The computation for determinant and inverse of covariance matrix
 is avoided when *cov_factor.shape[1] << cov_factor.shape[0]*
 thanks to Woodbury matrix identity and matrix determinant lemma.
 Thanks to these formulas, we just need to compute the determinant
 and inverse of the small size "capacitance" matrix:

    capacitance = I + cov_factor.T @ inv(cov_diag) @ cov_factor

arg_constraints = {'cov_diag': IndependentConstraint(GreaterThan(lower_bound=0.0), 1), 'cov_factor': IndependentConstraint(Real(), 2), 'loc': IndependentConstraint(Real(), 1)}
property covariance_matrix
entropy()
expand(batch_shape, _instance=None)
has_rsample = True
log_prob(value)
property mean
property mode
property precision_matrix


property mode
property precision_matrix
rsample(sample_shape=torch.Size([]))
property scale_tril
support = IndependentConstraint(Real(), 1)
property variance
MixtureSameFamily
class torch.distributions.mixture_same_family.MixtureSameFamily(mixture_distribution, component_distribution, validate_args=None)
Bases: "Distribution"
The MixtureSameFamily distribution implements a (batch of)
   mixture distribution where all component are from different
   parameterizations of the same distribution type. It is
   parameterized by a Categorical "selecting distribution" (over k
   component) and a component distribution, i.e., a Distribution
   with a rightmost batch shape (equal to [k]) which indexes each
   (batch of) component.
Examples:
  >>> # Construct Gaussian Mixture Model in 1D consisting of 5 equally
  >>> # weighted normal distributions
  >>> mix = D.Categorical(torch.ones(5,))


mix = D.Categorical(torch.ones(5,))
      >>> comp = D.Normal(torch.randn(5,), torch.rand(5,))
      >>> gmm = MixtureSameFamily(mix, comp)

  >>> # Construct Gaussian Mixture Modle in 2D consisting of 5 equally
  >>> # weighted bivariate normal distributions
  >>> mix = D.Categorical(torch.ones(5,))
  >>> comp = D.Independent(D.Normal(
  ...          torch.randn(5,2), torch.rand(5,2)), 1)
  >>> gmm = MixtureSameFamily(mix, comp)

  >>> # Construct a batch of 3 Gaussian Mixture Models in 2D each
  >>> # consisting of 5 random weighted bivariate normal distributions
  >>> mix = D.Categorical(torch.rand(3,5))
  >>> comp = D.Independent(D.Normal(
  ...         torch.randn(3,5,2), torch.rand(3,5,2)), 1)
  >>> gmm = MixtureSameFamily(mix, comp)

Parameters:
      * mixture_distribution --
        torch.distributions.Categorical-like instance. Manages the
        probability of selecting component. The number of categories


must match the rightmost batch dimension of the
        component_distribution. Must have either scalar
        batch_shape or batch_shape matching
        component_distribution.batch_shape[:-1]
  * **component_distribution** --
    *torch.distributions.Distribution*-like instance. Right-most
    batch dimension indexes component.

arg_constraints: Dict[str, Constraint] = {}
cdf(x)
property component_distribution
expand(batch_shape, _instance=None)
has_rsample = False
log_prob(x)
property mean
property mixture_distribution
sample(sample_shape=torch.Size([]))
property support
property variance
Multinomial
class torch.distributions.multinomial.Multinomial(total_count=1, probs=None, logits=None, validate_args=None)
Bases: "Distribution"
Creates a Multinomial distribution parameterized by "total_count"
   and either "probs" or "logits" (but not both). The innermost


dimension of "probs" indexes over categories. All other dimensions
   index over batches.
Note that "total_count" need not be specified if only "log_prob()"
   is called (see example below)
Note:
 The *probs* argument must be non-negative, finite and have a non-
 zero sum, and it will be normalized to sum to 1 along the last
 dimension. "probs" will return this normalized value. The
 *logits* argument will be interpreted as unnormalized log
 probabilities and can therefore be any real number. It will
 likewise be normalized so that the resulting probabilities sum to
 1 along the last dimension. "logits" will return this normalized
 value.

"sample()" requires a single shared total_count for all
     parameters and samples.

"log_prob()" allows different total_count for each parameter
     and sample.

Example:
  >>> m = Multinomial(100, torch.tensor([ 1., 1., 1., 1.]))
  >>> x = m.sample()  # equal probability of 0, 1, 2, 3


tensor([ 21.,  24.,  30.,  25.])
  >>> Multinomial(probs=torch.tensor([1., 1., 1., 1.])).log_prob(x)
  tensor([-4.1338])

Parameters:
      * total_count (int) -- number of trials
  * **probs** (*Tensor*) -- event probabilities

  * **logits** (*Tensor*) -- event log probabilities
    (unnormalized)

arg_constraints = {'logits': IndependentConstraint(Real(), 1), 'probs': Simplex()}
entropy()
expand(batch_shape, _instance=None)
log_prob(value)
property logits
property mean
property param_shape
property probs
sample(sample_shape=torch.Size([]))
property support
total_count: int
property variance
MultivariateNormal
class torch.distributions.multivariate_normal.MultivariateNormal(loc, covariance_matrix=None, precision_matrix=None, scale_tril=None, validate_args=None)
Bases: "Distribution"
Creates a multivariate normal (also called Gaussian) distribution


parameterized by a mean vector and a covariance matrix.
The multivariate normal distribution can be parameterized either in
   terms of a positive definite covariance matrix \mathbf{\Sigma} or a
   positive definite precision matrix \mathbf{\Sigma}^{-1} or a lower-
   triangular matrix \mathbf{L} with positive-valued diagonal entries,
   such that \mathbf{\Sigma} = \mathbf{L}\mathbf{L}^\top. This
   triangular matrix can be obtained via e.g. Cholesky decomposition
   of the covariance.
-[ Example ]-

m = MultivariateNormal(torch.zeros(2), torch.eye(2))
m.sample()  # normally distributed with mean=[0,0] and covariance_matrix=I
   tensor([-0.2102, -0.5429])

Parameters:
      * loc (Tensor) -- mean of the distribution
  * **covariance_matrix** (*Tensor*) -- positive-definite
    covariance matrix

  * **precision_matrix** (*Tensor*) -- positive-definite precision
    matrix

  * **scale_tril** (*Tensor*) -- lower-triangular factor of


covariance, with positive-valued diagonal
Note:
 Only one of "covariance_matrix" or "precision_matrix" or
 "scale_tril" can be specified.Using "scale_tril" will be more
 efficient: all computations internally are based on "scale_tril".
 If "covariance_matrix" or "precision_matrix" is passed instead,
 it is only used to compute the corresponding lower triangular
 matrices using a Cholesky decomposition.

arg_constraints = {'covariance_matrix': PositiveDefinite(), 'loc': IndependentConstraint(Real(), 1), 'precision_matrix': PositiveDefinite(), 'scale_tril': LowerCholesky()}
property covariance_matrix
entropy()
expand(batch_shape, _instance=None)
has_rsample = True
log_prob(value)
property mean
property mode
property precision_matrix
rsample(sample_shape=torch.Size([]))
property scale_tril
support = IndependentConstraint(Real(), 1)
property variance
NegativeBinomial


NegativeBinomial
class torch.distributions.negative_binomial.NegativeBinomial(total_count, probs=None, logits=None, validate_args=None)
Bases: "Distribution"
Creates a Negative Binomial distribution, i.e. distribution of the
   number of successful independent and identical Bernoulli trials
   before "total_count" failures are achieved. The probability of
   success of each Bernoulli trial is "probs".
Parameters:
      * total_count (float or Tensor) -- non-negative number
        of negative Bernoulli trials to stop, although the
        distribution is still valid for real valued count
  * **probs** (*Tensor*) -- Event probabilities of success in the
    half open interval [0, 1)

  * **logits** (*Tensor*) -- Event log-odds for probabilities of
    success

arg_constraints = {'logits': Real(), 'probs': HalfOpenInterval(lower_bound=0.0, upper_bound=1.0), 'total_count': GreaterThanEq(lower_bound=0)}
expand(batch_shape, _instance=None)


expand(batch_shape, _instance=None)
log_prob(value)
property logits
property mean
property mode
property param_shape
property probs
sample(sample_shape=torch.Size([]))
support = IntegerGreaterThan(lower_bound=0)
property variance
Normal
class torch.distributions.normal.Normal(loc, scale, validate_args=None)
Bases: "ExponentialFamily"
Creates a normal (also called Gaussian) distribution parameterized
   by "loc" and "scale".
Example:
  >>> m = Normal(torch.tensor([0.0]), torch.tensor([1.0]))
  >>> m.sample()  # normally distributed with loc=0 and scale=1
  tensor([ 0.1046])

Parameters:
      * loc (float or Tensor) -- mean of the distribution
        (often referred to as mu)
  * **scale** (*float** or **Tensor*) -- standard deviation of the
    distribution (often referred to as sigma)

arg_constraints = {'loc': Real(), 'scale': GreaterThan(lower_bound=0.0)}
cdf(value)
entropy()


cdf(value)
entropy()
expand(batch_shape, _instance=None)
has_rsample = True
icdf(value)
log_prob(value)
property mean
property mode
rsample(sample_shape=torch.Size([]))
sample(sample_shape=torch.Size([]))
property stddev
support = Real()
property variance
OneHotCategorical
class torch.distributions.one_hot_categorical.OneHotCategorical(probs=None, logits=None, validate_args=None)
Bases: "Distribution"
Creates a one-hot categorical distribution parameterized by "probs"
   or "logits".
Samples are one-hot coded vectors of size "probs.size(-1)".
Note:
 The *probs* argument must be non-negative, finite and have a non-
 zero sum, and it will be normalized to sum to 1 along the last
 dimension. "probs" will return this normalized value. The
 *logits* argument will be interpreted as unnormalized log
 probabilities and can therefore be any real number. It will


likewise be normalized so that the resulting probabilities sum to
     1 along the last dimension. "logits" will return this normalized
     value.
See also: "torch.distributions.Categorical()" for specifications of
   "probs" and "logits".
Example:
  >>> m = OneHotCategorical(torch.tensor([ 0.25, 0.25, 0.25, 0.25 ]))
  >>> m.sample()  # equal probability of 0, 1, 2, 3
  tensor([ 0.,  0.,  0.,  1.])

Parameters:
      * probs (Tensor) -- event probabilities
  * **logits** (*Tensor*) -- event log probabilities
    (unnormalized)

arg_constraints = {'logits': IndependentConstraint(Real(), 1), 'probs': Simplex()}
entropy()
enumerate_support(expand=True)
expand(batch_shape, _instance=None)
has_enumerate_support = True
log_prob(value)
property logits
property mean
property mode
property param_shape
property probs
sample(sample_shape=torch.Size([]))
support = OneHot()
property variance
Pareto


property variance
Pareto
class torch.distributions.pareto.Pareto(scale, alpha, validate_args=None)
Bases: "TransformedDistribution"
Samples from a Pareto Type 1 distribution.
Example:
  >>> m = Pareto(torch.tensor([1.0]), torch.tensor([1.0]))
  >>> m.sample()  # sample from a Pareto distribution with scale=1 and alpha=1
  tensor([ 1.5623])

Parameters:
      * scale (float or Tensor) -- Scale parameter of the
        distribution
  * **alpha** (*float** or **Tensor*) -- Shape parameter of the
    distribution

arg_constraints: Dict[str, constraints.Constraint] = {'alpha': GreaterThan(lower_bound=0.0), 'scale': GreaterThan(lower_bound=0.0)}
entropy()
expand(batch_shape, _instance=None)
property mean
property mode
property support
property variance
Poisson
class torch.distributions.poisson.Poisson(rate, validate_args=None)
Bases: "ExponentialFamily"


Bases: "ExponentialFamily"
Creates a Poisson distribution parameterized by "rate", the rate
   parameter.
Samples are nonnegative integers, with a pmf given by
  \mathrm{rate}^k \frac{e^{-\mathrm{rate}}}{k!}

Example:
  >>> m = Poisson(torch.tensor([4]))
  >>> m.sample()
  tensor([ 3.])

Parameters:
      rate (Number, Tensor) -- the rate parameter
arg_constraints = {'rate': GreaterThanEq(lower_bound=0.0)}
expand(batch_shape, _instance=None)
log_prob(value)
property mean
property mode
sample(sample_shape=torch.Size([]))
support = IntegerGreaterThan(lower_bound=0)
property variance
RelaxedBernoulli
class torch.distributions.relaxed_bernoulli.RelaxedBernoulli(temperature, probs=None, logits=None, validate_args=None)
Bases: "TransformedDistribution"
Creates a RelaxedBernoulli distribution, parametrized by
   "temperature", and either "probs" or "logits" (but not both). This


is a relaxed version of the Bernoulli distribution, so the values
   are in (0, 1), and has reparametrizable samples.
Example:
  >>> m = RelaxedBernoulli(torch.tensor([2.2]),
  ...                      torch.tensor([0.1, 0.2, 0.3, 0.99]))
  >>> m.sample()
  tensor([ 0.2951,  0.3442,  0.8918,  0.9021])

Parameters:
      * temperature (Tensor) -- relaxation temperature
  * **probs** (*Number**, **Tensor*) -- the probability of
    sampling *1*

  * **logits** (*Number**, **Tensor*) -- the log-odds of sampling
    *1*

arg_constraints: Dict[str, constraints.Constraint] = {'logits': Real(), 'probs': Interval(lower_bound=0.0, upper_bound=1.0)}
expand(batch_shape, _instance=None)
has_rsample = True
property logits
property probs
support = Interval(lower_bound=0.0, upper_bound=1.0)
property temperature
LogitRelaxedBernoulli


LogitRelaxedBernoulli
class torch.distributions.relaxed_bernoulli.LogitRelaxedBernoulli(temperature, probs=None, logits=None, validate_args=None)
Bases: "Distribution"
Creates a LogitRelaxedBernoulli distribution parameterized by
   "probs" or "logits" (but not both), which is the logit of a
   RelaxedBernoulli distribution.
Samples are logits of values in (0, 1). See [1] for more details.
Parameters:
      * temperature (Tensor) -- relaxation temperature
  * **probs** (*Number**, **Tensor*) -- the probability of
    sampling *1*

  * **logits** (*Number**, **Tensor*) -- the log-odds of sampling
    *1*

[1] The Concrete Distribution: A Continuous Relaxation of Discrete
   Random Variables (Maddison et al, 2017)
[2] Categorical Reparametrization with Gumbel-Softmax (Jang et al,
   2017)
arg_constraints = {'logits': Real(), 'probs': Interval(lower_bound=0.0, upper_bound=1.0)}
expand(batch_shape, _instance=None)
log_prob(value)


log_prob(value)
property logits
property param_shape
property probs
rsample(sample_shape=torch.Size([]))
support = Real()
RelaxedOneHotCategorical
class torch.distributions.relaxed_categorical.RelaxedOneHotCategorical(temperature, probs=None, logits=None, validate_args=None)
Bases: "TransformedDistribution"
Creates a RelaxedOneHotCategorical distribution parametrized by
   "temperature", and either "probs" or "logits". This is a relaxed
   version of the "OneHotCategorical" distribution, so its samples are
   on simplex, and are reparametrizable.
Example:
  >>> m = RelaxedOneHotCategorical(torch.tensor([2.2]),
  ...                              torch.tensor([0.1, 0.2, 0.3, 0.4]))
  >>> m.sample()
  tensor([ 0.1294,  0.2324,  0.3859,  0.2523])

Parameters:
      * temperature (Tensor) -- relaxation temperature
  * **probs** (*Tensor*) -- event probabilities


logits (Tensor) -- unnormalized log probability for each
        event

arg_constraints: Dict[str, constraints.Constraint] = {'logits': IndependentConstraint(Real(), 1), 'probs': Simplex()}
expand(batch_shape, _instance=None)
has_rsample = True
property logits
property probs
support = Simplex()
property temperature
StudentT
class torch.distributions.studentT.StudentT(df, loc=0.0, scale=1.0, validate_args=None)
Bases: "Distribution"
Creates a Student's t-distribution parameterized by degree of
   freedom "df", mean "loc" and scale "scale".
Example:
  >>> m = StudentT(torch.tensor([2.0]))
  >>> m.sample()  # Student's t-distributed with degrees of freedom=2
  tensor([ 0.1046])

Parameters:
      * df (float or Tensor) -- degrees of freedom
  * **loc** (*float** or **Tensor*) -- mean of the distribution

  * **scale** (*float** or **Tensor*) -- scale of the distribution


arg_constraints = {'df': GreaterThan(lower_bound=0.0), 'loc': Real(), 'scale': GreaterThan(lower_bound=0.0)}
entropy()
expand(batch_shape, _instance=None)
has_rsample = True
log_prob(value)
property mean
property mode
rsample(sample_shape=torch.Size([]))
support = Real()
property variance
TransformedDistribution
class torch.distributions.transformed_distribution.TransformedDistribution(base_distribution, transforms, validate_args=None)
Bases: "Distribution"
Extension of the Distribution class, which applies a sequence of
   Transforms to a base distribution.  Let f be the composition of
   transforms applied:
  X ~ BaseDistribution
  Y = f(X) ~ TransformedDistribution(BaseDistribution, f)
  log p(Y) = log p(X) + log |det (dX/dY)|

Note that the ".event_shape" of a "TransformedDistribution" is the
   maximum shape of its base distribution and its transforms, since
   transforms can introduce correlations among events.


An example for the usage of "TransformedDistribution" would be:
  # Building a Logistic Distribution
  # X ~ Uniform(0, 1)
  # f = a + b * logit(X)
  # Y ~ f(X) ~ Logistic(a, b)
  base_distribution = Uniform(0, 1)
  transforms = [SigmoidTransform().inv, AffineTransform(loc=a, scale=b)]
  logistic = TransformedDistribution(base_distribution, transforms)

For more examples, please look at the implementations of "Gumbel",
   "HalfCauchy", "HalfNormal", "LogNormal", "Pareto", "Weibull",
   "RelaxedBernoulli" and "RelaxedOneHotCategorical"
arg_constraints: Dict[str, Constraint] = {}
cdf(value)
  Computes the cumulative distribution function by inverting the
  transform(s) and computing the score of the base distribution.

expand(batch_shape, _instance=None)
property has_rsample
icdf(value)
  Computes the inverse cumulative distribution function using
  transform(s) and computing the score of the base distribution.

log_prob(value)


log_prob(value)
  Scores the sample by inverting the transform(s) and computing
  the score using the score of the base distribution and the log
  abs det jacobian.

rsample(sample_shape=torch.Size([]))
  Generates a sample_shape shaped reparameterized sample or
  sample_shape shaped batch of reparameterized samples if the
  distribution parameters are batched. Samples first from base
  distribution and applies *transform()* for every transform in
  the list.

sample(sample_shape=torch.Size([]))
  Generates a sample_shape shaped sample or sample_shape shaped
  batch of samples if the distribution parameters are batched.
  Samples first from base distribution and applies *transform()*
  for every transform in the list.

property support
Uniform
class torch.distributions.uniform.Uniform(low, high, validate_args=None)
Bases: "Distribution"
Generates uniformly distributed random samples from the half-open
   interval "[low, high)".


interval "[low, high)".
Example:
  >>> m = Uniform(torch.tensor([0.0]), torch.tensor([5.0]))
  >>> m.sample()  # uniformly distributed in the range [0.0, 5.0)
  tensor([ 2.3418])

Parameters:
      * low (float or Tensor) -- lower range (inclusive).
  * **high** (*float** or **Tensor*) -- upper range (exclusive).

arg_constraints = {'high': Dependent(), 'low': Dependent()}
cdf(value)
entropy()
expand(batch_shape, _instance=None)
has_rsample = True
icdf(value)
log_prob(value)
property mean
property mode
rsample(sample_shape=torch.Size([]))
property stddev
property support
property variance
VonMises
class torch.distributions.von_mises.VonMises(loc, concentration, validate_args=None)
Bases: "Distribution"
A circular von Mises distribution.
This implementation uses polar coordinates. The "loc" and "value"
   args can be any real number (to facilitate unconstrained


optimization), but are interpreted as angles modulo 2 pi.
Example::
      >>> m = VonMises(torch.tensor([1.0]), torch.tensor([1.0]))
      >>> m.sample()  # von Mises distributed with loc=1 and concentration=1
      tensor([1.9777])
Parameters:
      * loc (torch.Tensor) -- an angle in radians.
  * **concentration** (*torch.Tensor*) -- concentration parameter

arg_constraints = {'concentration': GreaterThan(lower_bound=0.0), 'loc': Real()}
expand(batch_shape)
has_rsample = False
log_prob(value)
property mean
  The provided mean is the circular one.

property mode
sample(sample_shape=torch.Size([]))
  The sampling algorithm for the von Mises distribution is based
  on the following paper: Best, D. J., and Nicholas I. Fisher.
  "Efficient simulation of the von Mises distribution." Applied
  Statistics (1979): 152-157.

support = Real()
property variance
  The provided variance is the circular one.

Weibull


Weibull
class torch.distributions.weibull.Weibull(scale, concentration, validate_args=None)
Bases: "TransformedDistribution"
Samples from a two-parameter Weibull distribution.
-[ Example ]-

m = Weibull(torch.tensor([1.0]), torch.tensor([1.0]))
m.sample()  # sample from a Weibull distribution with scale=1, concentration=1
   tensor([ 0.4784])

Parameters:
      * scale (float or Tensor) -- Scale parameter of
        distribution (lambda).
  * **concentration** (*float** or **Tensor*) -- Concentration
    parameter of distribution (k/shape).

arg_constraints: Dict[str, constraints.Constraint] = {'concentration': GreaterThan(lower_bound=0.0), 'scale': GreaterThan(lower_bound=0.0)}
entropy()
expand(batch_shape, _instance=None)
property mean
property mode
support = GreaterThan(lower_bound=0.0)
property variance
Wishart


property variance
Wishart
class torch.distributions.wishart.Wishart(df, covariance_matrix=None, precision_matrix=None, scale_tril=None, validate_args=None)
Bases: "ExponentialFamily"
Creates a Wishart distribution parameterized by a symmetric
   positive definite matrix \Sigma, or its Cholesky decomposition
   \mathbf{\Sigma} = \mathbf{L}\mathbf{L}^\top
-[ Example ]-

m = Wishart(torch.eye(2), torch.Tensor([2]))
m.sample()  # Wishart distributed with mean=df * I and
            # variance(x_ij)=df for i != j and variance(x_ij)=2 * df for i == j

Parameters:
      * covariance_matrix (Tensor) -- positive-definite
        covariance matrix
  * **precision_matrix** (*Tensor*) -- positive-definite precision
    matrix

  * **scale_tril** (*Tensor*) -- lower-triangular factor of
    covariance, with positive-valued diagonal

  * **df** (*float** or **Tensor*) -- real-valued parameter larger


than the (dimension of Square matrix) - 1
Note:
 Only one of "covariance_matrix" or "precision_matrix" or
 "scale_tril" can be specified. Using "scale_tril" will be more
 efficient: all computations internally are based on "scale_tril".
 If "covariance_matrix" or "precision_matrix" is passed instead,
 it is only used to compute the corresponding lower triangular
 matrices using a Cholesky decomposition.
 'torch.distributions.LKJCholesky' is a restricted Wishart
 distribution.[1]

References
[1] Wang, Z., Wu, Y. and Chu, H., 2018. On equivalence of the LKJ
   distribution and the restricted Wishart distribution. [2] Sawyer,
   S., 2007. Wishart Distributions and Inverse-Wishart Sampling. [3]
   Anderson, T. W., 2003. An Introduction to Multivariate Statistical
   Analysis (3rd ed.). [4] Odell, P. L. & Feiveson, A. H., 1966. A
   Numerical Procedure to Generate a SampleCovariance Matrix. JASA,


61(313):199-203. [5] Ku, Y.-C. & Bloomfield, P., 2010. Generating
   Random Wishart Matrices with Fractional Degrees of Freedom in OX.
arg_constraints = {'covariance_matrix': PositiveDefinite(), 'df': GreaterThan(lower_bound=0), 'precision_matrix': PositiveDefinite(), 'scale_tril': LowerCholesky()}
property covariance_matrix
entropy()
expand(batch_shape, _instance=None)
has_rsample = True
log_prob(value)
property mean
property mode
property precision_matrix
rsample(sample_shape=torch.Size([]), max_try_correction=None)
  Warning:

    In some cases, sampling algorithm based on Bartlett
    decomposition may return singular matrix samples. Several
    tries to correct singular samples are performed by default,
    but it may end up returning singular matrix samples. Singular
    samples may return *-inf* values in *.log_prob()*. In those
    cases, the user should validate the samples and either fix the


value of df or adjust max_try_correction value for
        argument in .rsample accordingly.
property scale_tril
support = PositiveDefinite()
property variance
KL Divergence
torch.distributions.kl.kl_divergence(p, q)
Compute Kullback-Leibler divergence KL(p | q) between two
   distributions.
  KL(p \| q) = \int p(x) \log\frac {p(x)} {q(x)} \,dx

Parameters:
      * p (Distribution) -- A "Distribution" object.
  * **q** (*Distribution*) -- A "Distribution" object.

Returns:
      A batch of KL divergences of shape batch_shape.
Return type:
      Tensor
Raises:
      NotImplementedError -- If the distribution types have not
      been registered via     "register_kl()".
KL divergence is currently implemented for the following
   distribution pairs:
      * "Bernoulli" and "Bernoulli"
  * "Bernoulli" and "Poisson"

  * "Beta" and "Beta"

  * "Beta" and "ContinuousBernoulli"

  * "Beta" and "Exponential"


"Beta" and "Exponential"

"Beta" and "Gamma"

"Beta" and "Normal"

"Beta" and "Pareto"

"Beta" and "Uniform"

"Binomial" and "Binomial"

"Categorical" and "Categorical"

"Cauchy" and "Cauchy"

"ContinuousBernoulli" and "ContinuousBernoulli"

"ContinuousBernoulli" and "Exponential"

"ContinuousBernoulli" and "Normal"

"ContinuousBernoulli" and "Pareto"

"ContinuousBernoulli" and "Uniform"

"Dirichlet" and "Dirichlet"

"Exponential" and "Beta"

"Exponential" and "ContinuousBernoulli"

"Exponential" and "Exponential"

"Exponential" and "Gamma"

"Exponential" and "Gumbel"

"Exponential" and "Normal"

"Exponential" and "Pareto"

"Exponential" and "Uniform"

"ExponentialFamily" and "ExponentialFamily"

"Gamma" and "Beta"

"Gamma" and "ContinuousBernoulli"

"Gamma" and "Exponential"

"Gamma" and "Gamma"


"Gamma" and "Gamma"

"Gamma" and "Gumbel"

"Gamma" and "Normal"

"Gamma" and "Pareto"

"Gamma" and "Uniform"

"Geometric" and "Geometric"

"Gumbel" and "Beta"

"Gumbel" and "ContinuousBernoulli"

"Gumbel" and "Exponential"

"Gumbel" and "Gamma"

"Gumbel" and "Gumbel"

"Gumbel" and "Normal"

"Gumbel" and "Pareto"

"Gumbel" and "Uniform"

"HalfNormal" and "HalfNormal"

"Independent" and "Independent"

"Laplace" and "Beta"

"Laplace" and "ContinuousBernoulli"

"Laplace" and "Exponential"

"Laplace" and "Gamma"

"Laplace" and "Laplace"

"Laplace" and "Normal"

"Laplace" and "Pareto"

"Laplace" and "Uniform"

"LowRankMultivariateNormal" and "LowRankMultivariateNormal"

"LowRankMultivariateNormal" and "MultivariateNormal"

"MultivariateNormal" and "LowRankMultivariateNormal"


"MultivariateNormal" and "MultivariateNormal"

"Normal" and "Beta"

"Normal" and "ContinuousBernoulli"

"Normal" and "Exponential"

"Normal" and "Gamma"

"Normal" and "Gumbel"

"Normal" and "Laplace"

"Normal" and "Normal"

"Normal" and "Pareto"

"Normal" and "Uniform"

"OneHotCategorical" and "OneHotCategorical"

"Pareto" and "Beta"

"Pareto" and "ContinuousBernoulli"

"Pareto" and "Exponential"

"Pareto" and "Gamma"

"Pareto" and "Normal"

"Pareto" and "Pareto"

"Pareto" and "Uniform"

"Poisson" and "Bernoulli"

"Poisson" and "Binomial"

"Poisson" and "Poisson"

"TransformedDistribution" and "TransformedDistribution"

"Uniform" and "Beta"

"Uniform" and "ContinuousBernoulli"

"Uniform" and "Exponential"

"Uniform" and "Gamma"

"Uniform" and "Gumbel"

"Uniform" and "Normal"


"Uniform" and "Normal"

"Uniform" and "Pareto"

"Uniform" and "Uniform"

torch.distributions.kl.register_kl(type_p, type_q)
Decorator to register a pairwise function with "kl_divergence()".
   Usage:
  @register_kl(Normal, Normal)
  def kl_normal_normal(p, q):
      # insert implementation here

Lookup returns the most specific (type,type) match ordered by
   subclass. If the match is ambiguous, a RuntimeWarning is raised.
   For example to resolve the ambiguous situation:
  @register_kl(BaseP, DerivedQ)
  def kl_version1(p, q): ...
  @register_kl(DerivedP, BaseQ)
  def kl_version2(p, q): ...

you should register a third most-specific implementation, e.g.:
  register_kl(DerivedP, DerivedQ)(kl_version1)  # Break the tie.

Parameters:
      * type_p (type) -- A subclass of "Distribution".
  * **type_q** (*type*) -- A subclass of "Distribution".

Transforms


Transforms
class torch.distributions.transforms.AbsTransform(cache_size=0)
Transform via the mapping y = |x|.
class torch.distributions.transforms.AffineTransform(loc, scale, event_dim=0, cache_size=0)
Transform via the pointwise affine mapping y = \text{loc} +
   \text{scale} \times x.
Parameters:
      * loc (Tensor or float) -- Location parameter.
  * **scale** (*Tensor** or **float*) -- Scale parameter.

  * **event_dim** (*int*) -- Optional size of *event_shape*. This
    should be zero for univariate random variables, 1 for
    distributions over vectors, 2 for distributions over matrices,
    etc.

class torch.distributions.transforms.CatTransform(tseq, dim=0, lengths=None, cache_size=0)
Transform functor that applies a sequence of transforms tseq
   component-wise to each submatrix at dim, of length
   lengths[dim], in a way compatible with "torch.cat()".
Example:


Example:
  x0 = torch.cat([torch.range(1, 10), torch.range(1, 10)], dim=0)
  x = torch.cat([x0, x0], dim=0)
  t0 = CatTransform([ExpTransform(), identity_transform], dim=0, lengths=[10, 10])
  t = CatTransform([t0, t0], dim=0, lengths=[20, 20])
  y = t(x)

class torch.distributions.transforms.ComposeTransform(parts, cache_size=0)
Composes multiple transforms in a chain. The transforms being
   composed are responsible for caching.
Parameters:
      * parts (list of "Transform") -- A list of transforms to
        compose.
  * **cache_size** (*int*) -- Size of cache. If zero, no caching
    is done. If one, the latest single value is cached. Only 0 and
    1 are supported.

class torch.distributions.transforms.CorrCholeskyTransform(cache_size=0)
Transforms an uncontrained real vector x with length D*(D-1)/2 into
   the Cholesky factor of a D-dimension correlation matrix. This
   Cholesky factor is a lower triangular matrix with positive


diagonals and unit Euclidean norm for each row. The transform is
   processed as follows:
  1. First we convert x into a lower triangular matrix in row
     order.

  2. For each row X_i of the lower triangular part, we apply a
     *signed* version of class "StickBreakingTransform" to
     transform X_i into a unit Euclidean length vector using the
     following steps: - Scales into the interval (-1, 1) domain:
     r_i = \tanh(X_i). - Transforms into an unsigned domain: z_i =
     r_i^2. - Applies s_i = StickBreakingTransform(z_i). -
     Transforms back into signed domain: y_i = sign(r_i) *
     \sqrt{s_i}.

class torch.distributions.transforms.CumulativeDistributionTransform(distribution, cache_size=0)
Transform via the cumulative distribution function of a probability
   distribution.
Parameters:
      distribution (Distribution) -- Distribution whose
      cumulative distribution function to use for the transformation.
Example:


Example:
  # Construct a Gaussian copula from a multivariate normal.
  base_dist = MultivariateNormal(
      loc=torch.zeros(2),
      scale_tril=LKJCholesky(2).sample(),
  )
  transform = CumulativeDistributionTransform(Normal(0, 1))
  copula = TransformedDistribution(base_dist, [transform])

class torch.distributions.transforms.ExpTransform(cache_size=0)
Transform via the mapping y = \exp(x).
class torch.distributions.transforms.IndependentTransform(base_transform, reinterpreted_batch_ndims, cache_size=0)
Wrapper around another transform to treat
   "reinterpreted_batch_ndims"-many extra of the right most dimensions
   as dependent. This has no effect on the forward or backward
   transforms, but does sum out "reinterpreted_batch_ndims"-many of
   the rightmost dimensions in "log_abs_det_jacobian()".
Parameters:
      * base_transform ("Transform") -- A base transform.
  * **reinterpreted_batch_ndims** (*int*) -- The number of extra


rightmost dimensions to treat as dependent.
class torch.distributions.transforms.LowerCholeskyTransform(cache_size=0)
Transform from unconstrained matrices to lower-triangular matrices
   with nonnegative diagonal entries.
This is useful for parameterizing positive definite matrices in
   terms of their Cholesky factorization.
class torch.distributions.transforms.PositiveDefiniteTransform(cache_size=0)
Transform from unconstrained matrices to positive-definite
   matrices.
class torch.distributions.transforms.PowerTransform(exponent, cache_size=0)
Transform via the mapping y = x^{\text{exponent}}.
class torch.distributions.transforms.ReshapeTransform(in_shape, out_shape, cache_size=0)
Unit Jacobian transform to reshape the rightmost part of a tensor.
Note that "in_shape" and "out_shape" must have the same number of
   elements, just as for "torch.Tensor.reshape()".
Parameters:
      * in_shape (torch.Size) -- The input event shape.


out_shape (torch.Size) -- The output event shape.

class torch.distributions.transforms.SigmoidTransform(cache_size=0)
Transform via the mapping y = \frac{1}{1 + \exp(-x)} and x =
   \text{logit}(y).
class torch.distributions.transforms.SoftplusTransform(cache_size=0)
Transform via the mapping \text{Softplus}(x) = \log(1 + \exp(x)).
   The implementation reverts to the linear function when x > 20.
class torch.distributions.transforms.TanhTransform(cache_size=0)
Transform via the mapping y = \tanh(x).
It is equivalent to "ComposeTransform([AffineTransform(0., 2.),
   SigmoidTransform(), AffineTransform(-1., 2.)])" However this
   might not be numerically stable, thus it is recommended to use
   TanhTransform instead.
Note that one should use cache_size=1 when it comes to NaN/Inf
   values.
class torch.distributions.transforms.SoftmaxTransform(cache_size=0)
Transform from unconstrained space to the simplex via y = \exp(x)
   then normalizing.


then normalizing.
This is not bijective and cannot be used for HMC. However this acts
   mostly coordinate-wise (except for the final normalization), and
   thus is appropriate for coordinate-wise optimization algorithms.
class torch.distributions.transforms.StackTransform(tseq, dim=0, cache_size=0)
Transform functor that applies a sequence of transforms tseq
   component-wise to each submatrix at dim in a way compatible with
   "torch.stack()".
Example:
  x = torch.stack([torch.range(1, 10), torch.range(1, 10)], dim=1)
  t = StackTransform([ExpTransform(), identity_transform], dim=1)
  y = t(x)

class torch.distributions.transforms.StickBreakingTransform(cache_size=0)
Transform from unconstrained space to the simplex of one additional
   dimension via a stick-breaking process.
This transform arises as an iterated sigmoid transform in a stick-
   breaking construction of the Dirichlet distribution: the first


logit is transformed via sigmoid to the first probability and the
   probability of everything else, and then the process recurses.
This is bijective and appropriate for use in HMC; however it mixes
   coordinates together and is less appropriate for optimization.
class torch.distributions.transforms.Transform(cache_size=0)
Abstract class for invertable transformations with computable log
   det jacobians. They are primarily used in
   "torch.distributions.TransformedDistribution".
Caching is useful for transforms whose inverses are either
   expensive or numerically unstable. Note that care must be taken
   with memoized values since the autograd graph may be reversed. For
   example while the following works with or without caching:
  y = t(x)
  t.log_abs_det_jacobian(x, y).backward()  # x will receive gradients.

However the following will error when caching due to dependency
   reversal:
  y = t(x)
  z = t.inv(y)
  grad(z.sum(), [y])  # error because z is x


grad(z.sum(), [y])  # error because z is x
Derived classes should implement one or both of "_call()" or
   "_inverse()". Derived classes that set bijective=True should also
   implement "log_abs_det_jacobian()".
Parameters:
      cache_size (int) -- Size of cache. If zero, no caching is
      done. If one, the latest single value is cached. Only 0 and 1
      are supported.
Variables:
      * domain ("Constraint") -- The constraint representing valid
        inputs to this transform.
  * **codomain** ("Constraint") -- The constraint representing
    valid outputs to this transform which are inputs to the
    inverse transform.

  * **bijective** (*bool*) -- Whether this transform is bijective.
    A transform "t" is bijective iff "t.inv(t(x)) == x" and
    "t(t.inv(y)) == y" for every "x" in the domain and "y" in the
    codomain. Transforms that are not bijective should at least
    maintain the weaker pseudoinverse properties "t(t.inv(t(x)) ==


t(x)" and "t.inv(t(t.inv(y))) == t.inv(y)".
  * **sign** (*int** or **Tensor*) -- For bijective univariate
    transforms, this should be +1 or -1 depending on whether
    transform is monotone increasing or decreasing.

property inv
  Returns the inverse "Transform" of this transform. This should
  satisfy "t.inv.inv is t".

property sign
  Returns the sign of the determinant of the Jacobian, if
  applicable. In general this only makes sense for bijective
  transforms.

log_abs_det_jacobian(x, y)
  Computes the log det jacobian *log |dy/dx|* given input and
  output.

forward_shape(shape)
  Infers the shape of the forward computation, given the input
  shape. Defaults to preserving shape.

inverse_shape(shape)
  Infers the shapes of the inverse computation, given the output
  shape. Defaults to preserving shape.

Constraints
The following constraints are implemented:

"constraints.boolean"


"constraints.boolean"

"constraints.cat"

"constraints.corr_cholesky"

"constraints.dependent"

"constraints.greater_than(lower_bound)"

"constraints.greater_than_eq(lower_bound)"

"constraints.independent(constraint, reinterpreted_batch_ndims)"

"constraints.integer_interval(lower_bound, upper_bound)"

"constraints.interval(lower_bound, upper_bound)"

"constraints.less_than(upper_bound)"

"constraints.lower_cholesky"

"constraints.lower_triangular"

"constraints.multinomial"

"constraints.nonnegative_integer"

"constraints.one_hot"

"constraints.positive_integer"

"constraints.positive"

"constraints.positive_semidefinite"

"constraints.positive_definite"

"constraints.real_vector"

"constraints.real"

"constraints.simplex"

"constraints.symmetric"

"constraints.stack"

"constraints.square"

"constraints.symmetric"

"constraints.unit_interval"

class torch.distributions.constraints.Constraint
Abstract base class for constraints.


Abstract base class for constraints.
A constraint object represents a region over which a variable is
   valid, e.g. within which a variable can be optimized.
Variables:
      * is_discrete (bool) -- Whether constrained space is
        discrete. Defaults to False.
  * **event_dim** (*int*) -- Number of rightmost dimensions that
    together define an event. The "check()" method will remove
    this many dimensions when computing validity.

check(value)
  Returns a byte tensor of "sample_shape + batch_shape" indicating
  whether each event in value satisfies this constraint.

torch.distributions.constraints.cat
alias of "_Cat"
torch.distributions.constraints.dependent_property
alias of "_DependentProperty"
torch.distributions.constraints.greater_than
alias of "_GreaterThan"
torch.distributions.constraints.greater_than_eq
alias of "_GreaterThanEq"
torch.distributions.constraints.independent
alias of "_IndependentConstraint"


alias of "_IndependentConstraint"
torch.distributions.constraints.integer_interval
alias of "_IntegerInterval"
torch.distributions.constraints.interval
alias of "_Interval"
torch.distributions.constraints.half_open_interval
alias of "_HalfOpenInterval"
torch.distributions.constraints.less_than
alias of "_LessThan"
torch.distributions.constraints.multinomial
alias of "_Multinomial"
torch.distributions.constraints.stack
alias of "_Stack"
Constraint Registry
PyTorch provides two global "ConstraintRegistry" objects that link
"Constraint" objects to "Transform" objects. These objects both input
constraints and return transforms, but they have different guarantees
on bijectivity.

"biject_to(constraint)" looks up a bijective "Transform" from
   "constraints.real" to the given "constraint". The returned
   transform is guaranteed to have ".bijective = True" and should
   implement ".log_abs_det_jacobian()".


implement ".log_abs_det_jacobian()".

"transform_to(constraint)" looks up a not-necessarily bijective
   "Transform" from "constraints.real" to the given "constraint". The
   returned transform is not guaranteed to implement
   ".log_abs_det_jacobian()".

The "transform_to()" registry is useful for performing unconstrained
optimization on constrained parameters of probability distributions,
which are indicated by each distribution's ".arg_constraints" dict.
These transforms often overparameterize a space in order to avoid
rotation; they are thus more suitable for coordinate-wise optimization
algorithms like Adam:
loc = torch.zeros(100, requires_grad=True)
   unconstrained = torch.zeros(100, requires_grad=True)
   scale = transform_to(Normal.arg_constraints['scale'])(unconstrained)
   loss = -Normal(loc, scale).log_prob(data).sum()
The "biject_to()" registry is useful for Hamiltonian Monte Carlo,
where samples from a probability distribution with constrained


".support" are propagated in an unconstrained space, and algorithms
are typically rotation invariant.:
dist = Exponential(rate)
   unconstrained = torch.zeros(100, requires_grad=True)
   sample = biject_to(dist.support)(unconstrained)
   potential_energy = -dist.log_prob(sample).sum()
Note:
An example where "transform_to" and "biject_to" differ is
  "constraints.simplex": "transform_to(constraints.simplex)" returns a
  "SoftmaxTransform" that simply exponentiates and normalizes its
  inputs; this is a cheap and mostly coordinate-wise operation
  appropriate for algorithms like SVI. In contrast,
  "biject_to(constraints.simplex)" returns a "StickBreakingTransform"
  that bijects its input down to a one-fewer-dimensional space; this a
  more expensive less numerically stable transform but is needed for
  algorithms like HMC.
The "biject_to" and "transform_to" objects can be extended by user-
defined constraints and transforms using their ".register()" method


either as a function on singleton constraints:
transform_to.register(my_constraint, my_transform)
or as a decorator on parameterized constraints:
@transform_to.register(MyConstraintClass)
   def my_factory(constraint):
       assert isinstance(constraint, MyConstraintClass)
       return MyTransform(constraint.param1, constraint.param2)
You can create your own registry by creating a new
"ConstraintRegistry" object.
class torch.distributions.constraint_registry.ConstraintRegistry
Registry to link constraints to transforms.
register(constraint, factory=None)
  Registers a "Constraint" subclass in this registry. Usage:

     @my_registry.register(MyConstraintClass)
     def construct_transform(constraint):
         assert isinstance(constraint, MyConstraint)
         return MyTransform(constraint.arg_constraints)

  Parameters:
     * **constraint** (subclass of "Constraint") -- A subclass of
       "Constraint", or a singleton object of the desired class.


factory (Callable) -- A callable that inputs a
           constraint object and returns a  "Transform" object.


Named Tensors operator coverage
Please read Named Tensors first for an introduction to named tensors.
This document is a reference for name inference, a process that
defines how named tensors:

use names to provide additional automatic runtime correctness
   checks

propagate names from input tensors to output tensors

Below is a list of all operations that are supported with named
tensors and their associated name inference rules.
If you don't see an operation listed here, but it would help your use
case, please search if an issue has already been filed and if not,
file one.
Warning:
The named tensor API is experimental and subject to change.
Supported Operations
^^^^^^^^^^^^^^^^^^^^
+----------------------+----------------------+
| API                  | Name inference rule  |
|======================|======================|
| "Tensor.abs()",      | Keeps input names    |
| "torch.abs()"        |                      |


| "torch.abs()"        |                      |
+----------------------+----------------------+
| "Tensor.abs_()"      | Keeps input names    |
+----------------------+----------------------+
| "Tensor.acos()",     | Keeps input names    |
| "torch.acos()"       |                      |
+----------------------+----------------------+
| "Tensor.acos_()"     | Keeps input names    |
+----------------------+----------------------+
| "Tensor.add()",      | Unifies names from   |
| "torch.add()"        | inputs               |
+----------------------+----------------------+
| "Tensor.add_()"      | Unifies names from   |
|                      | inputs               |
+----------------------+----------------------+
| "Tensor.addmm()",    | Contracts away dims  |
| "torch.addmm()"      |                      |
+----------------------+----------------------+
| "Tensor.addmm_()"    | Contracts away dims  |
+----------------------+----------------------+
| "Tensor.addmv()",    | Contracts away dims  |


| "Tensor.addmv()",    | Contracts away dims  |
| "torch.addmv()"      |                      |
+----------------------+----------------------+
| "Tensor.addmv_()"    | Contracts away dims  |
+----------------------+----------------------+
| "Tensor.align_as()"  | See documentation    |
+----------------------+----------------------+
| "Tensor.align_to()"  | See documentation    |
+----------------------+----------------------+
| "Tensor.all()",      | None                 |
| "torch.all()"        |                      |
+----------------------+----------------------+
| "Tensor.any()",      | None                 |
| "torch.any()"        |                      |
+----------------------+----------------------+
| "Tensor.asin()",     | Keeps input names    |
| "torch.asin()"       |                      |
+----------------------+----------------------+
| "Tensor.asin_()"     | Keeps input names    |
+----------------------+----------------------+
| "Tensor.atan()",     | Keeps input names    |


| "Tensor.atan()",     | Keeps input names    |
| "torch.atan()"       |                      |
+----------------------+----------------------+
| "Tensor.atan2()",    | Unifies names from   |
| "torch.atan2()"      | inputs               |
+----------------------+----------------------+
| "Tensor.atan2_()"    | Unifies names from   |
|                      | inputs               |
+----------------------+----------------------+
| "Tensor.atan_()"     | Keeps input names    |
+----------------------+----------------------+
| "Tensor.bernoulli()  | Keeps input names    |
| ",                   |                      |
| "torch.bernoulli()"  |                      |
+----------------------+----------------------+
| "Tensor.bernoulli_(  | None                 |
| )"                   |                      |
+----------------------+----------------------+
| "Tensor.bfloat16()"  | Keeps input names    |
+----------------------+----------------------+
| "Tensor.bitwise_not  | Keeps input names    |


| "Tensor.bitwise_not  | Keeps input names    |
| ()", "torch.bitwise  |                      |
| not()"              |                      |
+----------------------+----------------------+
| "Tensor.bitwise_not  | None                 |
| ()"                 |                      |
+----------------------+----------------------+
| "Tensor.bmm()",      | Contracts away dims  |
| "torch.bmm()"        |                      |
+----------------------+----------------------+
| "Tensor.bool()"      | Keeps input names    |
+----------------------+----------------------+
| "Tensor.byte()"      | Keeps input names    |
+----------------------+----------------------+
| "torch.cat()"        | Unifies names from   |
|                      | inputs               |
+----------------------+----------------------+
| "Tensor.cauchy_()"   | None                 |
+----------------------+----------------------+
| "Tensor.ceil()",     | Keeps input names    |
| "torch.ceil()"       |                      |


| "torch.ceil()"       |                      |
+----------------------+----------------------+
| "Tensor.ceil_()"     | None                 |
+----------------------+----------------------+
| "Tensor.char()"      | Keeps input names    |
+----------------------+----------------------+
| "Tensor.chunk()",    | Keeps input names    |
| "torch.chunk()"      |                      |
+----------------------+----------------------+
| "Tensor.clamp()",    | Keeps input names    |
| "torch.clamp()"      |                      |
+----------------------+----------------------+
| "Tensor.clamp_()"    | None                 |
+----------------------+----------------------+
| "Tensor.copy_()"     | out function and in- |
|                      | place variants       |
+----------------------+----------------------+
| "Tensor.cos()",      | Keeps input names    |
| "torch.cos()"        |                      |
+----------------------+----------------------+
| "Tensor.cos_()"      | None                 |


| "Tensor.cos_()"      | None                 |
+----------------------+----------------------+
| "Tensor.cosh()",     | Keeps input names    |
| "torch.cosh()"       |                      |
+----------------------+----------------------+
| "Tensor.cosh_()"     | None                 |
+----------------------+----------------------+
| "Tensor.acosh()",    | Keeps input names    |
| "torch.acosh()"      |                      |
+----------------------+----------------------+
| "Tensor.acosh_()"    | None                 |
+----------------------+----------------------+
| "Tensor.cpu()"       | Keeps input names    |
+----------------------+----------------------+
| "Tensor.cuda()"      | Keeps input names    |
+----------------------+----------------------+
| "Tensor.cumprod()",  | Keeps input names    |
| "torch.cumprod()"    |                      |
+----------------------+----------------------+
| "Tensor.cumsum()",   | Keeps input names    |
| "torch.cumsum()"     |                      |


| "torch.cumsum()"     |                      |
+----------------------+----------------------+
| "Tensor.data_ptr()"  | None                 |
+----------------------+----------------------+
| "Tensor.deg2rad()",  | Keeps input names    |
| "torch.deg2rad()"    |                      |
+----------------------+----------------------+
| "Tensor.deg2rad_()"  | None                 |
+----------------------+----------------------+
| "Tensor.detach()",   | Keeps input names    |
| "torch.detach()"     |                      |
+----------------------+----------------------+
| "Tensor.detach_()"   | None                 |
+----------------------+----------------------+
| "Tensor.device",     | None                 |
| "torch.device()"     |                      |
+----------------------+----------------------+
| "Tensor.digamma()",  | Keeps input names    |
| "torch.digamma()"    |                      |
+----------------------+----------------------+
| "Tensor.digamma_()"  | None                 |


| "Tensor.digamma_()"  | None                 |
+----------------------+----------------------+
| "Tensor.dim()"       | None                 |
+----------------------+----------------------+
| "Tensor.div()",      | Unifies names from   |
| "torch.div()"        | inputs               |
+----------------------+----------------------+
| "Tensor.div_()"      | Unifies names from   |
|                      | inputs               |
+----------------------+----------------------+
| "Tensor.dot()",      | None                 |
| "torch.dot()"        |                      |
+----------------------+----------------------+
| "Tensor.double()"    | Keeps input names    |
+----------------------+----------------------+
| "Tensor.element_siz  | None                 |
| e()"                 |                      |
+----------------------+----------------------+
| "torch.empty()"      | Factory functions    |
+----------------------+----------------------+
| "torch.empty_like()" | Factory functions    |


| "torch.empty_like()" | Factory functions    |
+----------------------+----------------------+
| "Tensor.eq()",       | Unifies names from   |
| "torch.eq()"         | inputs               |
+----------------------+----------------------+
| "Tensor.erf()",      | Keeps input names    |
| "torch.erf()"        |                      |
+----------------------+----------------------+
| "Tensor.erf_()"      | None                 |
+----------------------+----------------------+
| "Tensor.erfc()",     | Keeps input names    |
| "torch.erfc()"       |                      |
+----------------------+----------------------+
| "Tensor.erfc_()"     | None                 |
+----------------------+----------------------+
| "Tensor.erfinv()",   | Keeps input names    |
| "torch.erfinv()"     |                      |
+----------------------+----------------------+
| "Tensor.erfinv_()"   | None                 |
+----------------------+----------------------+
| "Tensor.exp()",      | Keeps input names    |


| "Tensor.exp()",      | Keeps input names    |
| "torch.exp()"        |                      |
+----------------------+----------------------+
| "Tensor.exp_()"      | None                 |
+----------------------+----------------------+
| "Tensor.expand()"    | Keeps input names    |
+----------------------+----------------------+
| "Tensor.expm1()",    | Keeps input names    |
| "torch.expm1()"      |                      |
+----------------------+----------------------+
| "Tensor.expm1_()"    | None                 |
+----------------------+----------------------+
| "Tensor.exponential  | None                 |
| ()"                 |                      |
+----------------------+----------------------+
| "Tensor.fill()"     | None                 |
+----------------------+----------------------+
| "Tensor.flatten()",  | See documentation    |
| "torch.flatten()"    |                      |
+----------------------+----------------------+
| "Tensor.float()"     | Keeps input names    |


| "Tensor.float()"     | Keeps input names    |
+----------------------+----------------------+
| "Tensor.floor()",    | Keeps input names    |
| "torch.floor()"      |                      |
+----------------------+----------------------+
| "Tensor.floor_()"    | None                 |
+----------------------+----------------------+
| "Tensor.frac()",     | Keeps input names    |
| "torch.frac()"       |                      |
+----------------------+----------------------+
| "Tensor.frac_()"     | None                 |
+----------------------+----------------------+
| "Tensor.ge()",       | Unifies names from   |
| "torch.ge()"         | inputs               |
+----------------------+----------------------+
| "Tensor.get_device(  | None                 |
| )",                  |                      |
| "torch.get_device()" |                      |
+----------------------+----------------------+
| "Tensor.grad"        | None                 |
+----------------------+----------------------+


+----------------------+----------------------+
| "Tensor.gt()",       | Unifies names from   |
| "torch.gt()"         | inputs               |
+----------------------+----------------------+
| "Tensor.half()"      | Keeps input names    |
+----------------------+----------------------+
| "Tensor.has_names()" | See documentation    |
+----------------------+----------------------+
| "Tensor.index_fill(  | Keeps input names    |
| )",                  |                      |
| "torch.index_fill()" |                      |
+----------------------+----------------------+
| "Tensor.index_fill_  | None                 |
| ()"                  |                      |
+----------------------+----------------------+
| "Tensor.int()"       | Keeps input names    |
+----------------------+----------------------+
| "Tensor.is_contiguo  | None                 |
| us()"                |                      |
+----------------------+----------------------+
| "Tensor.is_cuda"     | None                 |


| "Tensor.is_cuda"     | None                 |
+----------------------+----------------------+
| "Tensor.is_floating  | None                 |
| _point()", "torch.i  |                      |
| s_floating_point()"  |                      |
+----------------------+----------------------+
| "Tensor.is_leaf"     | None                 |
+----------------------+----------------------+
| "Tensor.is_pinned()" | None                 |
+----------------------+----------------------+
| "Tensor.is_shared()" | None                 |
+----------------------+----------------------+
| "Tensor.is_signed()  | None                 |
| ",                   |                      |
| "torch.is_signed()"  |                      |
+----------------------+----------------------+
| "Tensor.is_sparse"   | None                 |
+----------------------+----------------------+
| "Tensor.is_sparse_c  | None                 |
| sr"                  |                      |
+----------------------+----------------------+


+----------------------+----------------------+
| "torch.is_tensor()"  | None                 |
+----------------------+----------------------+
| "Tensor.item()"      | None                 |
+----------------------+----------------------+
| "Tensor.kthvalue()", | Removes dimensions   |
| "torch.kthvalue()"   |                      |
+----------------------+----------------------+
| "Tensor.le()",       | Unifies names from   |
| "torch.le()"         | inputs               |
+----------------------+----------------------+
| "Tensor.log()",      | Keeps input names    |
| "torch.log()"        |                      |
+----------------------+----------------------+
| "Tensor.log10()",    | Keeps input names    |
| "torch.log10()"      |                      |
+----------------------+----------------------+
| "Tensor.log10_()"    | None                 |
+----------------------+----------------------+
| "Tensor.log1p()",    | Keeps input names    |
| "torch.log1p()"      |                      |


| "torch.log1p()"      |                      |
+----------------------+----------------------+
| "Tensor.log1p_()"    | None                 |
+----------------------+----------------------+
| "Tensor.log2()",     | Keeps input names    |
| "torch.log2()"       |                      |
+----------------------+----------------------+
| "Tensor.log2_()"     | None                 |
+----------------------+----------------------+
| "Tensor.log_()"      | None                 |
+----------------------+----------------------+
| "Tensor.log_normal_  | None                 |
| ()"                  |                      |
+----------------------+----------------------+
| "Tensor.logical_not  | Keeps input names    |
| ()", "torch.logical  |                      |
| not()"              |                      |
+----------------------+----------------------+
| "Tensor.logical_not  | None                 |
| ()"                 |                      |
+----------------------+----------------------+


+----------------------+----------------------+
| "Tensor.logsumexp()  | Removes dimensions   |
| ",                   |                      |
| "torch.logsumexp()"  |                      |
+----------------------+----------------------+
| "Tensor.long()"      | Keeps input names    |
+----------------------+----------------------+
| "Tensor.lt()",       | Unifies names from   |
| "torch.lt()"         | inputs               |
+----------------------+----------------------+
| "torch.manual_seed(  | None                 |
| )"                   |                      |
+----------------------+----------------------+
| "Tensor.masked_fill  | Keeps input names    |
| ()", "torch.masked_  |                      |
| fill()"              |                      |
+----------------------+----------------------+
| "Tensor.masked_fill  | None                 |
| _()"                 |                      |
+----------------------+----------------------+
| "Tensor.masked_sele  | Aligns mask up to    |


| "Tensor.masked_sele  | Aligns mask up to    |
| ct()", "torch.maske  | input and then unif  |
| d_select()"          | ies_names_from_inpu  |
|                      | t_tensors            |
+----------------------+----------------------+
| "Tensor.matmul()",   | Contracts away dims  |
| "torch.matmul()"     |                      |
+----------------------+----------------------+
| "Tensor.mean()",     | Removes dimensions   |
| "torch.mean()"       |                      |
+----------------------+----------------------+
| "Tensor.median()",   | Removes dimensions   |
| "torch.median()"     |                      |
+----------------------+----------------------+
| "Tensor.nanmedian()  | Removes dimensions   |
| ",                   |                      |
| "torch.nanmedian()"  |                      |
+----------------------+----------------------+
| "Tensor.mm()",       | Contracts away dims  |
| "torch.mm()"         |                      |
+----------------------+----------------------+


+----------------------+----------------------+
| "Tensor.mode()",     | Removes dimensions   |
| "torch.mode()"       |                      |
+----------------------+----------------------+
| "Tensor.mul()",      | Unifies names from   |
| "torch.mul()"        | inputs               |
+----------------------+----------------------+
| "Tensor.mul_()"      | Unifies names from   |
|                      | inputs               |
+----------------------+----------------------+
| "Tensor.mv()",       | Contracts away dims  |
| "torch.mv()"         |                      |
+----------------------+----------------------+
| "Tensor.names"       | See documentation    |
+----------------------+----------------------+
| "Tensor.narrow()",   | Keeps input names    |
| "torch.narrow()"     |                      |
+----------------------+----------------------+
| "Tensor.ndim"        | None                 |
+----------------------+----------------------+
| "Tensor.ndimension(  | None                 |


| "Tensor.ndimension(  | None                 |
| )"                   |                      |
+----------------------+----------------------+
| "Tensor.ne()",       | Unifies names from   |
| "torch.ne()"         | inputs               |
+----------------------+----------------------+
| "Tensor.neg()",      | Keeps input names    |
| "torch.neg()"        |                      |
+----------------------+----------------------+
| "Tensor.neg_()"      | None                 |
+----------------------+----------------------+
| "torch.normal()"     | Keeps input names    |
+----------------------+----------------------+
| "Tensor.normal_()"   | None                 |
+----------------------+----------------------+
| "Tensor.numel()",    | None                 |
| "torch.numel()"      |                      |
+----------------------+----------------------+
| "torch.ones()"       | Factory functions    |
+----------------------+----------------------+
| "Tensor.pow()",      | Unifies names from   |


| "Tensor.pow()",      | Unifies names from   |
| "torch.pow()"        | inputs               |
+----------------------+----------------------+
| "Tensor.pow_()"      | None                 |
+----------------------+----------------------+
| "Tensor.prod()",     | Removes dimensions   |
| "torch.prod()"       |                      |
+----------------------+----------------------+
| "Tensor.rad2deg()",  | Keeps input names    |
| "torch.rad2deg()"    |                      |
+----------------------+----------------------+
| "Tensor.rad2deg_()"  | None                 |
+----------------------+----------------------+
| "torch.rand()"       | Factory functions    |
+----------------------+----------------------+
| "torch.rand()"       | Factory functions    |
+----------------------+----------------------+
| "torch.randn()"      | Factory functions    |
+----------------------+----------------------+
| "torch.randn()"      | Factory functions    |
+----------------------+----------------------+


+----------------------+----------------------+
| "Tensor.random_()"   | None                 |
+----------------------+----------------------+
| "Tensor.reciprocal(  | Keeps input names    |
| )",                  |                      |
| "torch.reciprocal()" |                      |
+----------------------+----------------------+
| "Tensor.reciprocal_  | None                 |
| ()"                  |                      |
+----------------------+----------------------+
| "Tensor.refine_name  | See documentation    |
| s()"                 |                      |
+----------------------+----------------------+
| "Tensor.register_ho  | None                 |
| ok()"                |                      |
+----------------------+----------------------+
| "Tensor.rename()"    | See documentation    |
+----------------------+----------------------+
| "Tensor.rename_()"   | See documentation    |
+----------------------+----------------------+
| "Tensor.requires_gr  | None                 |


| "Tensor.requires_gr  | None                 |
| ad"                  |                      |
+----------------------+----------------------+
| "Tensor.requires_gr  | None                 |
| ad_()"               |                      |
+----------------------+----------------------+
| "Tensor.resize_()"   | Only allow resizes   |
|                      | that do not change   |
|                      | shape                |
+----------------------+----------------------+
| "Tensor.resize_as_(  | Only allow resizes   |
| )"                   | that do not change   |
|                      | shape                |
+----------------------+----------------------+
| "Tensor.round()",    | Keeps input names    |
| "torch.round()"      |                      |
+----------------------+----------------------+
| "Tensor.round_()"    | None                 |
+----------------------+----------------------+
| "Tensor.rsqrt()",    | Keeps input names    |
| "torch.rsqrt()"      |                      |


| "torch.rsqrt()"      |                      |
+----------------------+----------------------+
| "Tensor.rsqrt_()"    | None                 |
+----------------------+----------------------+
| "Tensor.select()",   | Removes dimensions   |
| "torch.select()"     |                      |
+----------------------+----------------------+
| "Tensor.short()"     | Keeps input names    |
+----------------------+----------------------+
| "Tensor.sigmoid()",  | Keeps input names    |
| "torch.sigmoid()"    |                      |
+----------------------+----------------------+
| "Tensor.sigmoid_()"  | None                 |
+----------------------+----------------------+
| "Tensor.sign()",     | Keeps input names    |
| "torch.sign()"       |                      |
+----------------------+----------------------+
| "Tensor.sign_()"     | None                 |
+----------------------+----------------------+
| "Tensor.sgn()",      | Keeps input names    |
| "torch.sgn()"        |                      |


| "torch.sgn()"        |                      |
+----------------------+----------------------+
| "Tensor.sgn_()"      | None                 |
+----------------------+----------------------+
| "Tensor.sin()",      | Keeps input names    |
| "torch.sin()"        |                      |
+----------------------+----------------------+
| "Tensor.sin_()"      | None                 |
+----------------------+----------------------+
| "Tensor.sinh()",     | Keeps input names    |
| "torch.sinh()"       |                      |
+----------------------+----------------------+
| "Tensor.sinh_()"     | None                 |
+----------------------+----------------------+
| "Tensor.asinh()",    | Keeps input names    |
| "torch.asinh()"      |                      |
+----------------------+----------------------+
| "Tensor.asinh_()"    | None                 |
+----------------------+----------------------+
| "Tensor.size()"      | None                 |
+----------------------+----------------------+


+----------------------+----------------------+
| "Tensor.softmax()",  | Keeps input names    |
| "torch.softmax()"    |                      |
+----------------------+----------------------+
| "Tensor.split()",    | Keeps input names    |
| "torch.split()"      |                      |
+----------------------+----------------------+
| "Tensor.sqrt()",     | Keeps input names    |
| "torch.sqrt()"       |                      |
+----------------------+----------------------+
| "Tensor.sqrt_()"     | None                 |
+----------------------+----------------------+
| "Tensor.squeeze()",  | Removes dimensions   |
| "torch.squeeze()"    |                      |
+----------------------+----------------------+
| "Tensor.std()",      | Removes dimensions   |
| "torch.std()"        |                      |
+----------------------+----------------------+
| "torch.std_mean()"   | Removes dimensions   |
+----------------------+----------------------+
| "Tensor.stride()"    | None                 |


| "Tensor.stride()"    | None                 |
+----------------------+----------------------+
| "Tensor.sub()",      | Unifies names from   |
| "torch.sub()"        | inputs               |
+----------------------+----------------------+
| "Tensor.sub_()"      | Unifies names from   |
|                      | inputs               |
+----------------------+----------------------+
| "Tensor.sum()",      | Removes dimensions   |
| "torch.sum()"        |                      |
+----------------------+----------------------+
| "Tensor.tan()",      | Keeps input names    |
| "torch.tan()"        |                      |
+----------------------+----------------------+
| "Tensor.tan_()"      | None                 |
+----------------------+----------------------+
| "Tensor.tanh()",     | Keeps input names    |
| "torch.tanh()"       |                      |
+----------------------+----------------------+
| "Tensor.tanh_()"     | None                 |
+----------------------+----------------------+


+----------------------+----------------------+
| "Tensor.atanh()",    | Keeps input names    |
| "torch.atanh()"      |                      |
+----------------------+----------------------+
| "Tensor.atanh_()"    | None                 |
+----------------------+----------------------+
| "torch.tensor()"     | Factory functions    |
+----------------------+----------------------+
| "Tensor.to()"        | Keeps input names    |
+----------------------+----------------------+
| "Tensor.topk()",     | Removes dimensions   |
| "torch.topk()"       |                      |
+----------------------+----------------------+
| "Tensor.transpose()  | Permutes dimensions  |
| ",                   |                      |
| "torch.transpose()"  |                      |
+----------------------+----------------------+
| "Tensor.trunc()",    | Keeps input names    |
| "torch.trunc()"      |                      |
+----------------------+----------------------+
| "Tensor.trunc_()"    | None                 |


| "Tensor.trunc_()"    | None                 |
+----------------------+----------------------+
| "Tensor.type()"      | None                 |
+----------------------+----------------------+
| "Tensor.type_as()"   | Keeps input names    |
+----------------------+----------------------+
| "Tensor.unbind()",   | Removes dimensions   |
| "torch.unbind()"     |                      |
+----------------------+----------------------+
| "Tensor.unflatten()" | See documentation    |
+----------------------+----------------------+
| "Tensor.uniform_()"  | None                 |
+----------------------+----------------------+
| "Tensor.var()",      | Removes dimensions   |
| "torch.var()"        |                      |
+----------------------+----------------------+
| "torch.var_mean()"   | Removes dimensions   |
+----------------------+----------------------+
| "Tensor.zero_()"     | None                 |
+----------------------+----------------------+
| "torch.zeros()"      | Factory functions    |


| "torch.zeros()"      | Factory functions    |
+----------------------+----------------------+
Keeps input names
All pointwise unary functions follow this rule as well as some other
unary functions.

Check names: None

Propagate names: input tensor's names are propagated to the output.

x = torch.randn(3, 3, names=('N', 'C'))
x.abs().names
   ('N', 'C')

Removes dimensions
All reduction ops like "sum()" remove dimensions by reducing over the
desired dimensions. Other operations like "select()" and "squeeze()"
remove dimensions.
Wherever one can pass an integer dimension index to an operator, one
can also pass a dimension name. Functions that take lists of dimension
indices can also take in a list of dimension names.

Check names: If "dim" or "dims" is passed in as a list of names,
  check that those names exist in "self".

Propagate names: If the dimensions of the input tensor specified by


"dim" or "dims" are not present in the output tensor, then the
  corresponding names of those dimensions do not appear in
  "output.names".

x = torch.randn(1, 3, 3, 3, names=('N', 'C', 'H', 'W'))
x.squeeze('N').names
   ('C', 'H', 'W')
x = torch.randn(3, 3, 3, 3, names=('N', 'C', 'H', 'W'))
x.sum(['N', 'C']).names
   ('H', 'W')

# Reduction ops with keepdim=True don't actually remove dimensions.

x = torch.randn(3, 3, 3, 3, names=('N', 'C', 'H', 'W'))
x.sum(['N', 'C'], keepdim=True).names
   ('N', 'C', 'H', 'W')

Unifies names from inputs
All binary arithmetic ops follow this rule. Operations that broadcast
still broadcast positionally from the right to preserve compatibility
with unnamed tensors. To perform explicit broadcasting by names, use
"Tensor.align_as()".

Check names: All names must match positionally from the right. i.e.,
  in "tensor + other", "match(tensor.names[i], other.names[i])" must


be true for all "i" in "(-min(tensor.dim(), other.dim()) + 1, -1]".

Check names: Furthermore, all named dimensions must be aligned from
  the right. During matching, if we match a named dimension "A" with
  an unnamed dimension "None", then "A" must not appear in the tensor
  with the unnamed dimension.

Propagate names: unify pairs of names from the right from both
  tensors to produce output names.

For example,
# tensor: Tensor[   N, None]
   # other:  Tensor[None,    C]

tensor = torch.randn(3, 3, names=('N', None))
other = torch.randn(3, 3, names=(None, 'C'))
(tensor + other).names
   ('N', 'C')

Check names:

"match(tensor.names[-1], other.names[-1])" is "True"

"match(tensor.names[-2], tensor.names[-2])" is "True"

Because we matched "None" in "tensor" with "'C'", check to make sure
  "'C'" doesn't exist in "tensor" (it does not).

Check to make sure "'N'" doesn't exists in "other" (it does not).

Finally, the output names are computed with "[unify('N', None),


unify(None, 'C')] = ['N', 'C']"
More examples:
# Dimensions don't match from the right:
   # tensor: Tensor[N, C]
   # other:  Tensor[   N]

tensor = torch.randn(3, 3, names=('N', 'C'))
other = torch.randn(3, names=('N',))
(tensor + other).names
   RuntimeError: Error when attempting to broadcast dims ['N', 'C'] and dims
   ['N']: dim 'C' and dim 'N' are at the same position from the right but do
   not match.

# Dimensions aren't aligned when matching tensor.names[-1] and other.names[-1]:
   # tensor: Tensor[N, None]
   # other:  Tensor[      N]

tensor = torch.randn(3, 3, names=('N', None))
other = torch.randn(3, names=('N',))
(tensor + other).names
   RuntimeError: Misaligned dims when attempting to broadcast dims ['N'] and
   dims ['N', None]: dim 'N' appears in a different position from the right
   across both lists.

Note:
In both of the last examples, it is possible to align the tensors by


names and then perform the addition. Use "Tensor.align_as()" to
  align tensors by name or "Tensor.align_to()" to align tensors to a
  custom dimension ordering.
Permutes dimensions
Some operations, like "Tensor.t()", permute the order of dimensions.
Dimension names are attached to individual dimensions so they get
permuted as well.
If the operator takes in positional index "dim", it is also able to
take a dimension name as "dim".

Check names: If "dim" is passed as a name, check that it exists in
  the tensor.

Propagate names: Permute dimension names in the same way as the
  dimensions that are being permuted.

x = torch.randn(3, 3, names=('N', 'C'))
x.transpose('N', 'C').names
   ('C', 'N')

Contracts away dims
Matrix multiply functions follow some variant of this. Let's go
through "torch.mm()" first and then generalize the rule for batch
matrix multiplication.
For "torch.mm(tensor, other)":

Check names: None


Check names: None

Propagate names: result names are "(tensor.names[-2],
  other.names[-1])".

x = torch.randn(3, 3, names=('N', 'D'))
y = torch.randn(3, 3, names=('in', 'out'))
x.mm(y).names
   ('N', 'out')

Inherently, a matrix multiplication performs a dot product over two
dimensions, collapsing them. When two tensors are matrix-multiplied,
the contracted dimensions disappear and do not show up in the output
tensor.
"torch.mv()", "torch.dot()" work in a similar way: name inference does
not check input names and removes the dimensions that are involved in
the dot product:

x = torch.randn(3, 3, names=('N', 'D'))
y = torch.randn(3, names=('something',))
x.mv(y).names
   ('N',)

Now, let's take a look at "torch.matmul(tensor, other)". Assume that
"tensor.dim() >= 2" and "other.dim() >= 2".

Check names: Check that the batch dimensions of the inputs are
  aligned and broadcastable. See Unifies names from inputs for what it
  means for the inputs to be aligned.


means for the inputs to be aligned.

Propagate names: result names are obtained by unifying the batch
  dimensions and removing the contracted dimensions:
  "unify(tensor.names[:-2], other.names[:-2]) + (tensor.names[-2],
  other.names[-1])".

Examples:
# Batch matrix multiply of matrices Tensor['C', 'D'] and Tensor['E', 'F'].
   # 'A', 'B' are batch dimensions.

x = torch.randn(3, 3, 3, 3, names=('A', 'B', 'C', 'D'))
y = torch.randn(3, 3, 3, names=('B', 'E', 'F'))
torch.matmul(x, y).names
   ('A', 'B', 'C', 'F')

Finally, there are fused "add" versions of many matmul functions.
i.e., "addmm()" and "addmv()". These are treated as composing name
inference for i.e. "mm()" and name inference for "add()".
Factory functions
Factory functions now take a new "names" argument that associates a
name with each dimension.

torch.zeros(2, 3, names=('N', 'C'))
   tensor([[0., 0., 0.],
           [0., 0., 0.]], names=('N', 'C'))

out function and in-place variants


out function and in-place variants
A tensor specified as an "out=" tensor has the following behavior:

If it has no named dimensions, then the names computed from the
  operation get propagated to it.

If it has any named dimensions, then the names computed from the
  operation must be exactly equal to the existing names. Otherwise,
  the operation errors.

All in-place methods modify inputs to have names equal to the computed
names from name inference. For example:

x = torch.randn(3, 3)
y = torch.randn(3, 3, names=('N', 'C'))
x.names
   (None, None)
x += y
x.names
   ('N', 'C')


Tensor Parallelism


torch.library
Python operator registration API provides capabilities for extending
PyTorch's core library of operators with user defined operators.
Currently, this can be done in two ways:

Creating new libraries

Lets you to register new operators and kernels for various
     backends and functionalities by specifying the appropriate
     dispatch keys. For example,
* Consider registering a new operator "add" in your newly
  created namespace "foo". You can access this operator using
  the "torch.ops" API and calling into by calling
  "torch.ops.foo.add". You can also access specific registered
  overloads by calling "torch.ops.foo.add.{overload_name}".

* If you registered a new kernel for the "CUDA" dispatch key
  for this operator, then your custom defined function will be
  called for CUDA tensor inputs.

This can be done by creating Library class objects of ""DEF""
     kind.


kind.

Extending existing C++ libraries (e.g., aten)

Lets you register kernels for existing operators
     corresponding to various backends and functionalities by
     specifying the appropriate dispatch keys.

This may come in handy to fill up spotty operator support for a
     feature implemented through a dispatch key. For example.,
* You can add operator support for Meta Tensors (by
  registering function to the "Meta" dispatch key).

This can be done by creating Library class objects of ""IMPL""
     kind.

A tutorial that walks you through some examples on how to use this API
is available on Google Colab.
Warning:
Dispatcher is a complicated PyTorch concept and having a sound
  understanding of Dispatcher is crucial to be able to do anything
  advanced with this API. This blog post is a good starting point to
  learn about Dispatcher.
class torch.library.Library(ns, kind, dispatch_key='')
A class to create libraries that can be used to register new


operators or override operators in existing libraries from Python.
   A user can optionally pass in a dispatch keyname if they only want
   to register kernels corresponding to only one specific dispatch
   key.
To create a library to override operators in an existing library
   (with name ns), set the kind to "IMPL". To create a new library
   (with name ns) to register new operators, set the kind to "DEF".
   :param ns: library name :param kind: "DEF", "IMPL" (default:
   "IMPL") :param dispatch_key: PyTorch dispatch key (default: "")
define(schema, alias_analysis='')
  Defines a new operator and its semantics in the ns namespace.

  Parameters:
     * **schema** -- function schema to define a new operator.

     * **alias_analysis** (*optional*) -- Indicates if the
       aliasing properties of the operator arguments can be
       inferred from the schema (default behavior) or not
       ("CONSERVATIVE").

  Returns:


("CONSERVATIVE").
  Returns:
     name of the operator as inferred from the schema.

  Example::
     >>> my_lib = Library("foo", "DEF")
     >>> my_lib.define("sum(Tensor self) -> Tensor")

impl(op_name, fn, dispatch_key='')
  Registers the function implementation for an operator defined in
  the library.

  Parameters:
     * **op_name** -- operator name (along with the overload) or
       OpOverload object.

     * **fn** -- function that's the operator implementation for
       the input dispatch key.

     * **dispatch_key** -- dispatch key that the input function
       should be registered for. By default, it uses the dispatch
       key that the library was created with.

  Example::
     >>> my_lib = Library("aten", "IMPL")
     >>> def div_cpu(self, other):
     >>>     return self * (1 / other)
     >>> my_lib.impl("div.Tensor", "CPU")

We have also added some function decorators to make it convenient to


register functions for operators:

"torch.library.impl()"

"torch.library.define()"


Named Tensors
Named Tensors allow users to give explicit names to tensor dimensions.
In most cases, operations that take dimension parameters will accept
dimension names, avoiding the need to track dimensions by position. In
addition, named tensors use names to automatically check that APIs are
being used correctly at runtime, providing extra safety. Names can
also be used to rearrange dimensions, for example, to support
"broadcasting by name" rather than "broadcasting by position".
Warning:
The named tensor API is a prototype feature and subject to change.
Creating named tensors
Factory functions now take a new "names" argument that associates a
name with each dimension.

torch.zeros(2, 3, names=('N', 'C'))
   tensor([[0., 0., 0.],
           [0., 0., 0.]], names=('N', 'C'))

Named dimensions, like regular Tensor dimensions, are ordered.
"tensor.names[i]" is the name of dimension "i" of "tensor".
The following factory functions support named tensors:


"torch.empty()"

"torch.rand()"

"torch.randn()"

"torch.ones()"

"torch.tensor()"

"torch.zeros()"

Named dimensions
See "names" for restrictions on tensor names.
Use "names" to access the dimension names of a tensor and "rename()"
to rename named dimensions.

imgs = torch.randn(1, 2, 2, 3 , names=('N', 'C', 'H', 'W'))
imgs.names
   ('N', 'C', 'H', 'W')
renamed_imgs = imgs.rename(H='height', W='width')
renamed_imgs.names
   ('N', 'C', 'height', 'width)

Named tensors can coexist with unnamed tensors; named tensors are
instances of "torch.Tensor". Unnamed tensors have "None"-named
dimensions. Named tensors do not require all dimensions to be named.

imgs = torch.randn(1, 2, 2, 3 , names=(None, 'C', 'H', 'W'))
imgs.names
   (None, 'C', 'H', 'W')

Name propagation semantics
Named tensors use names to automatically check that APIs are being
called correctly at runtime. This occurs in a process called *name


inference*. More formally, name inference consists of the following
two steps:

Check names: an operator may perform automatic checks at runtime
  that check that certain dimension names must match.

Propagate names: name inference propagates names to output
  tensors.

All operations that support named tensors propagate names.

x = torch.randn(3, 3, names=('N', 'C'))
x.abs().names
   ('N', 'C')

match semantics
Two names match if they are equal (string equality) or if at least
one is "None". Nones are essentially a special "wildcard" name.
"unify(A, B)" determines which of the names "A" and "B" to propagate
to the outputs. It returns the more specific of the two names, if
they match. If the names do not match, then it errors.
Note:
In practice, when working with named tensors, one should avoid
  having unnamed dimensions because their handling can be complicated.
  It is recommended to lift all unnamed dimensions to be named


dimensions by using "refine_names()".
Basic name inference rules
Let's see how "match" and "unify" are used in name inference in the
case of adding two one-dim tensors with no broadcasting.
x = torch.randn(3, names=('X',))
   y = torch.randn(3)
   z = torch.randn(3, names=('Z',))
Check names: check that the names of the two tensors match.
For the following examples:

x + y  # match('X', None) is True
x + z  # match('X', 'Z') is False
x + x  # match('X', 'X') is True
x + z
   Error when attempting to broadcast dims ['X'] and dims ['Z']: dim 'X' and dim 'Z' are at the same position from the right but do not match.

Propagate names: unify the names to select which one to
propagate. In the case of "x + y", "unify('X', None) = 'X'" because
"'X'" is more specific than "None".

(x + y).names
   ('X',)
(x + x).names
   ('X',)

For a comprehensive list of name inference rules, see Named Tensors


operator coverage. Here are two common operations that may be useful
to go over:

Binary arithmetic ops: Unifies names from inputs

Matrix multiplication ops: Contracts away dims

Explicit alignment by names
Use "align_as()" or "align_to()" to align tensor dimensions by name to
a specified ordering. This is useful for performing "broadcasting by
names".
# This function is agnostic to the dimension ordering of input,
   # as long as it has a C dimension somewhere.
   def scale_channels(input, scale):
       scale = scale.refine_names('C')
       return input * scale.align_as(input)

num_channels = 3
scale = torch.randn(num_channels, names=('C',))
imgs = torch.rand(3, 3, 3, num_channels, names=('N', 'H', 'W', 'C'))
more_imgs = torch.rand(3, num_channels, 3, 3, names=('N', 'C', 'H', 'W'))
videos = torch.randn(3, num_channels, 3, 3, 3, names=('N', 'C', 'H', 'W', 'D')
scale_channels(imgs, scale)


scale_channels(imgs, scale)
scale_channels(more_imgs, scale)
scale_channels(videos, scale)

Manipulating dimensions
Use "align_to()" to permute large amounts of dimensions without
mentioning all of them as in required by "permute()".

tensor = torch.randn(2, 2, 2, 2, 2, 2)
named_tensor = tensor.refine_names('A', 'B', 'C', 'D', 'E', 'F')

# Move the F (dim 5) and E dimension (dim 4) to the front while keeping
   # the rest in the same order

tensor.permute(5, 4, 0, 1, 2, 3)
named_tensor.align_to('F', 'E', ...)

Use "flatten()" and "unflatten()" to flatten and unflatten dimensions,
respectively. These methods are more verbose than "view()" and
"reshape()", but have more semantic meaning to someone reading the
code.

imgs = torch.randn(32, 3, 128, 128)
named_imgs = imgs.refine_names('N', 'C', 'H', 'W')
flat_imgs = imgs.view(32, -1)
named_flat_imgs = named_imgs.flatten(['C', 'H', 'W'], 'features')


named_flat_imgs.names
   ('N', 'features')
unflattened_imgs = imgs.view(32, 3, 128, 128)
unflattened_named_imgs = named_flat_imgs.unflatten(
           'features', [('C', 3), ('H', 128), ('W', 128)])

Autograd support
Autograd currently supports named tensors in a limited manner:
autograd ignores names on all tensors. Gradient computation is still
correct but we lose the safety that names give us.

x = torch.randn(3, names=('D',))
weight = torch.randn(3, names=('D',), requires_grad=True)
loss = (x - weight).abs()
grad_loss = torch.randn(3)
loss.backward(grad_loss)
weight.grad  # Unnamed for now. Will be named in the future
   tensor([-1.8107, -0.6357,  0.0783])
weight.grad.zero_()
grad_loss = grad_loss.refine_names('C')
loss = (x - weight).abs()
   # Ideally we'd check that the names of loss and grad_loss match but we don't yet.
loss.backward(grad_loss)
weight.grad


loss.backward(grad_loss)
weight.grad
   tensor([-1.8107, -0.6357,  0.0783])

Currently supported operations and subsystems
Operators
See Named Tensors operator coverage for a full list of the supported
torch and tensor operations. We do not yet support the following that
is not covered by the link:

indexing, advanced indexing.

For "torch.nn.functional" operators, we support the following:

"torch.nn.functional.relu()"

"torch.nn.functional.softmax()"

"torch.nn.functional.log_softmax()"

"torch.nn.functional.tanh()"

"torch.nn.functional.sigmoid()"

"torch.nn.functional.dropout()"

Subsystems
Autograd is supported, see Autograd support. Because gradients are
currently unnamed, optimizers may work but are untested.
NN modules are currently unsupported. This can lead to the following
when calling modules with named tensor inputs:

NN module parameters are unnamed, so outputs may be partially named.


NN module forward passes have code that don't support named tensors
  and will error out appropriately.

We also do not support the following subsystems, though some may work
out of the box:

distributions

serialization ("torch.load()", "torch.save()")

multiprocessing

JIT

distributed

ONNX

If any of these would help your use case, please search if an issue
has already been filed and if not, file one.
Named tensor API reference
In this section please find the documentation for named tensor
specific APIs. For a comprehensive reference for how names are
propagated through other PyTorch operators, see Named Tensors operator
coverage.
class torch.Tensor
names
  Stores names for each of this tensor's dimensions.

  "names[idx]" corresponds to the name of tensor dimension "idx".
  Names are either a string if the dimension is named or "None" if
  the dimension is unnamed.

  Dimension names may contain characters or underscore.


Furthermore, a dimension name must be a valid Python variable
      name (i.e., does not start with underscore).
  Tensors may not have two named dimensions with the same name.

  Warning:

    The named tensor API is experimental and subject to change.

rename(names, *rename_map)
  Renames dimension names of "self".

  There are two main usages:

  "self.rename(**rename_map)" returns a view on tensor that has
  dims renamed as specified in the mapping "rename_map".

  "self.rename(*names)" returns a view on tensor, renaming all
  dimensions positionally using "names". Use "self.rename(None)"
  to drop names on a tensor.

  One cannot specify both positional args "names" and keyword args
  "rename_map".

  Examples:

     >>> imgs = torch.rand(2, 3, 5, 7, names=('N', 'C', 'H', 'W'))
     >>> renamed_imgs = imgs.rename(N='batch', C='channels')
     >>> renamed_imgs.names
     ('batch', 'channels', 'H', 'W')