The distributions package contains parameterizable probability distributions and sampling functions. This allows the construction of stochastic computation graphs and stochastic gradient estimators for optimization. This package generally follows the design of the TensorFlow Distributions package.
It is not possible to directly backpropagate through random samples. However, there are two main methods for creating surrogate functions that can be backpropagated through. These are the score function estimator/likelihood ratio estimator/REINFORCE and the pathwise derivative estimator. REINFORCE is commonly seen as the basis for policy gradient methods in reinforcement learning, and the pathwise derivative estimator is commonly seen in the reparameterization trick in variational autoencoders. Whilst the score function only requires the value of samples f(x)f(x) , the pathwise derivative requires the derivative f′(x)f'(x) . The next sections discuss these two in a reinforcement learning example. For more details see Gradient Estimation Using Stochastic Computation Graphs .
When the probability density function is differentiable with respect to its parameters, we only need sample() and log_prob() to implement REINFORCE:
where θ\theta  are the parameters, α\alpha  is the learning rate, rr  is the reward and p(a∣πθ(s))p(a|\pi^\theta(s))  is the probability of taking action aa  in state ss  given policy πθ\pi^\theta .
In practice we would sample an action from the output of a network, apply this action in an environment, and then use log_prob to construct an equivalent loss function. Note that we use a negative because optimizers use gradient descent, whilst the rule above assumes gradient ascent. With a categorical policy, the code for implementing REINFORCE would be as follows:
The other way to implement these stochastic/policy gradients would be to use the reparameterization trick from the rsample() method, where the parameterized random variable can be constructed via a parameterized deterministic function of a parameter-free random variable. The reparameterized sample therefore becomes differentiable. The code for implementing the pathwise derivative would be as follows:
Bases: object
Distribution is the abstract base class for probability distributions.
Returns a dictionary from argument names to Constraint objects that should be satisfied by each argument of this distribution. Args that are not tensors need not appear in this dict.
Returns the shape over which parameters are batched.
Returns the cumulative density/mass function evaluated at value.
value (Tensor) –
Returns entropy of distribution, batched over batch_shape.
Tensor of shape batch_shape.
Returns tensor containing all values supported by a discrete distribution. The result will enumerate over dimension 0, so the shape of the result will be (cardinality,) + batch_shape + event_shape (where event_shape = () for univariate distributions).
Note that this enumerates over all batched tensors in lock-step [[0, 0], [1, 1], …]. With expand=False, enumeration happens along dim 0, but with the remaining batch dimensions being singleton dimensions, [[0], [1], ...
To iterate over the full Cartesian product use itertools.product(m.enumerate_support()).
expand (bool) – whether to expand the support over the batch dims to match the distribution’s batch_shape.
Tensor iterating over dimension 0.
Returns the shape of a single sample (without batching).
Returns a new distribution instance (or populates an existing instance provided by a derived class) with batch dimensions expanded to batch_shape. This method calls expand on the distribution’s parameters. As such, this does not allocate new memory for the expanded distribution instance. Additionally, this does not repeat any args checking or parameter broadcasting in __init__.py, when an instance is first created.
New distribution instance with batch dimensions expanded to batch_size.
Returns the inverse cumulative density/mass function evaluated at value.
value (Tensor) –
Returns the log of the probability density/mass function evaluated at value.
value (Tensor) –
Returns the mean of the distribution.
Returns perplexity of distribution, batched over batch_shape.
Tensor of shape batch_shape.
Generates a sample_shape shaped reparameterized sample or sample_shape shaped batch of reparameterized samples if the distribution parameters are batched.
Generates a sample_shape shaped sample or sample_shape shaped batch of samples if the distribution parameters are batched.
Generates n samples or n batches of samples if the distribution parameters are batched.
Sets whether validation is enabled or disabled.
The default behavior mimics Python’s assert statement: validation is on by default, but is disabled if Python is run in optimized mode (via python -O). Validation may be expensive, so you may want to disable it once a model is working.
value (bool) – Whether to enable validation.
Returns the standard deviation of the distribution.
Returns a Constraint object representing this distribution’s support.
Returns the variance of the distribution.
Bases: torch.distributions.distribution.Distribution
ExponentialFamily is the abstract base class for probability distributions belonging to an exponential family, whose probability mass/density function has the form is defined below
where θ\theta  denotes the natural parameters, t(x)t(x)  denotes the sufficient statistic, F(θ)F(\theta)  is the log normalizer function for a given family and k(x)k(x)  is the carrier measure.
Note
This class is an intermediary between the Distribution class and distributions which belong to an exponential family mainly to check the correctness of the .entropy() and analytic KL divergence methods. We use this class to compute the entropy and KL divergence using the AD framework and Bregman divergences (courtesy of: Frank Nielsen and Richard Nock, Entropies and Cross-entropies of Exponential Families).
Method to compute the entropy using Bregman divergence of the log normalizer.
Bases: torch.distributions.exp_family.ExponentialFamily
Creates a Bernoulli distribution parameterized by probs or logits (but not both).
Samples are binary (0 or 1). They take the value 1 with probability p and 0 with probability 1 - p.
Example:
Bases: torch.distributions.exp_family.ExponentialFamily
Beta distribution parameterized by concentration1 and concentration0.
Example:
Bases: torch.distributions.distribution.Distribution
Creates a Binomial distribution parameterized by total_count and either probs or logits (but not both). total_count must be broadcastable with probs/logits.
Example:
Bases: torch.distributions.distribution.Distribution
Creates a categorical distribution parameterized by either probs or logits (but not both).
Note
It is equivalent to the distribution that torch.multinomial() samples from.
Samples are integers from {0,…,K−1}\{0, \ldots, K-1\}  where K is probs.size(-1).
If probs is 1-dimensional with length-K, each element is the relative probability of sampling the class at that index.
If probs is N-dimensional, the first N-1 dimensions are treated as a batch of relative probability vectors.
Note
The probs argument must be non-negative, finite and have a non-zero sum, and it will be normalized to sum to 1 along the last dimension. attr:probs will return this normalized value. The logits argument will be interpreted as unnormalized log probabilities and can therefore be any real number. It will likewise be normalized so that the resulting probabilities sum to 1 along the last dimension. attr:logits will return this normalized value.
See also: torch.multinomial()
Example:
Bases: torch.distributions.distribution.Distribution
Samples from a Cauchy (Lorentz) distribution. The distribution of the ratio of independent normally distributed random variables with means 0 follows a Cauchy distribution.
Example:
Bases: torch.distributions.gamma.Gamma
Creates a Chi2 distribution parameterized by shape parameter df. This is exactly equivalent to Gamma(alpha=0.5*df, beta=0.5)
Example:
df (float or Tensor) – shape parameter of the distribution
Bases: torch.distributions.exp_family.ExponentialFamily
Creates a continuous Bernoulli distribution parameterized by probs or logits (but not both).
The distribution is supported in [0, 1] and parameterized by ‘probs’ (in (0,1)) or ‘logits’ (real-valued). Note that, unlike the Bernoulli, ‘probs’ does not correspond to a probability and ‘logits’ does not correspond to log-odds, but the same names are used due to the similarity with the Bernoulli. See [1] for more details.
Example:
[1] The continuous Bernoulli: fixing a pervasive error in variational autoencoders, Loaiza-Ganem G and Cunningham JP, NeurIPS 2019. https://arxiv.org/abs/1907.06845
Bases: torch.distributions.exp_family.ExponentialFamily
Creates a Dirichlet distribution parameterized by concentration concentration.
Example:
concentration (Tensor) – concentration parameter of the distribution (often referred to as alpha)
Bases: torch.distributions.exp_family.ExponentialFamily
Creates a Exponential distribution parameterized by rate.
Example:
rate (float or Tensor) – rate = 1 / scale of the distribution
Bases: torch.distributions.distribution.Distribution
Creates a Fisher-Snedecor distribution parameterized by df1 and df2.
Example:
Bases: torch.distributions.exp_family.ExponentialFamily
Creates a Gamma distribution parameterized by shape concentration and rate.
Example:
Bases: torch.distributions.distribution.Distribution
Creates a Geometric distribution parameterized by probs, where probs is the probability of success of Bernoulli trials. It represents the probability that in k+1k + 1  Bernoulli trials, the first kk  trials failed, before seeing a success.
Samples are non-negative integers [0, inf\inf ).
Example:
Bases: torch.distributions.transformed_distribution.TransformedDistribution
Samples from a Gumbel Distribution.
Examples:
Bases: torch.distributions.transformed_distribution.TransformedDistribution
Creates a half-Cauchy distribution parameterized by scale where:
Example:
scale (float or Tensor) – scale of the full Cauchy distribution
Bases: torch.distributions.transformed_distribution.TransformedDistribution
Creates a half-normal distribution parameterized by scale where:
Example:
scale (float or Tensor) – scale of the full Normal distribution
Bases: torch.distributions.distribution.Distribution
Reinterprets some of the batch dims of a distribution as event dims.
This is mainly useful for changing the shape of the result of log_prob(). For example to create a diagonal Normal distribution with the same shape as a Multivariate Normal distribution (so they are interchangeable), you can:
Bases: torch.distributions.transformed_distribution.TransformedDistribution
Samples from a Kumaraswamy distribution.
Example:
Bases: torch.distributions.distribution.Distribution
LKJ distribution for lower Cholesky factor of correlation matrices. The distribution is controlled by concentration parameter η\eta  to make the probability of the correlation matrix MM  generated from a Cholesky factor propotional to det(M)η−1\det(M)^{\eta - 1} . Because of that, when concentration == 1, we have a uniform distribution over Cholesky factors of correlation matrices. Note that this distribution samples the Cholesky factor of correlation matrices and not the correlation matrices themselves and thereby differs slightly from the derivations in [1] for the LKJCorr distribution. For sampling, this uses the Onion method from [1] Section 3.
L ~ LKJCholesky(dim, concentration) X = L @ L’ ~ LKJCorr(dim, concentration)
Example:
References
[1] Generating random correlation matrices based on vines and extended onion method, Daniel Lewandowski, Dorota Kurowicka, Harry Joe.
Bases: torch.distributions.distribution.Distribution
Creates a Laplace distribution parameterized by loc and scale.
Example:
Bases: torch.distributions.transformed_distribution.TransformedDistribution
Creates a log-normal distribution parameterized by loc and scale where:
Example:
Bases: torch.distributions.distribution.Distribution
Creates a multivariate normal distribution with covariance matrix having a low-rank form parameterized by cov_factor and cov_diag:
Note
The computation for determinant and inverse of covariance matrix is avoided when cov_factor.shape[1] << cov_factor.shape[0] thanks to Woodbury matrix identity and matrix determinant lemma. Thanks to these formulas, we just need to compute the determinant and inverse of the small size “capacitance” matrix:
Bases: torch.distributions.distribution.Distribution
The MixtureSameFamily distribution implements a (batch of) mixture distribution where all component are from different parameterizations of the same distribution type. It is parameterized by a Categorical “selecting distribution” (over k component) and a component distribution, i.e., a Distribution with a rightmost batch shape (equal to [k]) which indexes each (batch of) component.
Examples:
Bases: torch.distributions.distribution.Distribution
Creates a Multinomial distribution parameterized by total_count and either probs or logits (but not both). The innermost dimension of probs indexes over categories. All other dimensions index over batches.
Note that total_count need not be specified if only log_prob() is called (see example below)
Note
The probs argument must be non-negative, finite and have a non-zero sum, and it will be normalized to sum to 1 along the last dimension. attr:probs will return this normalized value. The logits argument will be interpreted as unnormalized log probabilities and can therefore be any real number. It will likewise be normalized so that the resulting probabilities sum to 1 along the last dimension. attr:logits will return this normalized value.
Example:
Bases: torch.distributions.distribution.Distribution
Creates a multivariate normal (also called Gaussian) distribution parameterized by a mean vector and a covariance matrix.
The multivariate normal distribution can be parameterized either in terms of a positive definite covariance matrix Σ\mathbf{\Sigma}  or a positive definite precision matrix Σ−1\mathbf{\Sigma}^{-1}  or a lower-triangular matrix L\mathbf{L}  with positive-valued diagonal entries, such that Σ=LL⊤\mathbf{\Sigma} = \mathbf{L}\mathbf{L}^\top . This triangular matrix can be obtained via e.g. Cholesky decomposition of the covariance.
Note
Only one of covariance_matrix or precision_matrix or scale_tril can be specified.
Using scale_tril will be more efficient: all computations internally are based on scale_tril. If covariance_matrix or precision_matrix is passed instead, it is only used to compute the corresponding lower triangular matrices using a Cholesky decomposition.
Bases: torch.distributions.distribution.Distribution
Creates a Negative Binomial distribution, i.e. distribution of the number of successful independent and identical Bernoulli trials before total_count failures are achieved. The probability of failure of each Bernoulli trial is probs.
Bases: torch.distributions.exp_family.ExponentialFamily
Creates a normal (also called Gaussian) distribution parameterized by loc and scale.
Example:
Bases: torch.distributions.distribution.Distribution
Creates a one-hot categorical distribution parameterized by probs or logits.
Samples are one-hot coded vectors of size probs.size(-1).
Note
The probs argument must be non-negative, finite and have a non-zero sum, and it will be normalized to sum to 1 along the last dimension. attr:probs will return this normalized value. The logits argument will be interpreted as unnormalized log probabilities and can therefore be any real number. It will likewise be normalized so that the resulting probabilities sum to 1 along the last dimension. attr:logits will return this normalized value.
See also: torch.distributions.Categorical() for specifications of probs and logits.
Example:
Bases: torch.distributions.transformed_distribution.TransformedDistribution
Samples from a Pareto Type 1 distribution.
Example:
Bases: torch.distributions.exp_family.ExponentialFamily
Creates a Poisson distribution parameterized by rate, the rate parameter.
Samples are nonnegative integers, with a pmf given by
Example:
rate (Number, Tensor) – the rate parameter
Bases: torch.distributions.transformed_distribution.TransformedDistribution
Creates a RelaxedBernoulli distribution, parametrized by temperature, and either probs or logits (but not both). This is a relaxed version of the Bernoulli distribution, so the values are in (0, 1), and has reparametrizable samples.
Example:
Bases: torch.distributions.distribution.Distribution
Creates a LogitRelaxedBernoulli distribution parameterized by probs or logits (but not both), which is the logit of a RelaxedBernoulli distribution.
Samples are logits of values in (0, 1). See [1] for more details.
[1] The Concrete Distribution: A Continuous Relaxation of Discrete Random Variables (Maddison et al, 2017)
[2] Categorical Reparametrization with Gumbel-Softmax (Jang et al, 2017)
Bases: torch.distributions.transformed_distribution.TransformedDistribution
Creates a RelaxedOneHotCategorical distribution parametrized by temperature, and either probs or logits. This is a relaxed version of the OneHotCategorical distribution, so its samples are on simplex, and are reparametrizable.
Example:
Bases: torch.distributions.distribution.Distribution
Creates a Student’s t-distribution parameterized by degree of freedom df, mean loc and scale scale.
Example:
Bases: torch.distributions.distribution.Distribution
Extension of the Distribution class, which applies a sequence of Transforms to a base distribution. Let f be the composition of transforms applied:
Note that the .event_shape of a TransformedDistribution is the maximum shape of its base distribution and its transforms, since transforms can introduce correlations among events.
An example for the usage of TransformedDistribution would be:
For more examples, please look at the implementations of Gumbel, HalfCauchy, HalfNormal, LogNormal, Pareto, Weibull, RelaxedBernoulli and RelaxedOneHotCategorical
Computes the cumulative distribution function by inverting the transform(s) and computing the score of the base distribution.
Computes the inverse cumulative distribution function using transform(s) and computing the score of the base distribution.
Scores the sample by inverting the transform(s) and computing the score using the score of the base distribution and the log abs det jacobian.
Generates a sample_shape shaped reparameterized sample or sample_shape shaped batch of reparameterized samples if the distribution parameters are batched. Samples first from base distribution and applies transform() for every transform in the list.
Generates a sample_shape shaped sample or sample_shape shaped batch of samples if the distribution parameters are batched. Samples first from base distribution and applies transform() for every transform in the list.
Bases: torch.distributions.distribution.Distribution
Generates uniformly distributed random samples from the half-open interval [low, high).
Example:
Bases: torch.distributions.distribution.Distribution
A circular von Mises distribution.
This implementation uses polar coordinates. The loc and value args can be any real number (to facilitate unconstrained optimization), but are interpreted as angles modulo 2 pi.
The provided mean is the circular one.
The sampling algorithm for the von Mises distribution is based on the following paper: Best, D. J., and Nicholas I. Fisher. “Efficient simulation of the von Mises distribution.” Applied Statistics (1979): 152-157.
The provided variance is the circular one.
Bases: torch.distributions.transformed_distribution.TransformedDistribution
Samples from a two-parameter Weibull distribution.
Compute Kullback-Leibler divergence KL(p∥q)KL(p \| q)  between two distributions.
A batch of KL divergences of shape batch_shape.
Tensor
NotImplementedError – If the distribution types have not been registered via register_kl().
Decorator to register a pairwise function with kl_divergence(). Usage:
Lookup returns the most specific (type,type) match ordered by subclass. If the match is ambiguous, a RuntimeWarning is raised. For example to resolve the ambiguous situation:
you should register a third most-specific implementation, e.g.:
Abstract class for invertable transformations with computable log det jacobians. They are primarily used in torch.distributions.TransformedDistribution.
Caching is useful for transforms whose inverses are either expensive or numerically unstable. Note that care must be taken with memoized values since the autograd graph may be reversed. For example while the following works with or without caching:
However the following will error when caching due to dependency reversal:
Derived classes should implement one or both of _call() or _inverse(). Derived classes that set bijective=True should also implement log_abs_det_jacobian().
cache_size (int) – Size of cache. If zero, no caching is done. If one, the latest single value is cached. Only 0 and 1 are supported.
Returns the inverse Transform of this transform. This should satisfy t.inv.inv is t.
Returns the sign of the determinant of the Jacobian, if applicable. In general this only makes sense for bijective transforms.
Computes the log det jacobian log |dy/dx| given input and output.
Infers the shape of the forward computation, given the input shape. Defaults to preserving shape.
Infers the shapes of the inverse computation, given the output shape. Defaults to preserving shape.
Composes multiple transforms in a chain. The transforms being composed are responsible for caching.
Wrapper around another transform to treat reinterpreted_batch_ndims-many extra of the right most dimensions as dependent. This has no effect on the forward or backward transforms, but does sum out reinterpreted_batch_ndims-many of the rightmost dimensions in log_abs_det_jacobian().
Unit Jacobian transform to reshape the rightmost part of a tensor.
Note that in_shape and out_shape must have the same number of elements, just as for torch.Tensor.reshape().
Transform via the mapping y=exp(x)y = \exp(x) .
Transform via the mapping y=xexponenty = x^{\text{exponent}} .
Transform via the mapping y=11+exp(−x)y = \frac{1}{1 + \exp(-x)}  and x=logit(y)x = \text{logit}(y) .
Transform via the mapping y=tanh(x)y = \tanh(x) .
It is equivalent to `
ComposeTransform([AffineTransform(0., 2.), SigmoidTransform(), AffineTransform(-1., 2.)])
` However this might not be numerically stable, thus it is recommended to use TanhTransform instead.
Note that one should use cache_size=1 when it comes to NaN/Inf values.
Transform via the mapping y=∣x∣y = |x| .
Transform via the pointwise affine mapping y=loc+scale×xy = \text{loc} + \text{scale} \times x .
Transforms an uncontrained real vector xx  with length D∗(D−1)/2D*(D-1)/2  into the Cholesky factor of a D-dimension correlation matrix. This Cholesky factor is a lower triangular matrix with positive diagonals and unit Euclidean norm for each row. The transform is processed as follows:
Transform from unconstrained space to the simplex via y=exp(x)y = \exp(x)  then normalizing.
This is not bijective and cannot be used for HMC. However this acts mostly coordinate-wise (except for the final normalization), and thus is appropriate for coordinate-wise optimization algorithms.
Transform from unconstrained space to the simplex of one additional dimension via a stick-breaking process.
This transform arises as an iterated sigmoid transform in a stick-breaking construction of the Dirichlet distribution: the first logit is transformed via sigmoid to the first probability and the probability of everything else, and then the process recurses.
This is bijective and appropriate for use in HMC; however it mixes coordinates together and is less appropriate for optimization.
Transform from unconstrained matrices to lower-triangular matrices with nonnegative diagonal entries.
This is useful for parameterizing positive definite matrices in terms of their Cholesky factorization.
Transform functor that applies a sequence of transforms tseq component-wise to each submatrix at dim in a way compatible with torch.stack().
x = torch.stack([torch.range(1, 10), torch.range(1, 10)], dim=1) t = StackTransform([ExpTransform(), identity_transform], dim=1) y = t(x)
The following constraints are implemented:
Abstract base class for constraints.
A constraint object represents a region over which a variable is valid, e.g. within which a variable can be optimized.
Returns a byte tensor of sample_shape + batch_shape indicating whether each event in value satisfies this constraint.
alias of torch.distributions.constraints._DependentProperty
alias of torch.distributions.constraints._IndependentConstraint
alias of torch.distributions.constraints._IntegerInterval
alias of torch.distributions.constraints._GreaterThan
alias of torch.distributions.constraints._GreaterThanEq
alias of torch.distributions.constraints._LessThan
alias of torch.distributions.constraints._Multinomial
alias of torch.distributions.constraints._Interval
alias of torch.distributions.constraints._HalfOpenInterval
alias of torch.distributions.constraints._Cat
alias of torch.distributions.constraints._Stack
PyTorch provides two global ConstraintRegistry objects that link Constraint objects to Transform objects. These objects both input constraints and return transforms, but they have different guarantees on bijectivity.
The transform_to() registry is useful for performing unconstrained optimization on constrained parameters of probability distributions, which are indicated by each distribution’s .arg_constraints dict. These transforms often overparameterize a space in order to avoid rotation; they are thus more suitable for coordinate-wise optimization algorithms like Adam:
The biject_to() registry is useful for Hamiltonian Monte Carlo, where samples from a probability distribution with constrained .support are propagated in an unconstrained space, and algorithms are typically rotation invariant.:
Note
An example where transform_to and biject_to differ is constraints.simplex: transform_to(constraints.simplex) returns a SoftmaxTransform that simply exponentiates and normalizes its inputs; this is a cheap and mostly coordinate-wise operation appropriate for algorithms like SVI. In contrast, biject_to(constraints.simplex) returns a StickBreakingTransform that bijects its input down to a one-fewer-dimensional space; this a more expensive less numerically stable transform but is needed for algorithms like HMC.
The biject_to and transform_to objects can be extended by user-defined constraints and transforms using their .register() method either as a function on singleton constraints:
or as a decorator on parameterized constraints:
You can create your own registry by creating a new ConstraintRegistry object.
Registry to link constraints to transforms.
Registers a Constraint subclass in this registry. Usage:


Bases: torch.distributions.exp_family.ExponentialFamily
Creates a Bernoulli distribution parameterized by probs or logits (but not both).
Samples are binary (0 or 1). They take the value 1 with probability p and 0 with probability 1 - p.
Example:


Bases: torch.distributions.exp_family.ExponentialFamily
Beta distribution parameterized by concentration1 and concentration0.
Example:


Bases: torch.distributions.distribution.Distribution
Creates a Binomial distribution parameterized by total_count and either probs or logits (but not both). total_count must be broadcastable with probs/logits.
Example:


Bases: torch.distributions.distribution.Distribution
Creates a categorical distribution parameterized by either probs or logits (but not both).
Note
It is equivalent to the distribution that torch.multinomial() samples from.
Samples are integers from {0,…,K−1}\{0, \ldots, K-1\}  where K is probs.size(-1).
If probs is 1-dimensional with length-K, each element is the relative probability of sampling the class at that index.
If probs is N-dimensional, the first N-1 dimensions are treated as a batch of relative probability vectors.
Note
The probs argument must be non-negative, finite and have a non-zero sum, and it will be normalized to sum to 1 along the last dimension. attr:probs will return this normalized value. The logits argument will be interpreted as unnormalized log probabilities and can therefore be any real number. It will likewise be normalized so that the resulting probabilities sum to 1 along the last dimension. attr:logits will return this normalized value.
See also: torch.multinomial()
Example:


Bases: torch.distributions.distribution.Distribution
Samples from a Cauchy (Lorentz) distribution. The distribution of the ratio of independent normally distributed random variables with means 0 follows a Cauchy distribution.
Example:


Bases: torch.distributions.gamma.Gamma
Creates a Chi2 distribution parameterized by shape parameter df. This is exactly equivalent to Gamma(alpha=0.5*df, beta=0.5)
Example:
df (float or Tensor) – shape parameter of the distribution


Abstract base class for constraints.
A constraint object represents a region over which a variable is valid, e.g. within which a variable can be optimized.
Returns a byte tensor of sample_shape + batch_shape indicating whether each event in value satisfies this constraint.


Returns a byte tensor of sample_shape + batch_shape indicating whether each event in value satisfies this constraint.


Registry to link constraints to transforms.
Registers a Constraint subclass in this registry. Usage:


Registers a Constraint subclass in this registry. Usage:


Bases: torch.distributions.exp_family.ExponentialFamily
Creates a continuous Bernoulli distribution parameterized by probs or logits (but not both).
The distribution is supported in [0, 1] and parameterized by ‘probs’ (in (0,1)) or ‘logits’ (real-valued). Note that, unlike the Bernoulli, ‘probs’ does not correspond to a probability and ‘logits’ does not correspond to log-odds, but the same names are used due to the similarity with the Bernoulli. See [1] for more details.
Example:
[1] The continuous Bernoulli: fixing a pervasive error in variational autoencoders, Loaiza-Ganem G and Cunningham JP, NeurIPS 2019. https://arxiv.org/abs/1907.06845


Bases: torch.distributions.exp_family.ExponentialFamily
Creates a Dirichlet distribution parameterized by concentration concentration.
Example:
concentration (Tensor) – concentration parameter of the distribution (often referred to as alpha)


Bases: object
Distribution is the abstract base class for probability distributions.
Returns a dictionary from argument names to Constraint objects that should be satisfied by each argument of this distribution. Args that are not tensors need not appear in this dict.
Returns the shape over which parameters are batched.
Returns the cumulative density/mass function evaluated at value.
value (Tensor) –
Returns entropy of distribution, batched over batch_shape.
Tensor of shape batch_shape.
Returns tensor containing all values supported by a discrete distribution. The result will enumerate over dimension 0, so the shape of the result will be (cardinality,) + batch_shape + event_shape (where event_shape = () for univariate distributions).
Note that this enumerates over all batched tensors in lock-step [[0, 0], [1, 1], …]. With expand=False, enumeration happens along dim 0, but with the remaining batch dimensions being singleton dimensions, [[0], [1], ...
To iterate over the full Cartesian product use itertools.product(m.enumerate_support()).
expand (bool) – whether to expand the support over the batch dims to match the distribution’s batch_shape.
Tensor iterating over dimension 0.
Returns the shape of a single sample (without batching).
Returns a new distribution instance (or populates an existing instance provided by a derived class) with batch dimensions expanded to batch_shape. This method calls expand on the distribution’s parameters. As such, this does not allocate new memory for the expanded distribution instance. Additionally, this does not repeat any args checking or parameter broadcasting in __init__.py, when an instance is first created.
New distribution instance with batch dimensions expanded to batch_size.
Returns the inverse cumulative density/mass function evaluated at value.
value (Tensor) –
Returns the log of the probability density/mass function evaluated at value.
value (Tensor) –
Returns the mean of the distribution.
Returns perplexity of distribution, batched over batch_shape.
Tensor of shape batch_shape.
Generates a sample_shape shaped reparameterized sample or sample_shape shaped batch of reparameterized samples if the distribution parameters are batched.
Generates a sample_shape shaped sample or sample_shape shaped batch of samples if the distribution parameters are batched.
Generates n samples or n batches of samples if the distribution parameters are batched.
Sets whether validation is enabled or disabled.
The default behavior mimics Python’s assert statement: validation is on by default, but is disabled if Python is run in optimized mode (via python -O). Validation may be expensive, so you may want to disable it once a model is working.
value (bool) – Whether to enable validation.
Returns the standard deviation of the distribution.
Returns a Constraint object representing this distribution’s support.
Returns the variance of the distribution.


Returns a dictionary from argument names to Constraint objects that should be satisfied by each argument of this distribution. Args that are not tensors need not appear in this dict.


Returns the shape over which parameters are batched.


Returns the cumulative density/mass function evaluated at value.
value (Tensor) –


Returns entropy of distribution, batched over batch_shape.
Tensor of shape batch_shape.


Returns tensor containing all values supported by a discrete distribution. The result will enumerate over dimension 0, so the shape of the result will be (cardinality,) + batch_shape + event_shape (where event_shape = () for univariate distributions).
Note that this enumerates over all batched tensors in lock-step [[0, 0], [1, 1], …]. With expand=False, enumeration happens along dim 0, but with the remaining batch dimensions being singleton dimensions, [[0], [1], ...
To iterate over the full Cartesian product use itertools.product(m.enumerate_support()).
expand (bool) – whether to expand the support over the batch dims to match the distribution’s batch_shape.
Tensor iterating over dimension 0.


Returns the shape of a single sample (without batching).


Returns a new distribution instance (or populates an existing instance provided by a derived class) with batch dimensions expanded to batch_shape. This method calls expand on the distribution’s parameters. As such, this does not allocate new memory for the expanded distribution instance. Additionally, this does not repeat any args checking or parameter broadcasting in __init__.py, when an instance is first created.
New distribution instance with batch dimensions expanded to batch_size.


Returns the inverse cumulative density/mass function evaluated at value.
value (Tensor) –


Returns the log of the probability density/mass function evaluated at value.
value (Tensor) –


Returns perplexity of distribution, batched over batch_shape.
Tensor of shape batch_shape.


Generates a sample_shape shaped reparameterized sample or sample_shape shaped batch of reparameterized samples if the distribution parameters are batched.


Generates a sample_shape shaped sample or sample_shape shaped batch of samples if the distribution parameters are batched.


Generates n samples or n batches of samples if the distribution parameters are batched.


Sets whether validation is enabled or disabled.
The default behavior mimics Python’s assert statement: validation is on by default, but is disabled if Python is run in optimized mode (via python -O). Validation may be expensive, so you may want to disable it once a model is working.
value (bool) – Whether to enable validation.


Returns the standard deviation of the distribution.


Returns a Constraint object representing this distribution’s support.


Returns the variance of the distribution.


Bases: torch.distributions.exp_family.ExponentialFamily
Creates a Exponential distribution parameterized by rate.
Example:
rate (float or Tensor) – rate = 1 / scale of the distribution


Bases: torch.distributions.distribution.Distribution
ExponentialFamily is the abstract base class for probability distributions belonging to an exponential family, whose probability mass/density function has the form is defined below
where θ\theta  denotes the natural parameters, t(x)t(x)  denotes the sufficient statistic, F(θ)F(\theta)  is the log normalizer function for a given family and k(x)k(x)  is the carrier measure.
Note
This class is an intermediary between the Distribution class and distributions which belong to an exponential family mainly to check the correctness of the .entropy() and analytic KL divergence methods. We use this class to compute the entropy and KL divergence using the AD framework and Bregman divergences (courtesy of: Frank Nielsen and Richard Nock, Entropies and Cross-entropies of Exponential Families).
Method to compute the entropy using Bregman divergence of the log normalizer.


Method to compute the entropy using Bregman divergence of the log normalizer.


Bases: torch.distributions.distribution.Distribution
Creates a Fisher-Snedecor distribution parameterized by df1 and df2.
Example:


Bases: torch.distributions.exp_family.ExponentialFamily
Creates a Gamma distribution parameterized by shape concentration and rate.
Example:


Bases: torch.distributions.distribution.Distribution
Creates a Geometric distribution parameterized by probs, where probs is the probability of success of Bernoulli trials. It represents the probability that in k+1k + 1  Bernoulli trials, the first kk  trials failed, before seeing a success.
Samples are non-negative integers [0, inf\inf ).
Example:


Bases: torch.distributions.transformed_distribution.TransformedDistribution
Samples from a Gumbel Distribution.
Examples:


Bases: torch.distributions.transformed_distribution.TransformedDistribution
Creates a half-Cauchy distribution parameterized by scale where:
Example:
scale (float or Tensor) – scale of the full Cauchy distribution


Bases: torch.distributions.transformed_distribution.TransformedDistribution
Creates a half-normal distribution parameterized by scale where:
Example:
scale (float or Tensor) – scale of the full Normal distribution


Bases: torch.distributions.distribution.Distribution
Reinterprets some of the batch dims of a distribution as event dims.
This is mainly useful for changing the shape of the result of log_prob(). For example to create a diagonal Normal distribution with the same shape as a Multivariate Normal distribution (so they are interchangeable), you can:


Compute Kullback-Leibler divergence KL(p∥q)KL(p \| q)  between two distributions.
A batch of KL divergences of shape batch_shape.
Tensor
NotImplementedError – If the distribution types have not been registered via register_kl().


Decorator to register a pairwise function with kl_divergence(). Usage:
Lookup returns the most specific (type,type) match ordered by subclass. If the match is ambiguous, a RuntimeWarning is raised. For example to resolve the ambiguous situation:
you should register a third most-specific implementation, e.g.:


Bases: torch.distributions.transformed_distribution.TransformedDistribution
Samples from a Kumaraswamy distribution.
Example:


Bases: torch.distributions.distribution.Distribution
Creates a Laplace distribution parameterized by loc and scale.
Example:


Bases: torch.distributions.distribution.Distribution
LKJ distribution for lower Cholesky factor of correlation matrices. The distribution is controlled by concentration parameter η\eta  to make the probability of the correlation matrix MM  generated from a Cholesky factor propotional to det(M)η−1\det(M)^{\eta - 1} . Because of that, when concentration == 1, we have a uniform distribution over Cholesky factors of correlation matrices. Note that this distribution samples the Cholesky factor of correlation matrices and not the correlation matrices themselves and thereby differs slightly from the derivations in [1] for the LKJCorr distribution. For sampling, this uses the Onion method from [1] Section 3.
L ~ LKJCholesky(dim, concentration) X = L @ L’ ~ LKJCorr(dim, concentration)
Example:
References
[1] Generating random correlation matrices based on vines and extended onion method, Daniel Lewandowski, Dorota Kurowicka, Harry Joe.


Bases: torch.distributions.transformed_distribution.TransformedDistribution
Creates a log-normal distribution parameterized by loc and scale where:
Example:


Bases: torch.distributions.distribution.Distribution
Creates a multivariate normal distribution with covariance matrix having a low-rank form parameterized by cov_factor and cov_diag:
Note
The computation for determinant and inverse of covariance matrix is avoided when cov_factor.shape[1] << cov_factor.shape[0] thanks to Woodbury matrix identity and matrix determinant lemma. Thanks to these formulas, we just need to compute the determinant and inverse of the small size “capacitance” matrix:


Bases: torch.distributions.distribution.Distribution
The MixtureSameFamily distribution implements a (batch of) mixture distribution where all component are from different parameterizations of the same distribution type. It is parameterized by a Categorical “selecting distribution” (over k component) and a component distribution, i.e., a Distribution with a rightmost batch shape (equal to [k]) which indexes each (batch of) component.
Examples:


Bases: torch.distributions.distribution.Distribution
Creates a Multinomial distribution parameterized by total_count and either probs or logits (but not both). The innermost dimension of probs indexes over categories. All other dimensions index over batches.
Note that total_count need not be specified if only log_prob() is called (see example below)
Note
The probs argument must be non-negative, finite and have a non-zero sum, and it will be normalized to sum to 1 along the last dimension. attr:probs will return this normalized value. The logits argument will be interpreted as unnormalized log probabilities and can therefore be any real number. It will likewise be normalized so that the resulting probabilities sum to 1 along the last dimension. attr:logits will return this normalized value.
Example:


Bases: torch.distributions.distribution.Distribution
Creates a multivariate normal (also called Gaussian) distribution parameterized by a mean vector and a covariance matrix.
The multivariate normal distribution can be parameterized either in terms of a positive definite covariance matrix Σ\mathbf{\Sigma}  or a positive definite precision matrix Σ−1\mathbf{\Sigma}^{-1}  or a lower-triangular matrix L\mathbf{L}  with positive-valued diagonal entries, such that Σ=LL⊤\mathbf{\Sigma} = \mathbf{L}\mathbf{L}^\top . This triangular matrix can be obtained via e.g. Cholesky decomposition of the covariance.
Note
Only one of covariance_matrix or precision_matrix or scale_tril can be specified.
Using scale_tril will be more efficient: all computations internally are based on scale_tril. If covariance_matrix or precision_matrix is passed instead, it is only used to compute the corresponding lower triangular matrices using a Cholesky decomposition.


Bases: torch.distributions.distribution.Distribution
Creates a Negative Binomial distribution, i.e. distribution of the number of successful independent and identical Bernoulli trials before total_count failures are achieved. The probability of failure of each Bernoulli trial is probs.


Bases: torch.distributions.exp_family.ExponentialFamily
Creates a normal (also called Gaussian) distribution parameterized by loc and scale.
Example:


Bases: torch.distributions.distribution.Distribution
Creates a one-hot categorical distribution parameterized by probs or logits.
Samples are one-hot coded vectors of size probs.size(-1).
Note
The probs argument must be non-negative, finite and have a non-zero sum, and it will be normalized to sum to 1 along the last dimension. attr:probs will return this normalized value. The logits argument will be interpreted as unnormalized log probabilities and can therefore be any real number. It will likewise be normalized so that the resulting probabilities sum to 1 along the last dimension. attr:logits will return this normalized value.
See also: torch.distributions.Categorical() for specifications of probs and logits.
Example:


Bases: torch.distributions.transformed_distribution.TransformedDistribution
Samples from a Pareto Type 1 distribution.
Example:


Bases: torch.distributions.exp_family.ExponentialFamily
Creates a Poisson distribution parameterized by rate, the rate parameter.
Samples are nonnegative integers, with a pmf given by
Example:
rate (Number, Tensor) – the rate parameter


Bases: torch.distributions.distribution.Distribution
Creates a LogitRelaxedBernoulli distribution parameterized by probs or logits (but not both), which is the logit of a RelaxedBernoulli distribution.
Samples are logits of values in (0, 1). See [1] for more details.
[1] The Concrete Distribution: A Continuous Relaxation of Discrete Random Variables (Maddison et al, 2017)
[2] Categorical Reparametrization with Gumbel-Softmax (Jang et al, 2017)


Bases: torch.distributions.transformed_distribution.TransformedDistribution
Creates a RelaxedBernoulli distribution, parametrized by temperature, and either probs or logits (but not both). This is a relaxed version of the Bernoulli distribution, so the values are in (0, 1), and has reparametrizable samples.
Example:


Bases: torch.distributions.transformed_distribution.TransformedDistribution
Creates a RelaxedOneHotCategorical distribution parametrized by temperature, and either probs or logits. This is a relaxed version of the OneHotCategorical distribution, so its samples are on simplex, and are reparametrizable.
Example:


Bases: torch.distributions.distribution.Distribution
Creates a Student’s t-distribution parameterized by degree of freedom df, mean loc and scale scale.
Example:


Bases: torch.distributions.distribution.Distribution
Extension of the Distribution class, which applies a sequence of Transforms to a base distribution. Let f be the composition of transforms applied:
Note that the .event_shape of a TransformedDistribution is the maximum shape of its base distribution and its transforms, since transforms can introduce correlations among events.
An example for the usage of TransformedDistribution would be:
For more examples, please look at the implementations of Gumbel, HalfCauchy, HalfNormal, LogNormal, Pareto, Weibull, RelaxedBernoulli and RelaxedOneHotCategorical
Computes the cumulative distribution function by inverting the transform(s) and computing the score of the base distribution.
Computes the inverse cumulative distribution function using transform(s) and computing the score of the base distribution.
Scores the sample by inverting the transform(s) and computing the score using the score of the base distribution and the log abs det jacobian.
Generates a sample_shape shaped reparameterized sample or sample_shape shaped batch of reparameterized samples if the distribution parameters are batched. Samples first from base distribution and applies transform() for every transform in the list.
Generates a sample_shape shaped sample or sample_shape shaped batch of samples if the distribution parameters are batched. Samples first from base distribution and applies transform() for every transform in the list.


Computes the cumulative distribution function by inverting the transform(s) and computing the score of the base distribution.


Computes the inverse cumulative distribution function using transform(s) and computing the score of the base distribution.


Scores the sample by inverting the transform(s) and computing the score using the score of the base distribution and the log abs det jacobian.


Generates a sample_shape shaped reparameterized sample or sample_shape shaped batch of reparameterized samples if the distribution parameters are batched. Samples first from base distribution and applies transform() for every transform in the list.


Generates a sample_shape shaped sample or sample_shape shaped batch of samples if the distribution parameters are batched. Samples first from base distribution and applies transform() for every transform in the list.


Transform via the mapping y=∣x∣y = |x| .


Transform via the pointwise affine mapping y=loc+scale×xy = \text{loc} + \text{scale} \times x .


Composes multiple transforms in a chain. The transforms being composed are responsible for caching.


Transforms an uncontrained real vector xx  with length D∗(D−1)/2D*(D-1)/2  into the Cholesky factor of a D-dimension correlation matrix. This Cholesky factor is a lower triangular matrix with positive diagonals and unit Euclidean norm for each row. The transform is processed as follows:


Transform via the mapping y=exp(x)y = \exp(x) .


Wrapper around another transform to treat reinterpreted_batch_ndims-many extra of the right most dimensions as dependent. This has no effect on the forward or backward transforms, but does sum out reinterpreted_batch_ndims-many of the rightmost dimensions in log_abs_det_jacobian().


Transform from unconstrained matrices to lower-triangular matrices with nonnegative diagonal entries.
This is useful for parameterizing positive definite matrices in terms of their Cholesky factorization.


Transform via the mapping y=xexponenty = x^{\text{exponent}} .


Unit Jacobian transform to reshape the rightmost part of a tensor.
Note that in_shape and out_shape must have the same number of elements, just as for torch.Tensor.reshape().


Transform via the mapping y=11+exp(−x)y = \frac{1}{1 + \exp(-x)}  and x=logit(y)x = \text{logit}(y) .


Transform from unconstrained space to the simplex via y=exp(x)y = \exp(x)  then normalizing.
This is not bijective and cannot be used for HMC. However this acts mostly coordinate-wise (except for the final normalization), and thus is appropriate for coordinate-wise optimization algorithms.


Transform functor that applies a sequence of transforms tseq component-wise to each submatrix at dim in a way compatible with torch.stack().
x = torch.stack([torch.range(1, 10), torch.range(1, 10)], dim=1) t = StackTransform([ExpTransform(), identity_transform], dim=1) y = t(x)


Transform from unconstrained space to the simplex of one additional dimension via a stick-breaking process.
This transform arises as an iterated sigmoid transform in a stick-breaking construction of the Dirichlet distribution: the first logit is transformed via sigmoid to the first probability and the probability of everything else, and then the process recurses.
This is bijective and appropriate for use in HMC; however it mixes coordinates together and is less appropriate for optimization.


Transform via the mapping y=tanh(x)y = \tanh(x) .
It is equivalent to `
ComposeTransform([AffineTransform(0., 2.), SigmoidTransform(), AffineTransform(-1., 2.)])
` However this might not be numerically stable, thus it is recommended to use TanhTransform instead.
Note that one should use cache_size=1 when it comes to NaN/Inf values.


Abstract class for invertable transformations with computable log det jacobians. They are primarily used in torch.distributions.TransformedDistribution.
Caching is useful for transforms whose inverses are either expensive or numerically unstable. Note that care must be taken with memoized values since the autograd graph may be reversed. For example while the following works with or without caching:
However the following will error when caching due to dependency reversal:
Derived classes should implement one or both of _call() or _inverse(). Derived classes that set bijective=True should also implement log_abs_det_jacobian().
cache_size (int) – Size of cache. If zero, no caching is done. If one, the latest single value is cached. Only 0 and 1 are supported.
Returns the inverse Transform of this transform. This should satisfy t.inv.inv is t.
Returns the sign of the determinant of the Jacobian, if applicable. In general this only makes sense for bijective transforms.
Computes the log det jacobian log |dy/dx| given input and output.
Infers the shape of the forward computation, given the input shape. Defaults to preserving shape.
Infers the shapes of the inverse computation, given the output shape. Defaults to preserving shape.


Infers the shape of the forward computation, given the input shape. Defaults to preserving shape.


Returns the inverse Transform of this transform. This should satisfy t.inv.inv is t.


Infers the shapes of the inverse computation, given the output shape. Defaults to preserving shape.


Computes the log det jacobian log |dy/dx| given input and output.


Returns the sign of the determinant of the Jacobian, if applicable. In general this only makes sense for bijective transforms.


Bases: torch.distributions.distribution.Distribution
Generates uniformly distributed random samples from the half-open interval [low, high).
Example:


Bases: torch.distributions.distribution.Distribution
A circular von Mises distribution.
This implementation uses polar coordinates. The loc and value args can be any real number (to facilitate unconstrained optimization), but are interpreted as angles modulo 2 pi.
The provided mean is the circular one.
The sampling algorithm for the von Mises distribution is based on the following paper: Best, D. J., and Nicholas I. Fisher. “Efficient simulation of the von Mises distribution.” Applied Statistics (1979): 152-157.
The provided variance is the circular one.


The sampling algorithm for the von Mises distribution is based on the following paper: Best, D. J., and Nicholas I. Fisher. “Efficient simulation of the von Mises distribution.” Applied Statistics (1979): 152-157.


The provided variance is the circular one.


Bases: torch.distributions.transformed_distribution.TransformedDistribution
Samples from a two-parameter Weibull distribution.


Divides each element of the input input by the corresponding element of other.
Note
By default, this performs a “true” division like Python 3. See the rounding_mode argument for floor division.
Supports broadcasting to a common shape, type promotion, and integer, float, and complex inputs. Always promotes integer types to the default scalar type.
rounding_mode (str, optional) –
Type of rounding applied to the result:
Examples:


Computes the dot product of two 1D tensors.
Note
Unlike NumPy’s dot, torch.dot intentionally only supports computing the dot product of two 1D tensors with the same number of elements.
{out} –
Example:


Stack tensors in sequence depthwise (along third axis).
This is equivalent to concatenation along the third axis after 1-D and 2-D tensors have been reshaped by torch.atleast_3d().
tensors (sequence of Tensors) – sequence of tensors to concatenate
out (Tensor, optional) – the output tensor.


Computes the eigenvalues and eigenvectors of a real square matrix.
Note
Since eigenvalues and eigenvectors might be complex, backward pass is supported only if eigenvalues and eigenvectors are all real valued.
When input is on CUDA, torch.eig() causes host-device synchronization.
out (tuple, optional) – the output tensors
A namedtuple (eigenvalues, eigenvectors) containing
(Tensor, Tensor)
Example:


Sums the product of the elements of the input operands along dimensions specified using a notation based on the Einstein summation convention.
Einsum allows computing many common multi-dimensional linear algebraic array operations by representing them in a short-hand format based on the Einstein summation convention, given by equation. The details of this format are described below, but the general idea is to label every dimension of the input operands with some subscript and define which subscripts are part of the output. The output is then computed by summing the product of the elements of the operands along the dimensions whose subscripts are not part of the output. For example, matrix multiplication can be computed using einsum as torch.einsum(“ij,jk->ik”, A, B). Here, j is the summation subscript and i and k the output subscripts (see section below for more details on why).
Equation:
The equation string specifies the subscripts (lower case letters [‘a’, ‘z’]) for each dimension of the input operands in the same order as the dimensions, separating subcripts for each operand by a comma (‘,’), e.g. ‘ij,jk’ specify subscripts for two 2D operands. The dimensions labeled with the same subscript must be broadcastable, that is, their size must either match or be 1. The exception is if a subscript is repeated for the same input operand, in which case the dimensions labeled with this subscript for this operand must match in size and the operand will be replaced by its diagonal along these dimensions. The subscripts that appear exactly once in the equation will be part of the output, sorted in increasing alphabetical order. The output is computed by multiplying the input operands element-wise, with their dimensions aligned based on the subscripts, and then summing out the dimensions whose subscripts are not part of the output.
Optionally, the output subscripts can be explicitly defined by adding an arrow (‘->’) at the end of the equation followed by the subscripts for the output. For instance, the following equation computes the transpose of a matrix multiplication: ‘ij,jk->ki’. The output subscripts must appear at least once for some input operand and at most once for the output.
Ellipsis (‘…’) can be used in place of subscripts to broadcast the dimensions covered by the ellipsis. Each input operand may contain at most one ellipsis which will cover the dimensions not covered by subscripts, e.g. for an input operand with 5 dimensions, the ellipsis in the equation ‘ab…c’ cover the third and fourth dimensions. The ellipsis does not need to cover the same number of dimensions across the operands but the ‘shape’ of the ellipsis (the size of the dimensions covered by them) must broadcast together. If the output is not explicitly defined with the arrow (‘->’) notation, the ellipsis will come first in the output (left-most dimensions), before the subscript labels that appear exactly once for the input operands. e.g. the following equation implements batch matrix multiplication ‘…ij,…jk’.
A few final notes: the equation may contain whitespaces between the different elements (subscripts, ellipsis, arrow and comma) but something like ‘…’ is not valid. An empty string ‘’ is valid for scalar operands.
Note
torch.einsum handles ellipsis (‘…’) differently from NumPy in that it allows dimensions covered by the ellipsis to be summed over, that is, ellipsis are not required to be part of the output.
Note
This function does not optimize the given expression, so a different formula for the same computation may run faster or consume less memory. Projects like opt_einsum (https://optimized-einsum.readthedocs.io/en/stable/) can optimize the formula for you.
Examples:


Returns a tensor filled with uninitialized data. The shape of the tensor is defined by the variable argument size.
size (int...) – a sequence of integers defining the shape of the output tensor. Can be a variable number of arguments or a collection like a list or tuple.
Example:


Returns an uninitialized tensor with the same size as input. torch.empty_like(input) is equivalent to torch.empty(input.size(), dtype=input.dtype, layout=input.layout, device=input.device).
input (Tensor) – the size of input will determine size of the output tensor.
Example:


Returns a tensor filled with uninitialized data. The shape and strides of the tensor is defined by the variable argument size and stride respectively. torch.empty_strided(size, stride) is equivalent to torch.empty(size).as_strided(size, stride).
Warning
More than one element of the created tensor may refer to a single memory location. As a result, in-place operations (especially ones that are vectorized) may result in incorrect behavior. If you need to write to the tensors, please clone them first.
Example:


Context-manager that enables gradient calculation.
Enables gradient calculation, if it has been disabled via no_grad or set_grad_enabled.
This context manager is thread local; it will not affect computation in other threads.
Also functions as a decorator. (Make sure to instantiate with parenthesis.)
Example:


Computes element-wise equality
The second argument can be a number or a tensor whose shape is broadcastable with the first argument.
out (Tensor, optional) – the output tensor.
A boolean tensor that is True where input is equal to other and False elsewhere
Example:


True if two tensors have the same size and elements, False otherwise.
Example:


Computes the error function of each element. The error function is defined as follows:
input (Tensor) – the input tensor.
out (Tensor, optional) – the output tensor.
Example:


Computes the complementary error function of each element of input. The complementary error function is defined as follows:
input (Tensor) – the input tensor.
out (Tensor, optional) – the output tensor.
Example:


Computes the inverse error function of each element of input. The inverse error function is defined in the range (−1,1)(-1, 1)  as:
input (Tensor) – the input tensor.
out (Tensor, optional) – the output tensor.
Example:


Returns a new tensor with the exponential of the elements of the input tensor input.
input (Tensor) – the input tensor.
out (Tensor, optional) – the output tensor.
Example:


Computes the base two exponential function of input.
input (Tensor) – the input tensor.
out (Tensor, optional) – the output tensor.
Example:


Returns a new tensor with the exponential of the elements minus 1 of input.
input (Tensor) – the input tensor.
out (Tensor, optional) – the output tensor.
Example:


Returns a 2-D tensor with ones on the diagonal and zeros elsewhere.
A 2-D tensor with ones on the diagonal and zeros elsewhere
Tensor
Example:


Returns a new tensor with the data in input fake quantized per channel using scale, zero_point, quant_min and quant_max, across the channel specified by axis.
A newly fake_quantized per channel tensor
Tensor
Example:


Returns a new tensor with the data in input fake quantized using scale, zero_point, quant_min and quant_max.
A newly fake_quantized tensor
Tensor
Example:


Discrete Fourier transforms and related functions.
Computes the one dimensional discrete Fourier transform of input.
Note
The Fourier domain representation of any real signal satisfies the Hermitian property: X[i] = conj(X[-i]). This function always returns both the positive and negative frequency terms even though, for real inputs, the negative frequencies are redundant. rfft() returns the more compact one-sided representation where only the positive frequencies are returned.
norm (str, optional) –
Normalization mode. For the forward transform (fft()), these correspond to:
Calling the backward transform (ifft()) with the same normalization mode will apply an overall normalization of 1/n between the two transforms. This is required to make ifft() the exact inverse.
Default is "backward" (no normalization).
Computes the one dimensional inverse discrete Fourier transform of input.
norm (str, optional) –
Normalization mode. For the backward transform (ifft()), these correspond to:
Calling the forward transform (fft()) with the same normalization mode will apply an overall normalization of 1/n between the two transforms. This is required to make ifft() the exact inverse.
Default is "backward" (normalize by 1/n).
Computes the 2 dimensional discrete Fourier transform of input. Equivalent to fftn() but FFTs only the last two dimensions by default.
Note
The Fourier domain representation of any real signal satisfies the Hermitian property: X[i, j] = conj(X[-i, -j]). This function always returns all positive and negative frequency terms even though, for real inputs, half of these values are redundant. rfft2() returns the more compact one-sided representation where only the positive frequencies of the last dimension are returned.
norm (str, optional) –
Normalization mode. For the forward transform (fft2()), these correspond to:
Where n = prod(s) is the logical FFT size. Calling the backward transform (ifft2()) with the same normalization mode will apply an overall normalization of 1/n between the two transforms. This is required to make ifft2() the exact inverse.
Default is "backward" (no normalization).
The discrete Fourier transform is separable, so fft2() here is equivalent to two one-dimensional fft() calls:
Computes the 2 dimensional inverse discrete Fourier transform of input. Equivalent to ifftn() but IFFTs only the last two dimensions by default.
norm (str, optional) –
Normalization mode. For the backward transform (ifft2()), these correspond to:
Where n = prod(s) is the logical IFFT size. Calling the forward transform (fft2()) with the same normalization mode will apply an overall normalization of 1/n between the two transforms. This is required to make ifft2() the exact inverse.
Default is "backward" (normalize by 1/n).
The discrete Fourier transform is separable, so ifft2() here is equivalent to two one-dimensional ifft() calls:
Computes the N dimensional discrete Fourier transform of input.
Note
The Fourier domain representation of any real signal satisfies the Hermitian property: X[i_1, ..., i_n] = conj(X[-i_1, ..., -i_n]). This function always returns all positive and negative frequency terms even though, for real inputs, half of these values are redundant. rfftn() returns the more compact one-sided representation where only the positive frequencies of the last dimension are returned.
norm (str, optional) –
Normalization mode. For the forward transform (fftn()), these correspond to:
Where n = prod(s) is the logical FFT size. Calling the backward transform (ifftn()) with the same normalization mode will apply an overall normalization of 1/n between the two transforms. This is required to make ifftn() the exact inverse.
Default is "backward" (no normalization).
The discrete Fourier transform is separable, so fftn() here is equivalent to two one-dimensional fft() calls:
Computes the N dimensional inverse discrete Fourier transform of input.
norm (str, optional) –
Normalization mode. For the backward transform (ifftn()), these correspond to:
Where n = prod(s) is the logical IFFT size. Calling the forward transform (fftn()) with the same normalization mode will apply an overall normalization of 1/n between the two transforms. This is required to make ifftn() the exact inverse.
Default is "backward" (normalize by 1/n).
The discrete Fourier transform is separable, so ifftn() here is equivalent to two one-dimensional ifft() calls:
Computes the one dimensional Fourier transform of real-valued input.
The FFT of a real signal is Hermitian-symmetric, X[i] = conj(X[-i]) so the output contains only the positive frequencies below the Nyquist frequency. To compute the full output, use fft()
norm (str, optional) –
Normalization mode. For the forward transform (rfft()), these correspond to:
Calling the backward transform (irfft()) with the same normalization mode will apply an overall normalization of 1/n between the two transforms. This is required to make irfft() the exact inverse.
Default is "backward" (no normalization).
Compare against the full output from fft():
Notice that the symmetric element T[-1] == T[1].conj() is omitted. At the Nyquist frequency T[-2] == T[2] is it’s own symmetric pair, and therefore must always be real-valued.
Computes the inverse of rfft().
input is interpreted as a one-sided Hermitian signal in the Fourier domain, as produced by rfft(). By the Hermitian property, the output will be real-valued.
Note
Some input frequencies must be real-valued to satisfy the Hermitian property. In these cases the imaginary component will be ignored. For example, any imaginary component in the zero-frequency term cannot be represented in a real output and so will always be ignored.
Note
The correct interpretation of the Hermitian input depends on the length of the original data, as given by n. This is because each input shape could correspond to either an odd or even length signal. By default, the signal is assumed to be even length and odd signals will not round-trip properly. So, it is recommended to always pass the signal length n.
norm (str, optional) –
Normalization mode. For the backward transform (irfft()), these correspond to:
Calling the forward transform (rfft()) with the same normalization mode will apply an overall normalization of 1/n between the two transforms. This is required to make irfft() the exact inverse.
Default is "backward" (normalize by 1/n).
Without specifying the output length to irfft(), the output will not round-trip properly because the input is odd-length:
So, it is recommended to always pass the signal length n:
Computes the 2-dimensional discrete Fourier transform of real input. Equivalent to rfftn() but FFTs only the last two dimensions by default.
The FFT of a real signal is Hermitian-symmetric, X[i, j] = conj(X[-i, -j]), so the full fft2() output contains redundant information. rfft2() instead omits the negative frequencies in the last dimension.
norm (str, optional) –
Normalization mode. For the forward transform (rfft2()), these correspond to:
Where n = prod(s) is the logical FFT size. Calling the backward transform (irfft2()) with the same normalization mode will apply an overall normalization of 1/n between the two transforms. This is required to make irfft2() the exact inverse.
Default is "backward" (no normalization).
Compared against the full output from fft2(), we have all elements up to the Nyquist frequency.
The discrete Fourier transform is separable, so rfft2() here is equivalent to a combination of fft() and rfft():
Computes the inverse of rfft2(). Equivalent to irfftn() but IFFTs only the last two dimensions by default.
input is interpreted as a one-sided Hermitian signal in the Fourier domain, as produced by rfft2(). By the Hermitian property, the output will be real-valued.
Note
Some input frequencies must be real-valued to satisfy the Hermitian property. In these cases the imaginary component will be ignored. For example, any imaginary component in the zero-frequency term cannot be represented in a real output and so will always be ignored.
Note
The correct interpretation of the Hermitian input depends on the length of the original data, as given by s. This is because each input shape could correspond to either an odd or even length signal. By default, the signal is assumed to be even length and odd signals will not round-trip properly. So, it is recommended to always pass the signal shape s.
norm (str, optional) –
Normalization mode. For the backward transform (irfft2()), these correspond to:
Where n = prod(s) is the logical IFFT size. Calling the forward transform (rfft2()) with the same normalization mode will apply an overall normalization of 1/n between the two transforms. This is required to make irfft2() the exact inverse.
Default is "backward" (normalize by 1/n).
Without specifying the output length to irfft2(), the output will not round-trip properly because the input is odd-length in the last dimension:
So, it is recommended to always pass the signal shape s.
Computes the N-dimensional discrete Fourier transform of real input.
The FFT of a real signal is Hermitian-symmetric, X[i_1, ..., i_n] = conj(X[-i_1, ..., -i_n]) so the full fftn() output contains redundant information. rfftn() instead omits the negative frequencies in the last dimension.
norm (str, optional) –
Normalization mode. For the forward transform (rfftn()), these correspond to:
Where n = prod(s) is the logical FFT size. Calling the backward transform (irfftn()) with the same normalization mode will apply an overall normalization of 1/n between the two transforms. This is required to make irfftn() the exact inverse.
Default is "backward" (no normalization).
Compared against the full output from fftn(), we have all elements up to the Nyquist frequency.
The discrete Fourier transform is separable, so rfftn() here is equivalent to a combination of fft() and rfft():
Computes the inverse of rfftn().
input is interpreted as a one-sided Hermitian signal in the Fourier domain, as produced by rfftn(). By the Hermitian property, the output will be real-valued.
Note
Some input frequencies must be real-valued to satisfy the Hermitian property. In these cases the imaginary component will be ignored. For example, any imaginary component in the zero-frequency term cannot be represented in a real output and so will always be ignored.
Note
The correct interpretation of the Hermitian input depends on the length of the original data, as given by s. This is because each input shape could correspond to either an odd or even length signal. By default, the signal is assumed to be even length and odd signals will not round-trip properly. So, it is recommended to always pass the signal shape s.
norm (str, optional) –
Normalization mode. For the backward transform (irfftn()), these correspond to:
Where n = prod(s) is the logical IFFT size. Calling the forward transform (rfftn()) with the same normalization mode will apply an overall normalization of 1/n between the two transforms. This is required to make irfftn() the exact inverse.
Default is "backward" (normalize by 1/n).
Without specifying the output length to irfft(), the output will not round-trip properly because the input is odd-length in the last dimension:
So, it is recommended to always pass the signal shape s.
Computes the one dimensional discrete Fourier transform of a Hermitian symmetric input signal.
Note
hfft()/ihfft() are analogous to rfft()/irfft(). The real FFT expects a real signal in the time-domain and gives a Hermitian symmetry in the frequency-domain. The Hermitian FFT is the opposite; Hermitian symmetric in the time-domain and real-valued in the frequency-domain. For this reason, special care needs to be taken with the length argument n, in the same way as with irfft().
Note
Because the signal is Hermitian in the time-domain, the result will be real in the frequency domain. Note that some input frequencies must be real-valued to satisfy the Hermitian property. In these cases the imaginary component will be ignored. For example, any imaginary component in input[0] would result in one or more complex frequency terms which cannot be represented in a real output and so will always be ignored.
Note
The correct interpretation of the Hermitian input depends on the length of the original data, as given by n. This is because each input shape could correspond to either an odd or even length signal. By default, the signal is assumed to be even length and odd signals will not round-trip properly. So, it is recommended to always pass the signal length n.
norm (str, optional) –
Normalization mode. For the forward transform (hfft()), these correspond to:
Calling the backward transform (ihfft()) with the same normalization mode will apply an overall normalization of 1/n between the two transforms. This is required to make ihfft() the exact inverse.
Default is "backward" (no normalization).
Taking a real-valued frequency signal and bringing it into the time domain gives Hermitian symmetric output:
Note that T[1] == T[-1].conj() and T[2] == T[-2].conj() is redundant. We can thus compute the forward transform without considering negative frequencies:
Like with irfft(), the output length must be given in order to recover an even length output:
Computes the inverse of hfft().
input must be a real-valued signal, interpreted in the Fourier domain. The IFFT of a real signal is Hermitian-symmetric, X[i] = conj(X[-i]). ihfft() represents this in the one-sided form where only the positive frequencies below the Nyquist frequency are included. To compute the full output, use ifft().
norm (str, optional) –
Normalization mode. For the backward transform (ihfft()), these correspond to:
Calling the forward transform (hfft()) with the same normalization mode will apply an overall normalization of 1/n between the two transforms. This is required to make ihfft() the exact inverse.
Default is "backward" (normalize by 1/n).
Compare against the full output from ifft():
Computes the discrete Fourier Transform sample frequencies for a signal of size n.
Note
By convention, fft() returns positive frequency terms first, followed by the negative frequencies in reverse order, so that f[-i] for all 0<i≤n/20 < i \leq n/2  in Python gives the negative frequency terms. For an FFT of length n and with inputs spaced in length unit d, the frequencies are:
Note
For even lengths, the Nyquist frequency at f[n/2] can be thought of as either negative or positive. fftfreq() follows NumPy’s convention of taking it to be negative.
For even input, we can see the Nyquist frequency at f[2] is given as negative:
Computes the sample frequencies for rfft() with a signal of size n.
Note
rfft() returns Hermitian one-sided output, so only the positive frequency terms are returned. For a real FFT of length n and with inputs spaced in length unit d, the frequencies are:
Note
For even lengths, the Nyquist frequency at f[n/2] can be thought of as either negative or positive. Unlike fftfreq(), rfftfreq() always returns it as positive.
Compared to the output from fftfreq(), we see that the Nyquist frequency at f[2] has changed sign: >>> torch.fft.fftfreq(4) tensor([ 0.0000, 0.2500, -0.5000, -0.2500])
Reorders n-dimensional FFT data, as provided by fftn(), to have negative frequency terms first.
This performs a periodic shift of n-dimensional data such that the origin (0, ..., 0) is moved to the center of the tensor. Specifically, to input.shape[dim] // 2 in each selected dimension.
Note
By convention, the FFT returns positive frequency terms first, followed by the negative frequencies in reverse order, so that f[-i] for all 0<i≤n/20 < i \leq n/2  in Python gives the negative frequency terms. fftshift() rearranges all frequencies into ascending order from negative to positive with the zero-frequency term in the center.
Note
For even lengths, the Nyquist frequency at f[n/2] can be thought of as either negative or positive. fftshift() always puts the Nyquist term at the 0-index. This is the same convention used by fftfreq().
Also notice that the Nyquist frequency term at f[2] was moved to the beginning of the tensor.
This also works for multi-dimensional transforms:
fftshift() can also be useful for spatial data. If our data is defined on a centered grid ([-(N//2), (N-1)//2]) then we can use the standard FFT defined on an uncentered grid ([0, N)) by first applying an ifftshift().
Similarly, we can convert the frequency domain components to centered convention by applying fftshift().
The inverse transform, from centered Fourier space back to centered spatial data, can be performed by applying the inverse shifts in reverse order:
Inverse of fftshift().
A round-trip through fftshift() and ifftshift() gives the same result:


Computes the one dimensional discrete Fourier transform of input.
Note
The Fourier domain representation of any real signal satisfies the Hermitian property: X[i] = conj(X[-i]). This function always returns both the positive and negative frequency terms even though, for real inputs, the negative frequencies are redundant. rfft() returns the more compact one-sided representation where only the positive frequencies are returned.
norm (str, optional) –
Normalization mode. For the forward transform (fft()), these correspond to:
Calling the backward transform (ifft()) with the same normalization mode will apply an overall normalization of 1/n between the two transforms. This is required to make ifft() the exact inverse.
Default is "backward" (no normalization).


Computes the 2 dimensional discrete Fourier transform of input. Equivalent to fftn() but FFTs only the last two dimensions by default.
Note
The Fourier domain representation of any real signal satisfies the Hermitian property: X[i, j] = conj(X[-i, -j]). This function always returns all positive and negative frequency terms even though, for real inputs, half of these values are redundant. rfft2() returns the more compact one-sided representation where only the positive frequencies of the last dimension are returned.
norm (str, optional) –
Normalization mode. For the forward transform (fft2()), these correspond to:
Where n = prod(s) is the logical FFT size. Calling the backward transform (ifft2()) with the same normalization mode will apply an overall normalization of 1/n between the two transforms. This is required to make ifft2() the exact inverse.
Default is "backward" (no normalization).
The discrete Fourier transform is separable, so fft2() here is equivalent to two one-dimensional fft() calls:


Computes the discrete Fourier Transform sample frequencies for a signal of size n.
Note
By convention, fft() returns positive frequency terms first, followed by the negative frequencies in reverse order, so that f[-i] for all 0<i≤n/20 < i \leq n/2  in Python gives the negative frequency terms. For an FFT of length n and with inputs spaced in length unit d, the frequencies are:
Note
For even lengths, the Nyquist frequency at f[n/2] can be thought of as either negative or positive. fftfreq() follows NumPy’s convention of taking it to be negative.
For even input, we can see the Nyquist frequency at f[2] is given as negative:


Computes the N dimensional discrete Fourier transform of input.
Note
The Fourier domain representation of any real signal satisfies the Hermitian property: X[i_1, ..., i_n] = conj(X[-i_1, ..., -i_n]). This function always returns all positive and negative frequency terms even though, for real inputs, half of these values are redundant. rfftn() returns the more compact one-sided representation where only the positive frequencies of the last dimension are returned.
norm (str, optional) –
Normalization mode. For the forward transform (fftn()), these correspond to:
Where n = prod(s) is the logical FFT size. Calling the backward transform (ifftn()) with the same normalization mode will apply an overall normalization of 1/n between the two transforms. This is required to make ifftn() the exact inverse.
Default is "backward" (no normalization).
The discrete Fourier transform is separable, so fftn() here is equivalent to two one-dimensional fft() calls:


Reorders n-dimensional FFT data, as provided by fftn(), to have negative frequency terms first.
This performs a periodic shift of n-dimensional data such that the origin (0, ..., 0) is moved to the center of the tensor. Specifically, to input.shape[dim] // 2 in each selected dimension.
Note
By convention, the FFT returns positive frequency terms first, followed by the negative frequencies in reverse order, so that f[-i] for all 0<i≤n/20 < i \leq n/2  in Python gives the negative frequency terms. fftshift() rearranges all frequencies into ascending order from negative to positive with the zero-frequency term in the center.
Note
For even lengths, the Nyquist frequency at f[n/2] can be thought of as either negative or positive. fftshift() always puts the Nyquist term at the 0-index. This is the same convention used by fftfreq().
Also notice that the Nyquist frequency term at f[2] was moved to the beginning of the tensor.
This also works for multi-dimensional transforms:
fftshift() can also be useful for spatial data. If our data is defined on a centered grid ([-(N//2), (N-1)//2]) then we can use the standard FFT defined on an uncentered grid ([0, N)) by first applying an ifftshift().
Similarly, we can convert the frequency domain components to centered convention by applying fftshift().
The inverse transform, from centered Fourier space back to centered spatial data, can be performed by applying the inverse shifts in reverse order:


Computes the one dimensional discrete Fourier transform of a Hermitian symmetric input signal.
Note
hfft()/ihfft() are analogous to rfft()/irfft(). The real FFT expects a real signal in the time-domain and gives a Hermitian symmetry in the frequency-domain. The Hermitian FFT is the opposite; Hermitian symmetric in the time-domain and real-valued in the frequency-domain. For this reason, special care needs to be taken with the length argument n, in the same way as with irfft().
Note
Because the signal is Hermitian in the time-domain, the result will be real in the frequency domain. Note that some input frequencies must be real-valued to satisfy the Hermitian property. In these cases the imaginary component will be ignored. For example, any imaginary component in input[0] would result in one or more complex frequency terms which cannot be represented in a real output and so will always be ignored.
Note
The correct interpretation of the Hermitian input depends on the length of the original data, as given by n. This is because each input shape could correspond to either an odd or even length signal. By default, the signal is assumed to be even length and odd signals will not round-trip properly. So, it is recommended to always pass the signal length n.
norm (str, optional) –
Normalization mode. For the forward transform (hfft()), these correspond to:
Calling the backward transform (ihfft()) with the same normalization mode will apply an overall normalization of 1/n between the two transforms. This is required to make ihfft() the exact inverse.
Default is "backward" (no normalization).
Taking a real-valued frequency signal and bringing it into the time domain gives Hermitian symmetric output:
Note that T[1] == T[-1].conj() and T[2] == T[-2].conj() is redundant. We can thus compute the forward transform without considering negative frequencies:
Like with irfft(), the output length must be given in order to recover an even length output:


Computes the one dimensional inverse discrete Fourier transform of input.
norm (str, optional) –
Normalization mode. For the backward transform (ifft()), these correspond to:
Calling the forward transform (fft()) with the same normalization mode will apply an overall normalization of 1/n between the two transforms. This is required to make ifft() the exact inverse.
Default is "backward" (normalize by 1/n).


Computes the 2 dimensional inverse discrete Fourier transform of input. Equivalent to ifftn() but IFFTs only the last two dimensions by default.
norm (str, optional) –
Normalization mode. For the backward transform (ifft2()), these correspond to:
Where n = prod(s) is the logical IFFT size. Calling the forward transform (fft2()) with the same normalization mode will apply an overall normalization of 1/n between the two transforms. This is required to make ifft2() the exact inverse.
Default is "backward" (normalize by 1/n).
The discrete Fourier transform is separable, so ifft2() here is equivalent to two one-dimensional ifft() calls:


Computes the N dimensional inverse discrete Fourier transform of input.
norm (str, optional) –
Normalization mode. For the backward transform (ifftn()), these correspond to:
Where n = prod(s) is the logical IFFT size. Calling the forward transform (fftn()) with the same normalization mode will apply an overall normalization of 1/n between the two transforms. This is required to make ifftn() the exact inverse.
Default is "backward" (normalize by 1/n).
The discrete Fourier transform is separable, so ifftn() here is equivalent to two one-dimensional ifft() calls:


Inverse of fftshift().
A round-trip through fftshift() and ifftshift() gives the same result:


Computes the inverse of hfft().
input must be a real-valued signal, interpreted in the Fourier domain. The IFFT of a real signal is Hermitian-symmetric, X[i] = conj(X[-i]). ihfft() represents this in the one-sided form where only the positive frequencies below the Nyquist frequency are included. To compute the full output, use ifft().
norm (str, optional) –
Normalization mode. For the backward transform (ihfft()), these correspond to:
Calling the forward transform (hfft()) with the same normalization mode will apply an overall normalization of 1/n between the two transforms. This is required to make ihfft() the exact inverse.
Default is "backward" (normalize by 1/n).
Compare against the full output from ifft():


Computes the inverse of rfft().
input is interpreted as a one-sided Hermitian signal in the Fourier domain, as produced by rfft(). By the Hermitian property, the output will be real-valued.
Note
Some input frequencies must be real-valued to satisfy the Hermitian property. In these cases the imaginary component will be ignored. For example, any imaginary component in the zero-frequency term cannot be represented in a real output and so will always be ignored.
Note
The correct interpretation of the Hermitian input depends on the length of the original data, as given by n. This is because each input shape could correspond to either an odd or even length signal. By default, the signal is assumed to be even length and odd signals will not round-trip properly. So, it is recommended to always pass the signal length n.
norm (str, optional) –
Normalization mode. For the backward transform (irfft()), these correspond to:
Calling the forward transform (rfft()) with the same normalization mode will apply an overall normalization of 1/n between the two transforms. This is required to make irfft() the exact inverse.
Default is "backward" (normalize by 1/n).
Without specifying the output length to irfft(), the output will not round-trip properly because the input is odd-length:
So, it is recommended to always pass the signal length n:


Computes the inverse of rfft2(). Equivalent to irfftn() but IFFTs only the last two dimensions by default.
input is interpreted as a one-sided Hermitian signal in the Fourier domain, as produced by rfft2(). By the Hermitian property, the output will be real-valued.
Note
Some input frequencies must be real-valued to satisfy the Hermitian property. In these cases the imaginary component will be ignored. For example, any imaginary component in the zero-frequency term cannot be represented in a real output and so will always be ignored.
Note
The correct interpretation of the Hermitian input depends on the length of the original data, as given by s. This is because each input shape could correspond to either an odd or even length signal. By default, the signal is assumed to be even length and odd signals will not round-trip properly. So, it is recommended to always pass the signal shape s.
norm (str, optional) –
Normalization mode. For the backward transform (irfft2()), these correspond to:
Where n = prod(s) is the logical IFFT size. Calling the forward transform (rfft2()) with the same normalization mode will apply an overall normalization of 1/n between the two transforms. This is required to make irfft2() the exact inverse.
Default is "backward" (normalize by 1/n).
Without specifying the output length to irfft2(), the output will not round-trip properly because the input is odd-length in the last dimension:
So, it is recommended to always pass the signal shape s.


Computes the inverse of rfftn().
input is interpreted as a one-sided Hermitian signal in the Fourier domain, as produced by rfftn(). By the Hermitian property, the output will be real-valued.
Note
Some input frequencies must be real-valued to satisfy the Hermitian property. In these cases the imaginary component will be ignored. For example, any imaginary component in the zero-frequency term cannot be represented in a real output and so will always be ignored.
Note
The correct interpretation of the Hermitian input depends on the length of the original data, as given by s. This is because each input shape could correspond to either an odd or even length signal. By default, the signal is assumed to be even length and odd signals will not round-trip properly. So, it is recommended to always pass the signal shape s.
norm (str, optional) –
Normalization mode. For the backward transform (irfftn()), these correspond to:
Where n = prod(s) is the logical IFFT size. Calling the forward transform (rfftn()) with the same normalization mode will apply an overall normalization of 1/n between the two transforms. This is required to make irfftn() the exact inverse.
Default is "backward" (normalize by 1/n).
Without specifying the output length to irfft(), the output will not round-trip properly because the input is odd-length in the last dimension:
So, it is recommended to always pass the signal shape s.


Computes the one dimensional Fourier transform of real-valued input.
The FFT of a real signal is Hermitian-symmetric, X[i] = conj(X[-i]) so the output contains only the positive frequencies below the Nyquist frequency. To compute the full output, use fft()
norm (str, optional) –
Normalization mode. For the forward transform (rfft()), these correspond to:
Calling the backward transform (irfft()) with the same normalization mode will apply an overall normalization of 1/n between the two transforms. This is required to make irfft() the exact inverse.
Default is "backward" (no normalization).
Compare against the full output from fft():
Notice that the symmetric element T[-1] == T[1].conj() is omitted. At the Nyquist frequency T[-2] == T[2] is it’s own symmetric pair, and therefore must always be real-valued.


Computes the 2-dimensional discrete Fourier transform of real input. Equivalent to rfftn() but FFTs only the last two dimensions by default.
The FFT of a real signal is Hermitian-symmetric, X[i, j] = conj(X[-i, -j]), so the full fft2() output contains redundant information. rfft2() instead omits the negative frequencies in the last dimension.
norm (str, optional) –
Normalization mode. For the forward transform (rfft2()), these correspond to:
Where n = prod(s) is the logical FFT size. Calling the backward transform (irfft2()) with the same normalization mode will apply an overall normalization of 1/n between the two transforms. This is required to make irfft2() the exact inverse.
Default is "backward" (no normalization).
Compared against the full output from fft2(), we have all elements up to the Nyquist frequency.
The discrete Fourier transform is separable, so rfft2() here is equivalent to a combination of fft() and rfft():


Computes the sample frequencies for rfft() with a signal of size n.
Note
rfft() returns Hermitian one-sided output, so only the positive frequency terms are returned. For a real FFT of length n and with inputs spaced in length unit d, the frequencies are:
Note
For even lengths, the Nyquist frequency at f[n/2] can be thought of as either negative or positive. Unlike fftfreq(), rfftfreq() always returns it as positive.
Compared to the output from fftfreq(), we see that the Nyquist frequency at f[2] has changed sign: >>> torch.fft.fftfreq(4) tensor([ 0.0000, 0.2500, -0.5000, -0.2500])


Computes the N-dimensional discrete Fourier transform of real input.
The FFT of a real signal is Hermitian-symmetric, X[i_1, ..., i_n] = conj(X[-i_1, ..., -i_n]) so the full fftn() output contains redundant information. rfftn() instead omits the negative frequencies in the last dimension.
norm (str, optional) –
Normalization mode. For the forward transform (rfftn()), these correspond to:
Where n = prod(s) is the logical FFT size. Calling the backward transform (irfftn()) with the same normalization mode will apply an overall normalization of 1/n between the two transforms. This is required to make irfftn() the exact inverse.
Default is "backward" (no normalization).
Compared against the full output from fftn(), we have all elements up to the Nyquist frequency.
The discrete Fourier transform is separable, so rfftn() here is equivalent to a combination of fft() and rfft():


Flattens input by reshaping it into a one-dimensional tensor. If start_dim or end_dim are passed, only dimensions starting with start_dim and ending with end_dim are flattened. The order of elements in input is unchanged.
Unlike NumPy’s flatten, which always copies input’s data, this function may return the original object, a view, or copy. If no dimensions are flattened, then the original object input is returned. Otherwise, if input can be viewed as the flattened shape, then that view is returned. Finally, only if the input cannot be viewed as the flattened shape is input’s data copied. See torch.Tensor.view() for details on when a view will be returned.
Note
Flattening a zero-dimensional tensor will return a one-dimensional view.
Example:


Reverse the order of a n-D tensor along given axis in dims.
Note
torch.flip makes a copy of input’s data. This is different from NumPy’s np.flip, which returns a view in constant time. Since copying a tensor’s data is more work than viewing that data, torch.flip is expected to be slower than np.flip.
Example:


Flip tensor in the left/right direction, returning a new tensor.
Flip the entries in each row in the left/right direction. Columns are preserved, but appear in a different order than before.
Note
Requires the tensor to be at least 2-D.
Note
torch.fliplr makes a copy of input’s data. This is different from NumPy’s np.fliplr, which returns a view in constant time. Since copying a tensor’s data is more work than viewing that data, torch.fliplr is expected to be slower than np.fliplr.
input (Tensor) – Must be at least 2-dimensional.
Example:


Flip tensor in the up/down direction, returning a new tensor.
Flip the entries in each column in the up/down direction. Rows are preserved, but appear in a different order than before.
Note
Requires the tensor to be at least 1-D.
Note
torch.flipud makes a copy of input’s data. This is different from NumPy’s np.flipud, which returns a view in constant time. Since copying a tensor’s data is more work than viewing that data, torch.flipud is expected to be slower than np.flipud.
input (Tensor) – Must be at least 1-dimensional.
Example: