Computes the product along segments of a tensor.
Compat aliases for migration
See Migration guide for more details.
tf.compat.v1.math.segment_prod, tf.compat.v1.segment_prod
Read the section on segmentation for an explanation of segments.
Computes a tensor such that \(output_i = \prod_j data_j\) where the product is over j such that segment_ids[j] == i.
If the product is empty for a given segment ID i, output[i] = 1.


Computes the sum along segments of a tensor.
Compat aliases for migration
See Migration guide for more details.
tf.compat.v1.math.segment_sum, tf.compat.v1.segment_sum
Read the section on segmentation for an explanation of segments.
Computes a tensor such that \(output_i = \sum_j data_j\) where sum is over j such that segment_ids[j] == i.
If the sum is empty for a given segment ID i, output[i] = 0.


Computes sigmoid of x element-wise.
Main aliases
tf.nn.sigmoid, tf.sigmoid
See Migration guide for more details.
tf.compat.v1.math.sigmoid, tf.compat.v1.nn.sigmoid, tf.compat.v1.sigmoid
Formula for calculating $\mathrm{sigmoid}(x) = y = 1 / (1 + \exp(-x))$.
For $x \in (-\infty, \infty)$, $\mathrm{sigmoid}(x) \in (0, 1)$.
If a positive number is large, then its sigmoid will approach to 1 since the formula will be y = <large_num> / (1 + <large_num>)
If a negative number is large, its sigmoid will approach to 0 since the formula will be y = 1 / (1 + <large_num>)
Equivalent to scipy.special.expit


Returns an element-wise indication of the sign of a number.
Main aliases
tf.sign
See Migration guide for more details.
tf.compat.v1.math.sign, tf.compat.v1.sign
y = sign(x) = -1 if x < 0; 0 if x == 0; 1 if x > 0.
For complex numbers, y = sign(x) = x / |x| if x != 0, otherwise y = 0.
If x is a SparseTensor, returns SparseTensor(x.indices, tf.math.sign(x.values, ...), x.dense_shape).
If x is a SparseTensor, returns SparseTensor(x.indices, tf.math.sign(x.values, ...), x.dense_shape)


Computes sine of x element-wise.
Main aliases
tf.sin
See Migration guide for more details.
tf.compat.v1.math.sin, tf.compat.v1.sin
Given an input tensor, this function computes sine of every element in the tensor. Input range is (-inf, inf) and output range is [-1,1].


Computes hyperbolic sine of x element-wise.
Main aliases
tf.sinh
See Migration guide for more details.
tf.compat.v1.math.sinh, tf.compat.v1.sinh
Given an input tensor, this function computes hyperbolic sine of every element in the tensor. Input range is [-inf,inf] and output range is [-inf,inf].


Generates points from the Sobol sequence.
Compat aliases for migration
See Migration guide for more details.
tf.compat.v1.math.sobol_sample
Creates a Sobol sequence with num_results samples. Each sample has dimension dim. Skips the first skip samples.


Computes softplus: log(exp(features) + 1).
Main aliases
tf.nn.softplus
See Migration guide for more details.
tf.compat.v1.math.softplus, tf.compat.v1.nn.softplus


Public API for tf.math.special namespace.
bessel_i0(...): Computes the Bessel i0 function of x element-wise.
bessel_i0e(...): Computes the Bessel i0e function of x element-wise.
bessel_i1(...): Computes the Bessel i1 function of x element-wise.
bessel_i1e(...): Computes the Bessel i1e function of x element-wise.
bessel_j0(...): Computes the Bessel j0 function of x element-wise.
bessel_j1(...): Computes the Bessel j1 function of x element-wise.
bessel_k0(...): Computes the Bessel k0 function of x element-wise.
bessel_k0e(...): Computes the Bessel k0e function of x element-wise.
bessel_k1(...): Computes the Bessel k1 function of x element-wise.
bessel_k1e(...): Computes the Bessel k1e function of x element-wise.
bessel_y0(...): Computes the Bessel y0 function of x element-wise.
bessel_y1(...): Computes the Bessel y1 function of x element-wise.
dawsn(...): Computes Dawson's integral of x element-wise.
expint(...): Computes the Exponential integral of x element-wise.
fresnel_cos(...): Computes Fresnel's cosine integral of x element-wise.
fresnel_sin(...): Computes Fresnel's sine integral of x element-wise.
spence(...): Computes Spence's integral of x element-wise.


Computes the Bessel j0 function of x element-wise.
Compat aliases for migration
See Migration guide for more details.
tf.compat.v1.math.special.bessel_j0
Modified Bessel function of order 0.
Equivalent to scipy.special.j0


Computes the Bessel j1 function of x element-wise.
Compat aliases for migration
See Migration guide for more details.
tf.compat.v1.math.special.bessel_j1
Modified Bessel function of order 1.
Equivalent to scipy.special.j1


Computes the Bessel k0 function of x element-wise.
Compat aliases for migration
See Migration guide for more details.
tf.compat.v1.math.special.bessel_k0
Modified Bessel function of order 0.
It is preferable to use the numerically stabler function k0e(x) instead.
Equivalent to scipy.special.k0


Computes the Bessel k0e function of x element-wise.
Compat aliases for migration
See Migration guide for more details.
tf.compat.v1.math.special.bessel_k0e
Modified Bessel function of order 0.
Equivalent to scipy.special.k0e


Computes the Bessel k1 function of x element-wise.
Compat aliases for migration
See Migration guide for more details.
tf.compat.v1.math.special.bessel_k1
Modified Bessel function of order 1.
It is preferable to use the numerically stabler function k1e(x) instead.
Equivalent to scipy.special.k1


Computes the Bessel k1e function of x element-wise.
Compat aliases for migration
See Migration guide for more details.
tf.compat.v1.math.special.bessel_k1e
Modified Bessel function of order 1.
Equivalent to scipy.special.k1e


Computes the Bessel y0 function of x element-wise.
Compat aliases for migration
See Migration guide for more details.
tf.compat.v1.math.special.bessel_y0
Modified Bessel function of order 0.
Equivalent to scipy.special.y0


Computes the Bessel y1 function of x element-wise.
Compat aliases for migration
See Migration guide for more details.
tf.compat.v1.math.special.bessel_y1
Modified Bessel function of order 1.
Equivalent to scipy.special.y1


Computes Dawson's integral of x element-wise.
Compat aliases for migration
See Migration guide for more details.
tf.compat.v1.math.special.dawsn
Dawson's integral is defined as exp(-x**2) times the integral of exp(t**2) from 0 to x, with the domain of definition all real numbers.
Dawson's function is odd.
This implementation is based off of the Cephes math library.
Equivalent to scipy.special.dawsn


Computes the Exponential integral of x element-wise.
Compat aliases for migration
See Migration guide for more details.
tf.compat.v1.math.special.expint
The Exponential integral is defined as the integral of exp(t) / t from -inf to x, with the domain of definition all positive real numbers.
This implementation is based off of the Cephes math library.
Equivalent to scipy.special.expi


Computes Fresnel's cosine integral of x element-wise.
Compat aliases for migration
See Migration guide for more details.
tf.compat.v1.math.special.fresnel_cos
The Fresnel cosine integral is defined as the integral of cos(t^2) from 0 to x, with the domain of definition all real numbers.
The Fresnel cosine integral is odd.
This implementation is based off of the Cephes math library.
Equivalent to scipy.special.fresnel second output.


Computes Fresnel's sine integral of x element-wise.
Compat aliases for migration
See Migration guide for more details.
tf.compat.v1.math.special.fresnel_sin
The Fresnel sine integral is defined as the integral of sin(t^2) from 0 to x, with the domain of definition all real numbers.
This implementation is based off of the Cephes math library.
Equivalent to scipy.special.fresnel first output.


Computes Spence's integral of x element-wise.
Compat aliases for migration
See Migration guide for more details.
tf.compat.v1.math.special.spence
Spence's integral is defined as the integral of log(t) / (1 - t) from 1 to x, with the domain of definition all non-negative real numbers.
This implementation is based off of the Cephes math library.
Equivalent to scipy.special.spence


Computes element-wise square root of the input tensor.
Main aliases
tf.sqrt
See Migration guide for more details.
tf.compat.v1.math.sqrt, tf.compat.v1.sqrt
If x is a SparseTensor, returns SparseTensor(x.indices, tf.math.sqrt(x.values, ...), x.dense_shape)


Computes square of x element-wise.
Main aliases
tf.square
See Migration guide for more details.
tf.compat.v1.math.square, tf.compat.v1.square
I.e., \(y = x * x = x^2\).
If x is a SparseTensor, returns SparseTensor(x.indices, tf.math.square(x.values, ...), x.dense_shape)


Returns conj(x - y)(x - y) element-wise.
Compat aliases for migration
See Migration guide for more details.
tf.compat.v1.math.squared_difference, tf.compat.v1.squared_difference


Returns x - y element-wise.
Main aliases
tf.subtract
See Migration guide for more details.
tf.compat.v1.math.subtract, tf.compat.v1.subtract


Computes tan of x element-wise.
Main aliases
tf.tan
See Migration guide for more details.
tf.compat.v1.math.tan, tf.compat.v1.tan
Given an input tensor, this function computes tangent of every element in the tensor. Input range is (-inf, inf) and output range is (-inf, inf). If input lies outside the boundary, nan is returned.


Computes hyperbolic tangent of x element-wise.
Main aliases
tf.nn.tanh, tf.tanh
See Migration guide for more details.
tf.compat.v1.math.tanh, tf.compat.v1.nn.tanh, tf.compat.v1.tanh
Given an input tensor, this function computes hyperbolic tangent of every element in the tensor. Input range is [-inf, inf] and output range is [-1,1].
If x is a SparseTensor, returns SparseTensor(x.indices, tf.math.tanh(x.values, ...), x.dense_shape)


Finds values and indices of the k largest entries for the last dimension.
Main aliases
tf.nn.top_k
See Migration guide for more details.
tf.compat.v1.math.top_k, tf.compat.v1.nn.top_k
If the input is a vector (rank=1), finds the k largest entries in the vector and outputs their values and indices as vectors. Thus values[j] is the j-th largest entry in input, and its index is indices[j].
For matrices (resp. higher rank input), computes the top k entries in each row (resp. vector along the last dimension). Thus,
If two elements are equal, the lower-index element appears first.


Divides x / y elementwise (using Python 3 division operator semantics).
Main aliases
tf.truediv
See Migration guide for more details.
tf.compat.v1.math.truediv, tf.compat.v1.truediv
This function forces Python 3 division operator semantics where all integer arguments are cast to floating types first. This op is generated by normal x / y division in Python 3 and in Python 2.7 with from __future__ import division. If you want integer division that rounds down, use x // y or tf.math.floordiv.
x and y must have the same numeric type. If the inputs are floating point, the output will have the same type. If the inputs are integral, the inputs are cast to float32 for int8 and int16 and float64 for int32 and int64 (matching the behavior of Numpy).


Computes the maximum along segments of a tensor.
Compat aliases for migration
See Migration guide for more details.
tf.compat.v1.math.unsorted_segment_max, tf.compat.v1.unsorted_segment_max
Read the section on segmentation for an explanation of segments.
This operator is similar to the unsorted segment sum operator found (here). Instead of computing the sum over segments, it computes the maximum such that:
\(output_i = \max_{j...} data[j...]\) where max is over tuples j... such that segment_ids[j...] == i.
If the maximum is empty for a given segment ID i, it outputs the smallest possible value for the specific numeric type, output[i] = numeric_limits<T>::lowest().
If the given segment ID i is negative, then the corresponding value is dropped, and will not be included in the result.


Computes the mean along segments of a tensor.
Compat aliases for migration
See Migration guide for more details.
tf.compat.v1.math.unsorted_segment_mean, tf.compat.v1.unsorted_segment_mean
Read the section on segmentation for an explanation of segments.
This operator is similar to the tf.math.unsorted_segment_sum operator. Instead of computing the sum over segments, it computes the mean of all entries belonging to a segment such that:
\(output_i = 1/N_i \sum_{j...} data[j...]\) where the sum is over tuples j... such that segment_ids[j...] == i with \N_i\ being the number of occurrences of id \i\.
If there is no entry for a given segment ID i, it outputs 0.
If the given segment ID i is negative, the value is dropped and will not be added to the sum of the segment.


Computes the minimum along segments of a tensor.
Compat aliases for migration
See Migration guide for more details.
tf.compat.v1.math.unsorted_segment_min, tf.compat.v1.unsorted_segment_min
Read the section on segmentation for an explanation of segments.
This operator is similar to the unsorted segment sum operator found (here). Instead of computing the sum over segments, it computes the minimum such that:
\(output_i = \min_{j...} data_[j...]\) where min is over tuples j... such that segment_ids[j...] == i.
If the minimum is empty for a given segment ID i, it outputs the largest possible value for the specific numeric type, output[i] = numeric_limits<T>::max().
If the given segment ID i is negative, then the corresponding value is dropped, and will not be included in the result.


Computes the product along segments of a tensor.
Compat aliases for migration
See Migration guide for more details.
tf.compat.v1.math.unsorted_segment_prod, tf.compat.v1.unsorted_segment_prod
Read the section on segmentation for an explanation of segments.
This operator is similar to the unsorted segment sum operator found (here). Instead of computing the sum over segments, it computes the product of all entries belonging to a segment such that:
\(output_i = \prod_{j...} data[j...]\) where the product is over tuples j... such that segment_ids[j...] == i.
If there is no entry for a given segment ID i, it outputs 1.
If the given segment ID i is negative, then the corresponding value is dropped, and will not be included in the result.


Computes the sum along segments of a tensor divided by the sqrt(N).
Compat aliases for migration
See Migration guide for more details.
tf.compat.v1.math.unsorted_segment_sqrt_n, tf.compat.v1.unsorted_segment_sqrt_n
Read the section on segmentation for an explanation of segments.
This operator is similar to the tf.math.unsorted_segment_sum operator. Additionally to computing the sum over segments, it divides the results by sqrt(N).
\(output_i = 1/sqrt(N_i) \sum_{j...} data[j...]\) where the sum is over tuples j... such that segment_ids[j...] == i with \N_i\ being the number of occurrences of id \i\.
If there is no entry for a given segment ID i, it outputs 0.
Note that this op only supports floating point and complex dtypes, due to tf.sqrt only supporting these types.
If the given segment ID i is negative, the value is dropped and will not be added to the sum of the segment.


Computes the sum along segments of a tensor.
Compat aliases for migration
See Migration guide for more details.
tf.compat.v1.math.unsorted_segment_sum, tf.compat.v1.unsorted_segment_sum
Read the section on segmentation for an explanation of segments.
Computes a tensor such that \(output[i] = \sum_{j...} data[j...]\) where the sum is over tuples j... such that segment_ids[j...] == i. Unlike SegmentSum, segment_ids need not be sorted and need not cover all values in the full range of valid values.
If the sum is empty for a given segment ID i, output[i] = 0. If the given segment ID i is negative, the value is dropped and will not be added to the sum of the segment.
num_segments should equal the number of distinct segment IDs.


Returns 0 if x == 0, and x / y otherwise, elementwise.
Compat aliases for migration
See Migration guide for more details.
tf.compat.v1.math.xdivy


Compute x * log1p(y).
Compat aliases for migration
See Migration guide for more details.
tf.compat.v1.math.xlog1py
Given x and y, compute x * log1p(y). This function safely returns zero when x = 0, no matter what the value of y is.
Equivalent to scipy.special.xlog1py


Returns 0 if x == 0, and x * log(y) otherwise, elementwise.
Compat aliases for migration
See Migration guide for more details.
tf.compat.v1.math.xlogy


Returns the fraction of zeros in value.
Main aliases
tf.nn.zero_fraction
See Migration guide for more details.
tf.compat.v1.math.zero_fraction, tf.compat.v1.nn.zero_fraction
If value is empty, the result is nan.
This is useful in summaries to measure and report sparsity. For example,


Compute the Hurwitz zeta function \(\zeta(x, q)\).
Compat aliases for migration
See Migration guide for more details.
tf.compat.v1.math.zeta, tf.compat.v1.zeta
The Hurwitz zeta function is defined as:
\(\zeta(x, q) = \sum_{n=0}^{\infty} (q + n)^{-x}\)


Broadcasts parameters for evaluation on an N-D grid.
Compat aliases for migration
See Migration guide for more details.
tf.compat.v1.meshgrid
Given N one-dimensional coordinate arrays *args, returns a list outputs of N-D coordinate arrays for evaluating expressions on an N-D grid.
meshgrid supports cartesian ('xy') and matrix ('ij') indexing conventions. When the indexing argument is set to 'xy' (the default), the broadcasting instructions for the first two dimensions are swapped.
Calling X, Y = meshgrid(x, y) with the tensors


Public API for tf.mixed_precision namespace.
experimental module: Public API for tf.mixed_precision.experimental namespace.


Public API for tf.mixed_precision.experimental namespace.
class DynamicLossScale: Loss scale that dynamically adjusts itself.
class FixedLossScale: Loss scale with a fixed value.
class LossScale: Base class for all TF1 loss scales.


Loss scale that dynamically adjusts itself.
Inherits From: LossScale
Main aliases
tf.train.experimental.DynamicLossScale
See Migration guide for more details.
tf.compat.v1.mixed_precision.DynamicLossScale, tf.compat.v1.mixed_precision.experimental.DynamicLossScale, tf.compat.v1.train.experimental.DynamicLossScale
Dynamic loss scaling works by adjusting the loss scale as training progresses. The goal is to keep the loss scale as high as possible without overflowing the gradients. As long as the gradients do not overflow, raising the loss scale never hurts.
The algorithm starts by setting the loss scale to an initial value. Every N steps that the gradients are finite, the loss scale is increased by some factor. However, if a NaN or Inf gradient is found, the gradients for that step are not applied, and the loss scale is decreased by the factor. This process tends to keep the loss scale as high as possible without gradients overflowing.
View source
Creates the LossScale from its config.
View source
Returns the config of this loss scale.
View source
Updates loss scale based on if gradients are finite in current step.
View source
Returns the current loss scale as a scalar float32 tensor.


Loss scale with a fixed value.
Inherits From: LossScale
Main aliases
tf.train.experimental.FixedLossScale
See Migration guide for more details.
tf.compat.v1.mixed_precision.FixedLossScale, tf.compat.v1.mixed_precision.experimental.FixedLossScale, tf.compat.v1.train.experimental.FixedLossScale
The loss scale is not updated for the lifetime of instances of this class. A given instance of this class always returns the same number when called.
View source
Creates the LossScale from its config.
View source
Returns the config of this loss scale.
View source
Updates the value of the loss scale.
The loss scale will be potentially updated, based on the value of grads. The tensor returned by calling this class is only updated when this function is evaluated.
In eager mode, this directly updates the loss scale, so that calling __call__ will return the newly updated loss scale. In graph mode, this returns an op that, when evaluated, updates the loss scale.
This function also returns a should_apply_gradients bool. If False, gradients should not be applied to the variables that step, as nonfinite gradients were found, and the loss scale has been be updated to reduce the chance of finding nonfinite gradients in the next step. Some loss scale classes will always return True, as they cannot adjust themselves in response to nonfinite gradients.
When a DistributionStrategy is used, this function may only be called in a cross-replica context.
View source
Returns the current loss scale as a scalar float32 tensor.


Base class for all TF1 loss scales.
Main aliases
tf.train.experimental.LossScale
See Migration guide for more details.
tf.compat.v1.mixed_precision.LossScale, tf.compat.v1.mixed_precision.experimental.LossScale, tf.compat.v1.train.experimental.LossScale
This is an abstract base class, so you cannot instantiate it directly. Instead, use one of its concrete subclasses:
Loss scaling is a process that multiplies the loss by a multiplier called the loss scale, and divides each gradient by the same multiplier. The pseudocode for this process is:
Mathematically, loss scaling has no effect, but can help avoid numerical underflow in intermediate gradients when float16 tensors are used for mixed precision training. By multiplying the loss, each intermediate gradient will have the same multiplier applied.
Instances of this class represent a loss scale. Calling instances of this class returns the loss scale as a scalar float32 tensor, while method update() updates the loss scale depending on the values of the gradients. Optimizers use instances of this class to scale loss and gradients.
In most functions that accept a LossScale, you can also pass an int (such as
8) to create a FixedLossScale or the string "dynamic" to create a dynamic loss scale.
View source
Creates the LossScale from its config.
View source
Returns the config of this loss scale.
View source
Updates the value of the loss scale.
The loss scale will be potentially updated, based on the value of grads. The tensor returned by calling this class is only updated when this function is evaluated.
In eager mode, this directly updates the loss scale, so that calling __call__ will return the newly updated loss scale. In graph mode, this returns an op that, when evaluated, updates the loss scale.
This function also returns a should_apply_gradients bool. If False, gradients should not be applied to the variables that step, as nonfinite gradients were found, and the loss scale has been be updated to reduce the chance of finding nonfinite gradients in the next step. Some loss scale classes will always return True, as they cannot adjust themselves in response to nonfinite gradients.
When a DistributionStrategy is used, this function may only be called in a cross-replica context.
View source
Returns the current loss scale as a scalar float32 tensor.


Public API for tf.mlir namespace.
experimental module: Public API for tf.mlir.experimental namespace.


Public API for tf.mlir.experimental namespace.
convert_function(...): Import a ConcreteFunction and convert it to a textual MLIR module.
convert_graph_def(...): Import a GraphDef and convert it to a textual MLIR module.


Import a ConcreteFunction and convert it to a textual MLIR module.
Compat aliases for migration
See Migration guide for more details.
tf.compat.v1.mlir.experimental.convert_function
This API is only intended for inspecting the internals of TensorFlow and the string returned is at the moment intended for debugging purposes.
A tf.function can be imported and converted from TensorFlow to TensorFlow MLIR with this API by extracting its ConcreteFunction (eagerly-executing wrapper around a tf.Graph).


Import a GraphDef and convert it to a textual MLIR module.
Compat aliases for migration
See Migration guide for more details.
tf.compat.v1.mlir.experimental.convert_graph_def
This API is only intended for inspecting the internals of TensorFlow and the string returned is at the moment intended for debugging purposes.


Base neural network module class.
Compat aliases for migration
See Migration guide for more details.
tf.compat.v1.Module
A module is a named container for tf.Variables, other tf.Modules and functions which apply to user input. For example a dense layer in a neural network might be implemented as a tf.Module:
You can use the Dense layer as you would expect:
By subclassing tf.Module instead of object any tf.Variable or tf.Module instances assigned to object properties can be collected using the variables, trainable_variables or submodules property:
Subclasses of tf.Module can also take advantage of the _flatten method which can be used to implement tracking of any other types.
All tf.Module classes have an associated tf.name_scope which can be used to group operations in TensorBoard and create hierarchies for variable names which can help with debugging. We suggest using the name scope when creating nested submodules/parameters or for forward methods whose graph you might want to inspect in TensorBoard. You can enter the name scope explicitly using with self.name_scope: or you can annotate methods (apart from __init__) with @tf.Module.with_name_scope.
Submodules are modules which are properties of this module, or found as properties of modules which are properties of this module (and so on).
View source
Decorator to automatically enter the module name scope.
Using the above module would produce tf.Variables and tf.Tensors whose names included the module name:


A context manager for use when defining a Python op.
This context manager pushes a name scope, which will make the name of all operations added within it have a prefix.
For example, to define a new Python op called my_op:
When executed, the Tensors a, b, c, will have names MyOp/a, MyOp/b, and MyOp/c.
Inside a tf.function, if the scope name already exists, the name will be made unique by appending _n. For example, calling my_op the second time will generate MyOp_1/a, etc.
View source
Start the scope block.
View source


Public API for tf.nest namespace.
assert_same_structure(...): Asserts that two structures are nested in the same way.
flatten(...): Returns a flat list from a given nested structure.
is_nested(...): Returns true if its input is a collections.abc.Sequence (except strings).
map_structure(...): Applies func to each entry in structure and returns a new structure.
pack_sequence_as(...): Returns a given flattened sequence packed into a given structure.


Asserts that two structures are nested in the same way.
Compat aliases for migration
See Migration guide for more details.
tf.compat.v1.nest.assert_same_structure
Note that namedtuples with identical name and fields are always considered to have the same shallow structure (even with check_types=True). For instance, this code will print True:


Returns a flat list from a given nested structure.
Compat aliases for migration
See Migration guide for more details.
tf.compat.v1.nest.flatten
If nest is not a structure , tuple (or a namedtuple), dict, or an attrs class, then returns a single-element list: [nest].
In the case of dict instances, the sequence consists of the values, sorted by key to ensure deterministic behavior. This is true also for OrderedDict instances: their sequence order is ignored, the sorting order of keys is used instead. The same convention is followed in pack_sequence_as. This correctly repacks dicts and OrderedDicts after they have been flattened, and also allows flattening an OrderedDict and then repacking it back using a corresponding plain dict, or vice-versa. Dictionaries with non-sortable keys cannot be flattened.
Users must not modify any collections used in nest while this function is running.


Returns true if its input is a collections.abc.Sequence (except strings).
Compat aliases for migration
See Migration guide for more details.
tf.compat.v1.nest.is_nested


Applies func to each entry in structure and returns a new structure.
Compat aliases for migration
See Migration guide for more details.
tf.compat.v1.nest.map_structure
Applies func(x[0], x[1], ...) where x[i] is an entry in structure[i]. All structures in structure must have the same arity, and the return value will contain results with the same structure layout.


Returns a given flattened sequence packed into a given structure.
Compat aliases for migration
See Migration guide for more details.
tf.compat.v1.nest.pack_sequence_as
If structure is a scalar, flat_sequence must be a single-element list; in this case the return value is flat_sequence[0].
If structure is or contains a dict instance, the keys will be sorted to pack the flat sequence in deterministic order. This is true also for OrderedDict instances: their sequence order is ignored, the sorting order of keys is used instead. The same convention is followed in flatten. This correctly repacks dicts and OrderedDicts after they have been flattened, and also allows flattening an OrderedDict and then repacking it back using a corresponding plain dict, or vice-versa. Dictionaries with non-sortable keys cannot be flattened.


Wrappers for primitive Neural Net (NN) Operations.
class RNNCellDeviceWrapper: Operator that ensures an RNNCell runs on a particular device.
class RNNCellDropoutWrapper: Operator adding dropout to inputs and outputs of the given cell.
class RNNCellResidualWrapper: RNNCell wrapper that ensures cell inputs are added to the outputs.
all_candidate_sampler(...): Generate the set of all classes.
atrous_conv2d(...): Atrous convolution (a.k.a. convolution with holes or dilated convolution).
atrous_conv2d_transpose(...): The transpose of atrous_conv2d.
avg_pool(...): Performs the avg pooling on the input.
avg_pool1d(...): Performs the average pooling on the input.
avg_pool2d(...): Performs the average pooling on the input.
avg_pool3d(...): Performs the average pooling on the input.
batch_norm_with_global_normalization(...): Batch normalization.
batch_normalization(...): Batch normalization.
bias_add(...): Adds bias to value.
collapse_repeated(...): Merge repeated labels into single labels.
compute_accidental_hits(...): Compute the position ids in sampled_candidates matching true_classes.
compute_average_loss(...): Scales per-example losses with sample_weights and computes their average.
conv1d(...): Computes a 1-D convolution given 3-D input and filter tensors.
conv1d_transpose(...): The transpose of conv1d.
conv2d(...): Computes a 2-D convolution given input and 4-D filters tensors.
conv2d_transpose(...): The transpose of conv2d.
conv3d(...): Computes a 3-D convolution given 5-D input and filters tensors.
conv3d_transpose(...): The transpose of conv3d.
conv_transpose(...): The transpose of convolution.
convolution(...): Computes sums of N-D convolutions (actually cross-correlation).
crelu(...): Computes Concatenated ReLU.
ctc_beam_search_decoder(...): Performs beam search decoding on the logits given in input.
ctc_greedy_decoder(...): Performs greedy decoding on the logits given in input (best path).
ctc_loss(...): Computes CTC (Connectionist Temporal Classification) loss.
ctc_unique_labels(...): Get unique labels and indices for batched labels for tf.nn.ctc_loss.
depth_to_space(...): DepthToSpace for tensors of type T.
depthwise_conv2d(...): Depthwise 2-D convolution.
depthwise_conv2d_backprop_filter(...): Computes the gradients of depthwise convolution with respect to the filter.
depthwise_conv2d_backprop_input(...): Computes the gradients of depthwise convolution with respect to the input.
dilation2d(...): Computes the grayscale dilation of 4-D input and 3-D filters tensors.
dropout(...): Computes dropout: randomly sets elements to zero to prevent overfitting.
elu(...): Computes exponential linear: exp(features) - 1 if < 0, features otherwise.
embedding_lookup(...): Looks up embeddings for the given ids from a list of tensors.
embedding_lookup_sparse(...): Looks up embeddings for the given ids and weights from a list of tensors.
erosion2d(...): Computes the grayscale erosion of 4-D value and 3-D filters tensors.
fixed_unigram_candidate_sampler(...): Samples a set of classes using the provided (fixed) base distribution.
fractional_avg_pool(...): Performs fractional average pooling on the input.
fractional_max_pool(...): Performs fractional max pooling on the input.
gelu(...): Compute the Gaussian Error Linear Unit (GELU) activation function.
in_top_k(...): Says whether the targets are in the top K predictions.
isotonic_regression(...): Solves isotonic regression problems along the given axis.
l2_loss(...): L2 Loss.
l2_normalize(...): Normalizes along dimension axis using an L2 norm.
leaky_relu(...): Compute the Leaky ReLU activation function.
learned_unigram_candidate_sampler(...): Samples a set of classes from a distribution learned during training.
local_response_normalization(...): Local Response Normalization.
log_poisson_loss(...): Computes log Poisson loss given log_input.
log_softmax(...): Computes log softmax activations.
lrn(...): Local Response Normalization.
max_pool(...): Performs the max pooling on the input.
max_pool1d(...): Performs the max pooling on the input.
max_pool2d(...): Performs the max pooling on the input.
max_pool3d(...): Performs the max pooling on the input.
max_pool_with_argmax(...): Performs max pooling on the input and outputs both max values and indices.
moments(...): Calculates the mean and variance of x.
nce_loss(...): Computes and returns the noise-contrastive estimation training loss.
normalize_moments(...): Calculate the mean and variance of based on the sufficient statistics.
pool(...): Performs an N-D pooling operation.
relu(...): Computes rectified linear: max(features, 0).
relu6(...): Computes Rectified Linear 6: min(max(features, 0), 6).
safe_embedding_lookup_sparse(...): Lookup embedding results, accounting for invalid IDs and empty features.
sampled_softmax_loss(...): Computes and returns the sampled softmax training loss.
scale_regularization_loss(...): Scales the sum of the given regularization losses by number of replicas.
selu(...): Computes scaled exponential linear: scale * alpha * (exp(features) - 1)
separable_conv2d(...): 2-D convolution with separable filters.
sigmoid(...): Computes sigmoid of x element-wise.
sigmoid_cross_entropy_with_logits(...): Computes sigmoid cross entropy given logits.
silu(...): Computes the SiLU or Swish activation function: x * sigmoid(x).
softmax(...): Computes softmax activations.
softmax_cross_entropy_with_logits(...): Computes softmax cross entropy between logits and labels.
softplus(...): Computes softplus: log(exp(features) + 1).
softsign(...): Computes softsign: features / (abs(features) + 1).
space_to_batch(...): SpaceToBatch for N-D tensors of type T.
space_to_depth(...): SpaceToDepth for tensors of type T.
sparse_softmax_cross_entropy_with_logits(...): Computes sparse softmax cross entropy between logits and labels.
sufficient_statistics(...): Calculate the sufficient statistics for the mean and variance of x.
swish(...): Computes the SiLU or Swish activation function: x * sigmoid(x).
tanh(...): Computes hyperbolic tangent of x element-wise.
top_k(...): Finds values and indices of the k largest entries for the last dimension.
weighted_cross_entropy_with_logits(...): Computes a weighted cross entropy.
weighted_moments(...): Returns the frequency-weighted mean and variance of x.
with_space_to_batch(...): Performs op on the space-to-batch representation of input.
zero_fraction(...): Returns the fraction of zeros in value.


Atrous convolution (a.k.a. convolution with holes or dilated convolution).
Compat aliases for migration
See Migration guide for more details.
tf.compat.v1.nn.atrous_conv2d
This function is a simpler wrapper around the more general tf.nn.convolution, and exists only for backwards compatibility. You can use tf.nn.convolution to perform 1-D, 2-D, or 3-D atrous convolution.
Computes a 2-D atrous convolution, also known as convolution with holes or dilated convolution, given 4-D value and filters tensors. If the rate parameter is equal to one, it performs regular 2-D convolution. If the rate parameter is greater than one, it performs convolution with holes, sampling the input values every rate pixels in the height and width dimensions. This is equivalent to convolving the input with a set of upsampled filters, produced by inserting rate - 1 zeros between two consecutive values of the filters along the height and width dimensions, hence the name atrous convolution or convolution with holes (the French word trous means holes in English).
Atrous convolution allows us to explicitly control how densely to compute feature responses in fully convolutional networks. Used in conjunction with bilinear interpolation, it offers an alternative to conv2d_transpose in dense prediction tasks such as semantic image segmentation, optical flow computation, or depth estimation. It also allows us to effectively enlarge the field of view of filters without increasing the number of parameters or the amount of computation.
For a description of atrous convolution and how it can be used for dense feature extraction, please see: (Chen et al., 2015). The same operation is investigated further in (Yu et al., 2016). Previous works that effectively use atrous convolution in different ways are, among others, (Sermanet et al., 2014) and (Giusti et al., 2013). Atrous convolution is also closely related to the so-called noble identities in multi-rate signal processing.
There are many different ways to implement atrous convolution (see the refs above). The implementation here reduces
to the following three operations:
Advanced usage. Note the following optimization: A sequence of atrous_conv2d operations with identical rate parameters, 'SAME' padding, and filters with odd heights/ widths:
can be equivalently performed cheaper in terms of computation and memory as:
because a pair of consecutive space_to_batch and batch_to_space ops with the same block_size cancel out when their respective paddings and crops inputs are identical.
[batch, height - 2 * (filter_width - 1), width - 2 * (filter_height - 1), out_channels].
Output shape with 'SAME' padding is:
[batch, height, width, out_channels].
Multi-Scale Context Aggregation by Dilated Convolutions: Yu et al., 2016 (pdf) Semantic Image Segmentation with Deep Convolutional Nets and Fully Connected CRFs: Chen et al., 2015 (pdf) OverFeat - Integrated Recognition, Localization and Detection using Convolutional Networks: Sermanet et al., 2014 (pdf) Fast Image Scanning with Deep Max-Pooling Convolutional Neural Networks: Giusti et al., 2013 (pdf)


The transpose of atrous_conv2d.
Compat aliases for migration
See Migration guide for more details.
tf.compat.v1.nn.atrous_conv2d_transpose
This operation is sometimes called "deconvolution" after (Zeiler et al., 2010), but is really the transpose (gradient) of atrous_conv2d rather than an actual deconvolution.
Deconvolutional Networks: Zeiler et al., 2010 (pdf)


Performs the avg pooling on the input.
Compat aliases for migration
See Migration guide for more details.
tf.compat.v1.nn.avg_pool_v2
Each entry in output is the mean of the corresponding size ksize window in value.


Performs the average pooling on the input.
Compat aliases for migration
See Migration guide for more details.
tf.compat.v1.nn.avg_pool1d
Each entry in output is the mean of the corresponding size ksize window in value.
Note internally this op reshapes and uses the underlying 2d operation.


Performs the average pooling on the input.
Each entry in output is the mean of the corresponding size ksize window in value.


Performs the average pooling on the input.
Compat aliases for migration
See Migration guide for more details.
tf.compat.v1.nn.avg_pool3d
Each entry in output is the mean of the corresponding size ksize window in value.


Batch normalization.
Compat aliases for migration
See Migration guide for more details.
tf.compat.v1.nn.batch_normalization
Normalizes a tensor by mean and variance, and applies (optionally) a scale \(\gamma\) to it, as well as an offset \(\beta\):
\(\frac{\gamma(x-\mu)}{\sigma}+\beta\)
mean, variance, offset and scale are all expected to be of one of two shapes:
See equation 11 in Algorithm 2 of source: Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift; S. Ioffe, C. Szegedy.
Batch Normalization - Accelerating Deep Network Training by Reducing Internal Covariate Shift: Ioffe et al., 2015 (pdf)


Batch normalization.
This op is deprecated. See tf.nn.batch_normalization.
Batch Normalization - Accelerating Deep Network Training by Reducing Internal Covariate Shift: Ioffe et al., 2015 (pdf)


Adds bias to value.
Compat aliases for migration
See Migration guide for more details.
tf.compat.v1.nn.bias_add
This is (mostly) a special case of tf.add where bias is restricted to 1-D. Broadcasting is supported, so value may have any number of dimensions. Unlike tf.add, the type of bias is allowed to differ from value in the case where both types are quantized.


Merge repeated labels into single labels.
Compat aliases for migration
See Migration guide for more details.
tf.compat.v1.nn.collapse_repeated


Compute the position ids in sampled_candidates matching true_classes.
Compat aliases for migration
See Migration guide for more details.
tf.compat.v1.nn.compute_accidental_hits
In Candidate Sampling, this operation facilitates virtually removing sampled classes which happen to match target classes. This is done in Sampled Softmax and Sampled Logistic.
See our Candidate Sampling Algorithms Reference.
We presuppose that the sampled_candidates are unique.
We call it an 'accidental hit' when one of the target classes matches one of the sampled classes. This operation reports accidental hits as triples (index, id, weight), where index represents the row number in true_classes, id represents the position in sampled_candidates, and weight is -FLOAT_MAX.
The result of this op should be passed through a sparse_to_dense operation, then added to the logits of the sampled classes. This removes the contradictory effect of accidentally sampling the true target classes as noise classes for the same example.


Scales per-example losses with sample_weights and computes their average.
Compat aliases for migration
See Migration guide for more details.
tf.compat.v1.nn.compute_average_loss
Usage with distribution strategy and custom training loop:


Computes a 1-D convolution given 3-D input and filter tensors.
Given an input tensor of shape batch_shape + [in_width, in_channels] if data_format is "NWC", or batch_shape + [in_channels, in_width] if data_format is "NCW", and a filter / kernel tensor of shape [filter_width, in_channels, out_channels], this op reshapes the arguments to pass them to conv2d to perform the equivalent convolution operation.
Internally, this op reshapes the input tensors and invokes tf.nn.conv2d. For example, if data_format does not start with "NC", a tensor of shape batch_shape + [in_width, in_channels] is reshaped to batch_shape + [1, in_width, in_channels], and the filter is reshaped to [1, filter_width, in_channels, out_channels]. The result is then reshaped back to batch_shape + [out_width, out_channels] (where out_width is a function of the stride and padding as in conv2d) and returned to the caller.


The transpose of conv1d.
Compat aliases for migration
See Migration guide for more details.
tf.compat.v1.nn.conv1d_transpose
This operation is sometimes called "deconvolution" after (Zeiler et al., 2010), but is actually the transpose (gradient) of conv1d rather than an actual deconvolution.
Deconvolutional Networks: Zeiler et al., 2010 (pdf)


Computes a 2-D convolution given input and 4-D filters tensors.
The input tensor may have rank 4 or higher, where shape dimensions [:-3] are considered batch dimensions (batch_shape).
Given an input tensor of shape batch_shape + [in_height, in_width, in_channels] and a filter / kernel tensor of shape [filter_height, filter_width, in_channels, out_channels], this op performs the following:
In detail, with the default NHWC format,
Must have strides[0] = strides[3] = 1. For the most common case of the same horizontal and vertical strides, strides = [1, stride, stride, 1].


The transpose of conv2d.
This operation is sometimes called "deconvolution" after (Zeiler et al., 2010), but is really the transpose (gradient) of atrous_conv2d rather than an actual deconvolution.
Deconvolutional Networks: Zeiler et al., 2010 (pdf)


Computes a 3-D convolution given 5-D input and filters tensors.
In signal processing, cross-correlation is a measure of similarity of two waveforms as a function of a time-lag applied to one of them. This is also known as a sliding dot product or sliding inner-product.
Our Conv3D implements a form of cross-correlation.


The transpose of conv3d.
This operation is sometimes called "deconvolution" after (Zeiler et al., 2010), but is really the transpose (gradient) of conv3d rather than an actual deconvolution.
Deconvolutional Networks: Zeiler et al., 2010 (pdf)


Computes sums of N-D convolutions (actually cross-correlation).
This also supports either output striding via the optional strides parameter or atrous convolution (also known as convolution with holes or dilated convolution, based on the French word "trous" meaning holes in English) via the optional dilations parameter. Currently, however, output striding is not supported for atrous convolutions.
Specifically, in the case that data_format does not start with "NC", given a rank (N+2) input Tensor of shape
[num_batches, input_spatial_shape[0], ..., input_spatial_shape[N-1], num_input_channels],
a rank (N+2) filters Tensor of shape
[spatial_filter_shape[0], ..., spatial_filter_shape[N-1], num_input_channels, num_output_channels],
an optional dilations tensor of shape N specifying the filter upsampling/input downsampling rate, and an optional list of N strides (defaulting [1]*N), this computes for each N-D spatial output position (x[0], ..., x[N-1]):
where b is the index into the batch, k is the output channel number, q is the input channel number, and z is the N-D spatial offset within the filter. Here, padded_input is obtained by zero padding the input using an effective spatial filter shape of (spatial_filter_shape-1) * dilation_rate + 1 and output striding strides as described in the comment here.
In the case that data_format does start with "NC", the input and output (but not the filters) are simply transposed as follows:
convolution(input, data_format, **kwargs) = tf.transpose(convolution(tf.transpose(input, [0] + range(2,N+2) + [1]), **kwargs), [0, N+1] + range(1, N+1))
It is required that 1 <= N <= 3.
[batch_size] + output_spatial_shape + [out_channels]
if data_format is None or does not start with "NC", or
[batch_size, out_channels] + output_spatial_shape
if data_format starts with "NC", where output_spatial_shape depends on the value of padding.
If padding == "SAME": output_spatial_shape[i] = ceil(input_spatial_shape[i] / strides[i])
If padding == "VALID": output_spatial_shape[i] = ceil((input_spatial_shape[i] - (spatial_filter_shape[i]-1) * dilation_rate[i]) / strides[i]).


The transpose of convolution.
Compat aliases for migration
See Migration guide for more details.
tf.compat.v1.nn.conv_transpose
This operation is sometimes called "deconvolution" after (Zeiler et al., 2010), but is really the transpose (gradient) of conv3d rather than an actual deconvolution.
Deconvolutional Networks: Zeiler et al., 2010 (pdf)


Computes Concatenated ReLU.
Concatenates a ReLU which selects only the positive part of the activation with a ReLU which selects only the negative part of the activation. Note that as a result this non-linearity doubles the depth of the activations. Source: Understanding and Improving Convolutional Neural Networks via Concatenated Rectified Linear Units. W. Shang, et al.
Understanding and Improving Convolutional Neural Networks via Concatenated Rectified Linear Units: Shang et al., 2016 (pdf)


Performs beam search decoding on the logits given in input.
Compat aliases for migration
See Migration guide for more details.
tf.compat.v1.nn.ctc_beam_search_decoder_v2
decoded[j].indices: Indices matrix [total_decoded_outputs[j], 2]; The rows store: [batch, time].
decoded[j].values: Values vector, size [total_decoded_outputs[j]]. The vector stores the decoded classes for beam j.
decoded[j].dense_shape: Shape vector, size (2). The shape values are: [batch_size, max_decoded_length[j]].


Performs greedy decoding on the logits given in input (best path).
Compat aliases for migration
See Migration guide for more details.
tf.compat.v1.nn.ctc_greedy_decoder
If merge_repeated is True, merge repeated classes in output. This means that if consecutive logits' maximum indices are the same, only the first of these is emitted. The sequence A B B * B * B (where '*' is the blank label) becomes
decoded.indices: Indices matrix (total_decoded_outputs, 2). The rows store: [batch, time].
decoded.values: Values vector, size (total_decoded_outputs). The vector stores the decoded classes.
decoded.dense_shape: Shape vector, size (2). The shape values are: [batch_size, max_decoded_length]


Computes CTC (Connectionist Temporal Classification) loss.
This op implements the CTC loss as presented in (Graves et al., 2006).
Connectionist Temporal Classification - Labeling Unsegmented Sequence Data with Recurrent Neural Networks: Graves et al., 2006 (pdf)


Get unique labels and indices for batched labels for tf.nn.ctc_loss.
Compat aliases for migration
See Migration guide for more details.
tf.compat.v1.nn.ctc_unique_labels
For use with tf.nn.ctc_loss optional argument unique: This op can be used to preprocess labels in input pipeline to for better speed/memory use computing the ctc loss on TPU.
ctc_unique_labels([[3, 4, 4, 3]]) -> unique labels padded with 0: [[3, 4, 0, 0]] indices of original labels in unique: [0, 1, 1, 0]


Depthwise 2-D convolution.
Given a 4D input tensor ('NHWC' or 'NCHW' data formats) and a filter tensor of shape [filter_height, filter_width, in_channels, channel_multiplier] containing in_channels convolutional filters of depth 1, depthwise_conv2d applies a different filter to each input channel (expanding from 1 channel to channel_multiplier channels for each), then concatenates the results together. The output has in_channels * channel_multiplier channels.
In detail, with the default NHWC format,
Must have strides[0] = strides[3] = 1. For the most common case of the same horizontal and vertical strides, strides = [1, stride, stride, 1]. If any value in rate is greater than 1, we perform atrous depthwise convolution, in which case all values in the strides tensor must be equal to 1.


Computes the gradients of depthwise convolution with respect to the filter.
Compat aliases for migration
See Migration guide for more details.
tf.compat.v1.nn.depthwise_conv2d_backprop_filter, tf.compat.v1.nn.depthwise_conv2d_native_backprop_filter


Computes the gradients of depthwise convolution with respect to the input.
Compat aliases for migration
See Migration guide for more details.
tf.compat.v1.nn.depthwise_conv2d_backprop_input, tf.compat.v1.nn.depthwise_conv2d_native_backprop_input


DepthToSpace for tensors of type T.
Rearranges data from depth into blocks of spatial data. This is the reverse transformation of SpaceToDepth. More specifically, this op outputs a copy of the input tensor where values from the depth dimension are moved in spatial blocks to the height and width dimensions. The attr block_size indicates the input block size and how the data is moved.
The data_format attr specifies the layout of the input and output tensors with the following options: "NHWC": [ batch, height, width, channels ] "NCHW": [ batch, channels, height, width ] "NCHW_VECT_C": qint8 [ batch, channels / 4, height, width, 4 ]
It is useful to consider the operation as transforming a 6-D Tensor. e.g. for data_format = NHWC, Each element in the input tensor can be specified via 6 coordinates, ordered by decreasing memory layout significance as: n,iY,iX,bY,bX,oC (where n=batch index, iX, iY means X or Y coordinates within the input image, bX, bY means coordinates within the output block, oC means output channels). The output would be the input transposed to the following layout: n,iY,bY,iX,bX,oC
This operation is useful for resizing the activations between convolutions (but keeping all data), e.g. instead of pooling. It is also useful for training purely convolutional models.
For example, given an input of shape [1, 1, 1, 4], data_format = "NHWC" and block_size = 2:
This operation will output a tensor of shape [1, 2, 2, 1]:
Here, the input has a batch of 1 and each batch element has shape [1, 1, 4], the corresponding output will have 2x2 elements and will have a depth of 1 channel (1 = 4 / (block_size * block_size)). The output element shape is [2, 2, 1].
For an input tensor with larger depth, here of shape [1, 1, 1, 12], e.g.
This operation, for block size of 2, will return the following tensor of shape [1, 2, 2, 3]
Similarly, for the following input of shape [1 2 2 4], and a block size of 2:
the operator will return the following tensor of shape [1 4 4 1]:


Computes the grayscale dilation of 4-D input and 3-D filters tensors.
The input tensor has shape [batch, in_height, in_width, depth] and the filters tensor has shape [filter_height, filter_width, depth], i.e., each input channel is processed independently of the others with its own structuring function. The output tensor has shape [batch, out_height, out_width, depth]. The spatial dimensions of the output tensor depend on the padding algorithm. We currently only support the default "NHWC" data_format.
In detail, the grayscale morphological 2-D dilation is the max-sum correlation (for consistency with conv2d, we use unmirrored filters):
Max-pooling is a special case when the filter has size equal to the pooling kernel size and contains all zeros.
Note on duality: The dilation of input by the filters is equal to the negation of the erosion of -input by the reflected filters.


Computes dropout: randomly sets elements to zero to prevent overfitting.
See also: tf.keras.layers.Dropout for a dropout layer.
Dropout is useful for regularizing DNN models. Inputs elements are randomly set to zero (and the other elements are rescaled). This encourages each node to be independently useful, as it cannot rely on the output of other nodes.
More precisely: With probability rate elements of x are set to 0. The remaining elements are scaled up by 1.0 / (1 - rate), so that the expected value is preserved.
By default, each element is kept or dropped independently. If noise_shape is specified, it must be broadcastable to the shape of x, and only dimensions with noise_shape[i] == shape(x)[i] will make independent decisions. This is useful for dropping whole channels from an image or sequence. For example:


Computes exponential linear: exp(features) - 1 if < 0, features otherwise.
Compat aliases for migration
See Migration guide for more details.
tf.compat.v1.nn.elu
See Fast and Accurate Deep Network Learning by Exponential Linear Units (ELUs)


Looks up embeddings for the given ids from a list of tensors.
This function is used to perform parallel lookups on the list of tensors in params. It is a generalization of tf.gather, where params is interpreted as a partitioning of a large embedding tensor.
If len(params) > 1, each element id of ids is partitioned between the elements of params according to the "div" partition strategy, which means we assign ids to partitions in a contiguous manner. For instance, 13 ids are split across 5 partitions as: [[0, 1, 2], [3, 4, 5], [6, 7, 8], [9, 10], [11, 12]].
If the id space does not evenly divide the number of partitions, each of the first (max_id + 1) % len(params) partitions will be assigned one more id.
The results of the lookup are concatenated into a dense tensor. The returned tensor has shape shape(ids) + shape(params)[1:].
For instance, if params is a 5x2 matrix:
or a list of matrices:
and ids is:
The output will be a 3x2 matrix:


Looks up embeddings for the given ids and weights from a list of tensors.
This op assumes that there is at least one id for each row in the dense tensor represented by sp_ids (i.e. there are no rows with empty features), and that all the indices of sp_ids are in canonical row-major order.
sp_ids and sp_weights (if not None) are SparseTensors with rank of 2. Embeddings are always aggregated along the last dimension.
It also assumes that all id values lie in the range [0, p0), where p0 is the sum of the size of params along dimension 0.
If len(params) > 1, each element of sp_ids is partitioned between the elements of params according to the "div" partition strategy, which means we assign ids to partitions in a contiguous manner. For instance, 13 ids are split across 5 partitions as: [[0, 1, 2], [3, 4, 5], [6, 7, 8], [9, 10], [11, 12]].
If the id space does not evenly divide the number of partitions, each of the first (max_id + 1) % len(params) partitions will be assigned one more id.
In other words, if
shape(combined params) = [p0, p1, ..., pm]
and
shape(sp_ids) = shape(sp_weights) = [d0, d1]
then
shape(output) = [d0, p1, ..., pm].
For instance, if params is a 10x20 matrix, and sp_ids / sp_weights are
with combiner="mean", then the output will be a 3x20 matrix where


Computes the grayscale erosion of 4-D value and 3-D filters tensors.
The value tensor has shape [batch, in_height, in_width, depth] and the filters tensor has shape [filters_height, filters_width, depth], i.e., each input channel is processed independently of the others with its own structuring function. The output tensor has shape [batch, out_height, out_width, depth]. The spatial dimensions of the output tensor depend on the padding algorithm. We currently only support the default "NHWC" data_format.
In detail, the grayscale morphological 2-D erosion is given by:
Duality: The erosion of value by the filters is equal to the negation of the dilation of -value by the reflected filters.


Performs fractional average pooling on the input.
Fractional average pooling is similar to Fractional max pooling in the pooling region generation step. The only difference is that after pooling regions are generated, a mean operation is performed instead of a max operation in each pooling region.
A tuple of Tensor objects (output, row_pooling_sequence, col_pooling_sequence). output: Output Tensor after fractional avg pooling. Has the same type as value. row_pooling_sequence: A Tensor of type int64. col_pooling_sequence: A Tensor of type int64.
Fractional Max-Pooling: Graham, 2015 (pdf)


Performs fractional max pooling on the input.
Fractional max pooling is slightly different than regular max pooling. In regular max pooling, you downsize an input set by taking the maximum value of smaller N x N subsections of the set (often 2x2), and try to reduce the set by a factor of N, where N is an integer. Fractional max pooling, as you might expect from the word "fractional", means that the overall reduction ratio N does not have to be an integer.
The sizes of the pooling regions are generated randomly but are fairly uniform. For example, let's look at the height dimension, and the constraints on the list of rows that will be pool boundaries.
First we define the following:
Then, row_pooling_sequence should satisfy:
A tuple of Tensor objects (output, row_pooling_sequence, col_pooling_sequence). output: Output Tensor after fractional max pooling. Has the same type as value. row_pooling_sequence: A Tensor of type int64. col_pooling_sequence: A Tensor of type int64.
Fractional Max-Pooling: Graham, 2015 (pdf)


Compute the Gaussian Error Linear Unit (GELU) activation function.
Gaussian error linear unit (GELU) computes x * P(X <= x), where P(X) ~ N(0, 1). The (GELU) nonlinearity weights inputs by their value, rather than gates inputs by their sign as in ReLU.
Gaussian Error Linear Units (GELUs).


Solves isotonic regression problems along the given axis.
For each vector x, the problem solved is
As the solution is component-wise constant, a second tensor is returned that encodes the segments. The problems are solved over the given axis.
Consider the following example, where we solve a batch of two problems. The first input is [3, 1, 2], while the second 1, 3, 4.
Note that the first solution has two blocks [2] and [1.5, 1.5]. The second solution is constant, and thus has a single segment. These segments are exactly what the second returned tensor encodes:


L2 Loss.
Compat aliases for migration
See Migration guide for more details.
tf.compat.v1.nn.l2_loss
Computes half the L2 norm of a tensor without the sqrt:


Compute the Leaky ReLU activation function.
Compat aliases for migration
See Migration guide for more details.
tf.compat.v1.nn.leaky_relu
Source: Rectifier Nonlinearities Improve Neural Network Acoustic Models. AL Maas, AY Hannun, AY Ng - Proc. ICML, 2013. Args: features: A Tensor representing preactivation values. Must be one of the following types: float16, float32, float64, int32, int64. alpha: Slope of the activation function at x < 0. name: A name for the operation (optional).
Rectifier Nonlinearities Improve Neural Network Acoustic Models: Maas et al., 2013 (pdf)


Local Response Normalization.
Main aliases
tf.nn.lrn
See Migration guide for more details.
tf.compat.v1.nn.local_response_normalization, tf.compat.v1.nn.lrn
The 4-D input tensor is treated as a 3-D array of 1-D vectors (along the last dimension), and each vector is normalized independently. Within a given vector, each component is divided by the weighted, squared sum of inputs within depth_radius. In detail,
For details, see Krizhevsky et al., ImageNet classification with deep convolutional neural networks (NIPS 2012).


Computes log Poisson loss given log_input.
Compat aliases for migration
See Migration guide for more details.
tf.compat.v1.nn.log_poisson_loss
Gives the log-likelihood loss between the prediction and the target under the assumption that the target has a Poisson distribution. Caveat: By default, this is not the exact loss, but the loss minus a constant term [log(z!)]. That has no effect for optimization, but does not play well with relative loss comparisons. To compute an approximation of the log factorial term, specify compute_full_loss=True to enable Stirling's Approximation.
For brevity, let c = log(x) = log_input, z = targets. The log Poisson loss is


Computes log softmax activations.
Main aliases
tf.math.log_softmax
For each batch i and class j we have


Performs the max pooling on the input.
Compat aliases for migration
See Migration guide for more details.
tf.compat.v1.nn.max_pool_v2


Performs the max pooling on the input.
Compat aliases for migration
See Migration guide for more details.
tf.compat.v1.nn.max_pool1d
Note internally this op reshapes and uses the underlying 2d operation.


Performs the max pooling on the input.
Compat aliases for migration
See Migration guide for more details.
tf.compat.v1.nn.max_pool2d


Performs the max pooling on the input.
Compat aliases for migration
See Migration guide for more details.
tf.compat.v1.nn.max_pool3d


Performs max pooling on the input and outputs both max values and indices.
The indices in argmax are flattened, so that a maximum value at position [b, y, x, c] becomes flattened index: (y * width + x) * channels + c if include_batch_in_index is False; ((b * height + y) * width + x) * channels + c if include_batch_in_index is True.
The indices returned are always in [0, height) x [0, width) before flattening, even if padding is involved and the mathematically correct answer is outside (either negative or too large). This is a bug, but fixing it is difficult to do in a safe backwards compatible way, especially due to flattening.


Calculates the mean and variance of x.
The mean and variance are calculated by aggregating the contents of x across axes. If x is 1-D and axes = [0] this is just the mean and variance of a vector.
When using these moments for batch normalization (see tf.nn.batch_normalization):


Computes and returns the noise-contrastive estimation training loss.
See Noise-contrastive estimation: A new estimation principle for unnormalized statistical models. Also see our Candidate Sampling Algorithms Reference
A common use case is to use this method for training, and calculate the full sigmoid loss for evaluation or inference as in the following example:


Calculate the mean and variance of based on the sufficient statistics.
Compat aliases for migration
See Migration guide for more details.
tf.compat.v1.nn.normalize_moments


Performs an N-D pooling operation.
In the case that data_format does not start with "NC", computes for 0 <= b < batch_size, 0 <= x[i] < output_spatial_shape[i], 0 <= c < num_channels:
where the reduction function REDUCE depends on the value of pooling_type, and pad_before is defined based on the value of padding as described in the "returns" section of tf.nn.convolution for details. The reduction never includes out-of-bounds positions.
In the case that data_format starts with "NC", the input and output are simply transposed as follows:
if data_format is None or does not start with "NC", or
[batch_size, num_channels] + output_spatial_shape
if data_format starts with "NC", where output_spatial_shape depends on the value of padding:
If padding = "SAME": output_spatial_shape[i] = ceil(input_spatial_shape[i] / strides[i])
If padding = "VALID": output_spatial_shape[i] = ceil((input_spatial_shape[i] - (window_shape[i] - 1) * dilation_rate[i]) / strides[i]).


Computes rectified linear: max(features, 0).
Compat aliases for migration
See Migration guide for more details.
tf.compat.v1.nn.relu
See: https://en.wikipedia.org/wiki/Rectifier_(neural_networks) Example usage:


Computes Rectified Linear 6: min(max(features, 0), 6).
Compat aliases for migration
See Migration guide for more details.
tf.compat.v1.nn.relu6
Convolutional Deep Belief Networks on CIFAR-10: Krizhevsky et al., 2010 (pdf)


Operator that ensures an RNNCell runs on a particular device.
Inherits From: AbstractRNNCell, Layer, Module
View source
View source


Operator adding dropout to inputs and outputs of the given cell.
Inherits From: AbstractRNNCell, Layer, Module
View source
View source


RNNCell wrapper that ensures cell inputs are added to the outputs.
Inherits From: AbstractRNNCell, Layer, Module
View source
View source


Lookup embedding results, accounting for invalid IDs and empty features.
The partitioned embedding in embedding_weights must all be the same shape except for the first dimension. The first dimension is allowed to vary as the vocabulary size is not necessarily a multiple of num of shards.
Invalid IDs (< 0) are pruned from input IDs and weights, as well as any IDs with non-positive weight. For an entry with no features, the embedding vector for default_id is returned, or the 0-vector if default_id is not supplied.
The ids and weights may be multi-dimensional. Embeddings are always aggregated along the last dimension.
If len(embedding_weights) > 1, each element id of ids is partitioned between the elements of embedding_weights according to the "div" partition strategy, which means we assign ids to partitions in a contiguous manner. For instance, 13 ids are split across 5 partitions as: [[0, 1, 2], [3, 4, 5], [6, 7, 8], [9, 10], [11, 12]].
If the id space does not evenly divide the number of partitions, each of the first (max_id + 1) % len(embedding_weights) partitions will be assigned one more id.
In other words, if
shape(combined embedding_weights) = [p0, p1, ..., pm]
and
shape(sparse_ids) = shape(sparse_weights) = [d0, d1, ..., dn]
then
shape(output) = [d0, d1, ... dn-1, p1, ..., pm].
For instance, if params is a 10x20 matrix, and sp_ids / sp_weights are
default_id is 0.
with combiner="mean", then the output will be a 3x20 matrix where


Computes and returns the sampled softmax training loss.
This is a faster way to train a softmax classifier over a huge number of classes.
This operation is for training only. It is generally an underestimate of the full softmax loss.
A common use case is to use this method for training, and calculate the full sigmoid loss for evaluation or inference as in the following example:
See our Candidate Sampling Algorithms Reference
Also see Section 3 of Jean et al., 2014 (pdf) for the math.


Scales the sum of the given regularization losses by number of replicas.
Compat aliases for migration
See Migration guide for more details.
tf.compat.v1.nn.scale_regularization_loss
Usage with distribution strategy and custom training loop:


Computes scaled exponential linear: scale * alpha * (exp(features) - 1)
Compat aliases for migration
See Migration guide for more details.
tf.compat.v1.nn.selu
if < 0, scale * features otherwise.
To be used together with initializer = tf.variance_scaling_initializer(factor=1.0, mode='FAN_IN'). For correct dropout, use tf.contrib.nn.alpha_dropout.
See Self-Normalizing Neural Networks


2-D convolution with separable filters.
Performs a depthwise convolution that acts separately on channels followed by a pointwise convolution that mixes channels. Note that this is separability between dimensions [1, 2] and 3, not spatial separability between dimensions 1 and 2.
In detail, with the default NHWC format,
strides controls the strides for the depthwise convolution only, since the pointwise convolution has implicit strides of [1, 1, 1, 1]. Must have strides[0] = strides[3] = 1. For the most common case of the same horizontal and vertical strides, strides = [1, stride, stride, 1]. If any value in rate is greater than 1, we perform atrous depthwise convolution, in which case all values in the strides tensor must be equal to 1.


Computes sigmoid cross entropy given logits.
Measures the probability error in discrete classification tasks in which each class is independent and not mutually exclusive. For instance, one could perform multilabel classification where a picture can contain both an elephant and a dog at the same time.
For brevity, let x = logits, z = labels. The logistic loss is
For x < 0, to avoid overflow in exp(-x), we reformulate the above
Hence, to ensure stability and avoid overflow, the implementation uses this equivalent formulation
logits and labels must have the same type and shape.


Computes the SiLU or Swish activation function: x * sigmoid(x).
Main aliases
tf.nn.swish
See Migration guide for more details.
tf.compat.v1.nn.silu, tf.compat.v1.nn.swish
The SiLU activation function was introduced in "Gaussian Error Linear Units (GELUs)" Hendrycks et al. 2016 and "Sigmoid-Weighted Linear Units for Neural Network Function Approximation in Reinforcement Learning" Elfwing et al. 2017 and was independently discovered (and called swish) in "Searching for Activation Functions" Ramachandran et al. 2017


Computes softmax activations.
Main aliases
tf.math.softmax
This function performs the equivalent of


Computes softmax cross entropy between logits and labels.
Measures the probability error in discrete classification tasks in which the classes are mutually exclusive (each entry is in exactly one class). For example, each CIFAR-10 image is labeled with one and only one label: an image can be a dog or a truck, but not both.
If using exclusive labels (wherein one and only one class is true at a time), see sparse_softmax_cross_entropy_with_logits.
A common use case is to have logits and labels of shape [batch_size, num_classes], but higher dimensions are supported, with the axis argument specifying the class dimension.
logits and labels must have the same dtype (either float16, float32, or float64).
Backpropagation will happen into both logits and labels. To disallow backpropagation into labels, pass label tensors through tf.stop_gradient before feeding it to this function.
Note that to avoid confusion, it is required to pass only named arguments to this function.


Computes softsign: features / (abs(features) + 1).
Main aliases
tf.math.softsign
See Migration guide for more details.
tf.compat.v1.math.softsign, tf.compat.v1.nn.softsign


SpaceToDepth for tensors of type T.
Rearranges blocks of spatial data, into depth. More specifically, this op outputs a copy of the input tensor where values from the height and width dimensions are moved to the depth dimension. The attr block_size indicates the input block size.
The data_format attr specifies the layout of the input and output tensors with the following options: "NHWC": [ batch, height, width, channels ] "NCHW": [ batch, channels, height, width ] "NCHW_VECT_C": qint8 [ batch, channels / 4, height, width, 4 ]
It is useful to consider the operation as transforming a 6-D Tensor. e.g. for data_format = NHWC, Each element in the input tensor can be specified via 6 coordinates, ordered by decreasing memory layout significance as: n,oY,bY,oX,bX,iC (where n=batch index, oX, oY means X or Y coordinates within the output image, bX, bY means coordinates within the input block, iC means input channels). The output would be a transpose to the following layout: n,oY,oX,bY,bX,iC
This operation is useful for resizing the activations between convolutions (but keeping all data), e.g. instead of pooling. It is also useful for training purely convolutional models.
For example, given an input of shape [1, 2, 2, 1], data_format = "NHWC" and block_size = 2:
This operation will output a tensor of shape [1, 1, 1, 4]:
Here, the input has a batch of 1 and each batch element has shape [2, 2, 1], the corresponding output will have a single element (i.e. width and height are both 1) and will have a depth of 4 channels (1 * block_size * block_size). The output element shape is [1, 1, 4].
For an input tensor with larger depth, here of shape [1, 2, 2, 3], e.g.
This operation, for block_size of 2, will return the following tensor of shape [1, 1, 1, 12]
Similarly, for the following input of shape [1 4 4 1], and a block size of 2:
the operator will return the following tensor of shape [1 2 2 4]:


Computes sparse softmax cross entropy between logits and labels.
Measures the probability error in discrete classification tasks in which the classes are mutually exclusive (each entry is in exactly one class). For example, each CIFAR-10 image is labeled with one and only one label: an image can be a dog or a truck, but not both.
A common use case is to have logits of shape [batch_size, num_classes] and have labels of shape [batch_size], but higher dimensions are supported, in which case the dim-th dimension is assumed to be of size num_classes. logits must have the dtype of float16, float32, or float64, and labels must have the dtype of int32 or int64.
Note that to avoid confusion, it is required to pass only named arguments to this function.


Calculate the sufficient statistics for the mean and variance of x.
These sufficient statistics are computed using the one pass algorithm on an input that's optionally shifted. See: https://en.wikipedia.org/wiki/Algorithms_for_calculating_variance#Computing_shifted_data


Computes a weighted cross entropy.
This is like sigmoid_cross_entropy_with_logits() except that pos_weight, allows one to trade off recall and precision by up- or down-weighting the cost of a positive error relative to a negative error.
The usual cross-entropy cost is defined as:
A value pos_weight > 1 decreases the false negative count, hence increasing the recall. Conversely setting pos_weight < 1 decreases the false positive count and increases the precision. This can be seen from the fact that pos_weight is introduced as a multiplicative coefficient for the positive labels term in the loss expression:
For brevity, let x = logits, z = labels, q = pos_weight. The loss is:
Setting l = (1 + (q - 1) * z), to ensure stability and avoid overflow, the implementation uses
logits and labels must have the same type and shape.


Returns the frequency-weighted mean and variance of x.


Performs op on the space-to-batch representation of input.
Compat aliases for migration
See Migration guide for more details.
tf.compat.v1.nn.with_space_to_batch
This has the effect of transforming sliding window operations into the corresponding "atrous" operation in which the input is sampled at the specified dilation_rate.
In the special case that dilation_rate is uniformly 1, this simply returns:
op(input, num_spatial_dims, padding)
Otherwise, it returns:
batch_to_space_nd( op(space_to_batch_nd(input, adjusted_dilation_rate, adjusted_paddings), num_spatial_dims, "VALID") adjusted_dilation_rate, adjusted_crops),
where:
adjusted_dilation_rate is an int64 tensor of shape [max(spatialdims)], adjusted{paddings,crops} are int64 tensors of shape [max(spatial_dims), 2]
defined as follows:
We first define two int64 tensors paddings and crops of shape [num_spatial_dims, 2] based on the value of padding and the spatial dimensions of the input:
If padding = "VALID", then:
paddings, crops = required_space_to_batch_paddings( input_shape[spatial_dims], dilation_rate)
If padding = "SAME", then:
dilated_filter_shape = filter_shape + (filter_shape - 1) * (dilation_rate - 1)
paddings, crops = required_space_to_batch_paddings( input_shape[spatial_dims], dilation_rate, [(dilated_filter_shape - 1) // 2, dilated_filter_shape - 1 - (dilated_filter_shape - 1) // 2])
Because space_to_batch_nd and batch_to_space_nd assume that the spatial dimensions are contiguous starting at the second dimension, but the specified spatial_dims may not be, we must adjust dilation_rate, paddings and crops in order to be usable with these operations. For a given dimension, if the block size is 1, and both the starting and ending padding and crop amounts are 0, then space_to_batch_nd effectively leaves that dimension alone, which is what is needed for dimensions not part of spatial_dims. Furthermore, space_to_batch_nd and batch_to_space_nd handle this case efficiently for any number of leading and trailing dimensions.
For 0 <= i < len(spatial_dims), we assign:
adjusted_dilation_rate[spatial_dims[i] - 1] = dilation_rate[i] adjusted_paddings[spatial_dims[i] - 1, :] = paddings[i, :] adjusted_crops[spatial_dims[i] - 1, :] = crops[i, :]
All unassigned values of adjusted_dilation_rate default to 1, while all unassigned values of adjusted_paddings and adjusted_crops default to 0.
Note in the case that dilation_rate is not uniformly 1, specifying "VALID" padding is equivalent to specifying padding = "SAME" with a filter_shape of [1]*N.
Advanced usage. Note the following optimization: A sequence of with_space_to_batch operations with identical (not uniformly 1) dilation_rate parameters and "VALID" padding
net = with_space_to_batch(net, dilation_rate, "VALID", op_1) ... net = with_space_to_batch(net, dilation_rate, "VALID", op_k)
can be combined into a single with_space_to_batch operation as follows:
def combined_op(converted_input, num_spatial_dims, _): result = op_1(converted_input, num_spatial_dims, "VALID") ... result = op_k(result, num_spatial_dims, "VALID")
net = with_space_to_batch(net, dilation_rate, "VALID", combined_op)
This eliminates the overhead of k-1 calls to space_to_batch_nd and batch_to_space_nd.
Similarly, a sequence of with_space_to_batch operations with identical (not uniformly 1) dilation_rate parameters, "SAME" padding, and odd filter dimensions
net = with_space_to_batch(net, dilation_rate, "SAME", op_1, filter_shape_1) ... net = with_space_to_batch(net, dilation_rate, "SAME", op_k, filter_shape_k)
can be combined into a single with_space_to_batch operation as follows:
def combined_op(converted_input, num_spatial_dims, _): result = op_1(converted_input, num_spatial_dims, "SAME") ... result = op_k(result, num_spatial_dims, "SAME")
net = with_space_to_batch(net, dilation_rate, "VALID", combined_op)


Batches the computation done by the decorated function.
Compat aliases for migration
See Migration guide for more details.
tf.compat.v1.nondifferentiable_batch_function
So, for example, in the following code
if more than one session.run call is simultaneously trying to compute b the values of w will be gathered, non-deterministically concatenated along the first axis, and only one thread will run the computation. See the documentation of the Batch op for more details.
Assumes that all arguments of the decorated function are Tensors which will be batched along their first dimension.
SparseTensor is not supported. The return value of the decorated function must be a Tensor or a list/tuple of Tensors.


Computes the norm of vectors, matrices, and tensors.
Main aliases
tf.linalg.norm
This function can compute several different vector norms (the 1-norm, the Euclidean or 2-norm, the inf-norm, and in general the p-norm for p > 0) and matrix norms (Frobenius, 1-norm, 2-norm and inf-norm).
Mostly equivalent to numpy.linalg.norm. Not supported: ord <= 0, 2-norm for matrices, nuclear norm. Other differences: a) If axis is None, treats the flattened tensor as a vector regardless of rank. b) Explicitly supports 'euclidean' norm as the default, including for higher order tensors.


Specifies that ops of type op_type is not differentiable.
Compat aliases for migration
See Migration guide for more details.
tf.compat.v1.NoGradient, tf.compat.v1.NotDifferentiable, tf.compat.v1.no_gradient
This function should not be used for operations that have a well-defined gradient that is not yet implemented.
This function is only used when defining a new op type. It may be used for ops such as tf.size() that are not differentiable. For example:
The gradient computed for 'op_type' will then propagate zeros.
For ops that have a well-defined gradient but are not yet implemented, no declaration should be made, and an error must be thrown if an attempt to request its gradient is made.


Does nothing. Only useful as a placeholder for control edges.
Compat aliases for migration
See Migration guide for more details.
tf.compat.v1.no_op


Wraps a python function and uses it as a TensorFlow op.
Compat aliases for migration
See Migration guide for more details.
tf.compat.v1.numpy_function
Given a python function func wrap this function as an operation in a TensorFlow function. func must take numpy arrays as its arguments and return numpy arrays as its outputs.
The following example creates a TensorFlow graph with np.sinh() as an operation in the graph:
Comparison to tf.py_function: tf.py_function and tf.numpy_function are very similar, except that tf.numpy_function takes numpy arrays, and not tf.Tensors. If you want the function to contain tf.Tensors, and have any TensorFlow operations executed in the function be differentiable, please use tf.py_function.
The body of the function (i.e. func) will not be serialized in a tf.SavedModel. Therefore, you should not use this function if you need to serialize your model and restore it in a different environment.
The operation must run in the same address space as the Python program that calls tf.numpy_function(). If you are using distributed TensorFlow, you must run a tf.distribute.Server in the same process as the program that calls tf.numpy_function you must pin the created operation to a device in that server (e.g. using with tf.device():).
Since the function takes numpy arrays, you cannot take gradients through a numpy_function. If you require something that is differentiable, please consider using tf.py_function.
The resulting function is assumed stateful and will never be optimized.


Creates a tensor with all elements set to one (1).
Compat aliases for migration
See Migration guide for more details.
tf.compat.v1.ones
See also tf.ones_like, tf.zeros, tf.fill, tf.eye.
This operation returns a tensor of type dtype with shape shape and all elements set to one.


Initializer that generates tensors initialized to 1.
Initializers allow you to pre-specify an initialization strategy, encoded in the Initializer object, without knowing the shape and dtype of the variable being initialized.
View source
Instantiates an initializer from a configuration dictionary.
View source
Returns the configuration of the initializer as a JSON-serializable dict.
View source
Returns a tensor object initialized as specified by the initializer.


Creates a tensor of all ones that has the same shape as the input.
See also tf.ones.
Given a single tensor (tensor), this operation returns a tensor of the same type and shape as tensor with all elements set to 1. Optionally, you can use dtype to specify a new type for the returned tensor.


Returns a one-hot tensor.
Compat aliases for migration
See Migration guide for more details.
tf.compat.v1.one_hot
See also tf.fill, tf.eye.
The locations represented by indices in indices take value on_value, while all other locations take value off_value.
on_value and off_value must have matching data types. If dtype is also provided, they must be the same data type as specified by dtype.
If on_value is not provided, it will default to the value 1 with type dtype
If off_value is not provided, it will default to the value 0 with type dtype
If the input indices is rank N, the output will have rank N+1. The new axis is created at dimension axis (default: the new axis is appended at the end).
If indices is a scalar the output shape will be a vector of length depth
If indices is a vector of length features, the output shape will be:
If indices is a matrix (batch) with shape [batch, features], the output shape will be:
If indices is a RaggedTensor, the 'axis' argument must be positive and refer to a non-ragged axis. The output will be equivalent to applying 'one_hot' on the values of the RaggedTensor, and creating a new RaggedTensor from the result.
If dtype is not provided, it will attempt to assume the data type of on_value or off_value, if one or both are passed in. If none of on_value, off_value, or dtype are provided, dtype will default to the value tf.float32.


Represents a graph node that performs computation on tensors.
Compat aliases for migration
See Migration guide for more details.
tf.compat.v1.Operation
An Operation is a node in a tf.Graph that takes zero or more Tensor objects as input, and produces zero or more Tensor objects as output. Objects of type Operation are created by calling a Python op constructor (such as tf.matmul) within a tf.function or under a tf.Graph.as_default context manager.
For example, within a tf.function, c = tf.matmul(a, b) creates an Operation of type "MatMul" that takes tensors a and b as input, and produces c as output.
If a tf.compat.v1.Session is used, an Operation of a tf.Graph can be executed by passing it to tf.Session.run. op.run() is a shortcut for calling tf.compat.v1.get_default_session().run(op).
Before this op is executed, TensorFlow will ensure that the operations in self.control_inputs have finished executing. This mechanism can be used to run ops sequentially for performance reasons, or to ensure that the side effects of an op are observed in the correct order.
View source
Returns the list of colocation groups of the op.
View source
Returns the value of the attr of this op with the given name.
View source
Runs this operation in a Session.
Calling this method will execute all preceding operations that produce the inputs needed for this operation.
View source
DEPRECATED: Use outputs.


Type specification for tf.experimental.Optional.
Inherits From: TypeSpec
Compat aliases for migration
See Migration guide for more details.
tf.compat.v1.OptionalSpec, tf.compat.v1.data.experimental.OptionalStructure
For instance, tf.OptionalSpec can be used to define a tf.function that takes tf.experimental.Optional as an input argument:
In particular, all values that are compatible with this TypeSpec must be an instance of this type.
View source
View source
Returns true if spec_or_value is compatible with this TypeSpec.
View source
Returns the most specific TypeSpec compatible with self and other.
View source
Return self==value.
View source
Return self!=value.


Pads a tensor.
This operation pads a tensor according to the paddings you specify. paddings is an integer tensor with shape [n, 2], where n is the rank of tensor. For each dimension D of input, paddings[D, 0] indicates how many values to add before the contents of tensor in that dimension, and paddings[D, 1] indicates how many values to add after the contents of tensor in that dimension. If mode is "REFLECT" then both paddings[D, 0] and paddings[D, 1] must be no greater than tensor.dim_size(D) - 1. If mode is "SYMMETRIC" then both paddings[D, 0] and paddings[D, 1] must be no greater than tensor.dim_size(D).
The padded size of each dimension D of the output is:
paddings[D, 0] + tensor.dim_size(D) + paddings[D, 1]


Stacks a list of rank-R tensors into one rank-(R+1) tensor in parallel.
Compat aliases for migration
See Migration guide for more details.
tf.compat.v1.parallel_stack
Requires that the shape of inputs be known at graph construction time.
Packs the list of tensors in values into a tensor with rank one higher than each tensor in values, by packing them along the first dimension. Given a list of length N of tensors of shape (A, B, C); the output tensor will have the shape (N, A, B, C).
The difference between stack and parallel_stack is that stack requires all the inputs be computed before the operation will begin but doesn't require that the input shapes be known during graph construction.
parallel_stack will copy pieces of the input into the output as they become available, in some situations this can provide a performance benefit.
Unlike stack, parallel_stack does NOT support backpropagation.
This is the opposite of unstack. The numpy equivalent is


Print the specified inputs.
Compat aliases for migration
See Migration guide for more details.
tf.compat.v1.print
A TensorFlow operator that prints the specified inputs to a desired output stream or logging level. The inputs may be dense or sparse Tensors, primitive python objects, data structures that contain tensors, and printable Python objects. Printed tensors will recursively show the first and last elements of each dimension to summarize.
Single-input usage:
(This prints "[0 1 2 ... 7 8 9]" to sys.stderr)
Multi-input usage:
(This prints "tensors: [0 1 2 ... 7 8 9] {2: [0 2 4 ... 14 16 18]}" to sys.stdout)
Changing the input separator:
(This prints "[0 1],[0 2]" to sys.stderr)
Usage in a tf.function:
(This prints "[0 1 2 ... 7 8 9]" to sys.stderr)
@compatibility(TF 1.x Graphs and Sessions) In graphs manually created outside of tf.function, this method returns the created TF operator that prints the data. To make sure the operator runs, users need to pass the produced op to tf.compat.v1.Session's run method, or to use the op as a control dependency for executed ops by specifying with tf.compat.v1.control_dependencies([print_op]). @end_compatibility
Compatibility usage in TF 1.x graphs:
(This prints "tensors: [0 1 2 ... 7 8 9] {2: [0 2 4 ... 14 16 18]}" to sys.stdout)


Public API for tf.profiler namespace.
experimental module: Public API for tf.profiler.experimental namespace.


Public API for tf.profiler.experimental namespace.
client module: Public API for tf.profiler.experimental.client namespace.
server module: Public API for tf.profiler.experimental.server namespace.
class Profile: Context-manager profile API.
class ProfilerOptions: Options for finer control over the profiler.
class Trace: Context manager that generates a trace event in the profiler.
start(...): Start profiling TensorFlow performance.
stop(...): Stops the current profiling session.


Public API for tf.profiler.experimental.client namespace.
monitor(...): Sends grpc requests to profiler server to perform on-demand monitoring.
trace(...): Sends gRPC requests to one or more profiler servers to perform on-demand profiling.


Sends grpc requests to profiler server to perform on-demand monitoring.
The monitoring result is a light weight performance summary of your model execution. This method will block the caller thread until it receives the monitoring result. This method currently supports Cloud TPU only.


Sends gRPC requests to one or more profiler servers to perform on-demand profiling.
This method will block the calling thread until it receives responses from all servers or until deadline expiration. Both single host and multiple host profiling are supported on CPU, GPU, and TPU. The profiled results will be saved by each server to the specified TensorBoard log directory (i.e. the directory you save your model checkpoints). Use the TensorBoard profile plugin to view the visualization and analysis results.
Example usage (CPU/GPU):
Example usage (Multiple GPUs):
Example usage (TPU):
Example usage (Multiple TPUs):
Launch TensorBoard and point it to the same logdir you provided to this API.
Open your browser and go to localhost:6006/#profile to view profiling results.


Context-manager profile API.
Profiling will start when entering the scope, and stop and save the results to the logdir when exits the scope. Open TensorBoard profile tab to view results.
View source
View source


Options for finer control over the profiler.
Use tf.profiler.ProfilerOptions to control tf.profiler behavior.


Public API for tf.profiler.experimental.server namespace.
start(...): Start a profiler grpc server that listens to given port.


Start a profiler grpc server that listens to given port.
The profiler server will exit when the process finishes. The service is defined in tensorflow/core/profiler/profiler_service.proto.
Example usage: ```python tf.profiler.experimental.server.start('6009') # do your training here.


Start profiling TensorFlow performance.
To view the profiling results, launch TensorBoard and point it to logdir. Open your browser and go to localhost:6006/#profile to view profiling results.


Stops the current profiling session.
The profiler session will be stopped and profile results can be saved.


Context manager that generates a trace event in the profiler.
A trace event will start when entering the context, and stop and save the result to the profiler when exiting the context. Open TensorBoard Profile tab and choose trace viewer to view the trace event in the timeline.
Trace events are created only when the profiler is enabled. More information on how to use the profiler can be found at https://tensorflow.org/guide/profiler
Example usage:
The example above uses the keyword argument "step_num" to specify the training step being traced.
View source
Sets metadata in this trace event.
This method enables setting metadata in a trace event after it is created.
In this example, we want to trace how much time spent on calling a function, which includes compilation and execution. The compilation can be either getting a cached copy of the binary or actually generating the binary, which is indicated by the boolean "in_cache" returned by jit_compile(). We need to use set_metadata() to pass in_cache because we did not know the in_cache value when the trace was created (and we cannot create the trace after jit_compile(), because we want to measure the entire duration of call()).
View source
View source


Wraps a python function into a TensorFlow op that executes it eagerly.
Compat aliases for migration
See Migration guide for more details.
tf.compat.v1.py_function
This function allows expressing computations in a TensorFlow graph as Python functions. In particular, it wraps a Python function func in a once-differentiable TensorFlow operation that executes it with eager execution enabled. As a consequence, tf.py_function makes it possible to express control flow using Python constructs (if, while, for, etc.), instead of TensorFlow control flow constructs (tf.cond, tf.while_loop). For example, you might use tf.py_function to implement the log huber function:
You can also use tf.py_function to debug your models at runtime using Python tools, i.e., you can isolate portions of your code that you want to debug, wrap them in Python functions and insert pdb tracepoints or print statements as desired, and wrap those functions in tf.py_function.
For more information on eager execution, see the Eager guide.
tf.py_function is similar in spirit to tf.compat.v1.py_func, but unlike the latter, the former lets you use TensorFlow operations in the wrapped Python function. In particular, while tf.compat.v1.py_func only runs on CPUs and wraps functions that take NumPy arrays as inputs and return NumPy arrays as outputs, tf.py_function can be placed on GPUs and wraps functions that take Tensors as inputs, execute TensorFlow operations in their bodies, and return Tensors as outputs.
Like tf.compat.v1.py_func, tf.py_function has the following limitations with respect to serialization and distribution:
The body of the function (i.e. func) will not be serialized in a GraphDef. Therefore, you should not use this function if you need to serialize your model and restore it in a different environment.
The operation must run in the same address space as the Python program that calls tf.py_function(). If you are using distributed TensorFlow, you must run a tf.distribute.Server in the same process as the program that calls tf.py_function() and you must pin the created operation to a device in that server (e.g. using with tf.device():).


Public API for tf.quantization namespace.
dequantize(...): Dequantize the 'input' tensor into a float or bfloat16 Tensor.
fake_quant_with_min_max_args(...): Fake-quantize the 'inputs' tensor, type float to 'outputs' tensor of same type.
fake_quant_with_min_max_args_gradient(...): Compute gradients for a FakeQuantWithMinMaxArgs operation.
fake_quant_with_min_max_vars(...): Fake-quantize the 'inputs' tensor of type float via global float scalars
fake_quant_with_min_max_vars_gradient(...): Compute gradients for a FakeQuantWithMinMaxVars operation.
fake_quant_with_min_max_vars_per_channel(...): Fake-quantize the 'inputs' tensor of type float via per-channel floats
fake_quant_with_min_max_vars_per_channel_gradient(...): Compute gradients for a FakeQuantWithMinMaxVarsPerChannel operation.
quantize(...): Quantize the 'input' tensor of type float to 'output' tensor of type 'T'.
quantize_and_dequantize(...): Quantizes then dequantizes a tensor. (deprecated)
quantize_and_dequantize_v2(...): Quantizes then dequantizes a tensor.
quantized_concat(...): Concatenates quantized tensors along one dimension.


Dequantize the 'input' tensor into a float or bfloat16 Tensor.
Compat aliases for migration
See Migration guide for more details.
tf.compat.v1.dequantize, tf.compat.v1.quantization.dequantize
[min_range, max_range] are scalar floats that specify the range for the output. The 'mode' attribute controls exactly which calculations are used to convert the float values to their quantized equivalents.
In 'MIN_COMBINED' mode, each value of the tensor will undergo the following:
here range(T) = numeric_limits<T>::max() - numeric_limits<T>::min()
MIN_COMBINED Mode Example
If the input comes from a QuantizedRelu6, the output type is quint8 (range of 0-255) but the possible range of QuantizedRelu6 is 0-6. The min_range and max_range values are therefore 0.0 and 6.0. Dequantize on quint8 will take each value, cast to float, and multiply by 6 / 255. Note that if quantizedtype is qint8, the operation will additionally add each value by 128 prior to casting.
If the mode is 'MIN_FIRST', then this approach is used:
If the mode is SCALED, dequantization is performed by multiplying each input value by a scaling_factor. (Thus an input of 0 always maps to 0.0).
The scaling_factor is determined from min_range, max_range, and narrow_range in a way that is compatible with QuantizeAndDequantize{V2|V3} and QuantizeV2, using the following algorithm: