Computes various recall values for different thresholds on predictions.
The recall_at_thresholds function creates four local variables, true_positives, true_negatives, false_positives and false_negatives for various values of thresholds. recall[i] is defined as the total weight of values in predictions above thresholds[i] whose corresponding entry in labels is True, divided by the total weight of True values in labels (true_positives[i] / (true_positives[i] + false_negatives[i])).
For estimation of the metric over a stream of data, the function creates an update_op operation that updates these variables and returns the recall.
If weights is None, weights default to 1. Use weights of 0 to mask values.


Computes recall@k of top-k predictions with respect to sparse labels.
Differs from recall_at_k in that predictions must be in the form of top k class indices, whereas recall_at_k expects logits. Refer to recall_at_k for more details.


Computes the root mean squared error between the labels and predictions.
The root_mean_squared_error function creates two local variables, total and count that are used to compute the root mean squared error. This average is weighted by weights, and it is ultimately returned as root_mean_squared_error: an idempotent operation that takes the square root of the division of total by count.
For estimation of the metric over a stream of data, the function creates an update_op operation that updates these variables and returns the root_mean_squared_error. Internally, a squared_error operation computes the element-wise square of the difference between predictions and labels. Then update_op increments total with the reduced sum of the product of weights and squared_error, and it increments count with the reduced sum of weights.
If weights is None, weights default to 1. Use weights of 0 to mask values.


Computes the specificity at a given sensitivity.
The sensitivity_at_specificity function creates four local variables, true_positives, true_negatives, false_positives and false_negatives that are used to compute the sensitivity at the given specificity value. The threshold for the given specificity value is computed and used to evaluate the corresponding sensitivity.
For estimation of the metric over a stream of data, the function creates an update_op operation that updates these variables and returns the sensitivity. update_op increments the true_positives, true_negatives, false_positives and false_negatives counts with the weight of each case found in the predictions and labels.
If weights is None, weights default to 1. Use weights of 0 to mask values.
For additional information about specificity and sensitivity, see the following: https://en.wikipedia.org/wiki/Sensitivity_and_specificity


Renamed to average_precision_at_k, please use that method instead. (deprecated)


Renamed to precision_at_k, please use that method instead. (deprecated)


Computes the specificity at a given sensitivity.
The specificity_at_sensitivity function creates four local variables, true_positives, true_negatives, false_positives and false_negatives that are used to compute the specificity at the given sensitivity value. The threshold for the given sensitivity value is computed and used to evaluate the corresponding specificity.
For estimation of the metric over a stream of data, the function creates an update_op operation that updates these variables and returns the specificity. update_op increments the true_positives, true_negatives, false_positives and false_negatives counts with the weight of each case found in the predictions and labels.
If weights is None, weights default to 1. Use weights of 0 to mask values.
For additional information about specificity and sensitivity, see the following: https://en.wikipedia.org/wiki/Sensitivity_and_specificity


Sum the weights of true_negatives.
If weights is None, weights default to 1. Use weights of 0 to mask values.


Computes true negatives at provided threshold values.
If weights is None, weights default to 1. Use weights of 0 to mask values.


Sum the weights of true_positives.
If weights is None, weights default to 1. Use weights of 0 to mask values.


Computes true positives at provided threshold values.
If weights is None, weights default to 1. Use weights of 0 to mask values.


Partitioner to allocate minimum size per slice.
Returns a partitioner that partitions the variable of given shape and dtype such that each partition has a minimum of min_slice_size slice of the variable. The maximum number of such partitions (upper bound) is given by max_partitions.


Public API for tf.mixed_precision namespace.
experimental module: Public API for tf.mixed_precision.experimental namespace.
class DynamicLossScale: Loss scale that dynamically adjusts itself.
class FixedLossScale: Loss scale with a fixed value.
class LossScale: Base class for all TF1 loss scales.
class MixedPrecisionLossScaleOptimizer: An optimizer that applies loss scaling.
disable_mixed_precision_graph_rewrite(...): Disables the mixed precision graph rewrite.
enable_mixed_precision_graph_rewrite(...): Enable mixed precision via a graph rewrite.


Disables the mixed precision graph rewrite.
Compat aliases for migration
See Migration guide for more details.
tf.compat.v1.train.experimental.disable_mixed_precision_graph_rewrite
After this is called, the mixed precision graph rewrite will no longer run for new Sessions, and so float32 operations will no longer be converted to float16 in such Sessions. However, any existing Sessions will continue to have the graph rewrite enabled if they were created after enable_mixed_precision_graph_rewrite was called but before disable_mixed_precision_graph_rewrite was called.
This does not undo the effects of loss scaling. Any optimizers wrapped with a LossScaleOptimizer will continue to do loss scaling, although this loss scaling will no longer be useful if the optimizer is used in new Sessions, as the graph rewrite no longer converts the graph to use float16.
This function is useful for unit testing. A unit tests can test using the mixed precision graph rewrite, then disable it so future unit tests continue using float32. If this is done, unit tests should not share a single session, as enable_mixed_precision_graph_rewrite and disable_mixed_precision_graph_rewrite have no effect on existing sessions.


Enable mixed precision via a graph rewrite.
Compat aliases for migration
See Migration guide for more details.
tf.compat.v1.train.experimental.enable_mixed_precision_graph_rewrite
Mixed precision is the use of both float32 and float16 data types when training a model to improve performance. This is achieved via a graph rewrite operation and a loss-scale optimizer.
Performing arithmetic operations in float16 takes advantage of specialized processing units, such as NVIDIA Tensor Cores, for much higher arithmetic throughput. However, due to the smaller representable range, performing the entire training with float16 can result in gradient underflow, that is, small gradient values becoming zeroes. Instead, performing only select arithmetic operations in float16 results in higher throughput and decreased training time when using compatible hardware accelerators while also reducing memory usage, typically without sacrificing model accuracy.
Calling enable_mixed_precision_graph_rewrite(opt) enables the graph rewrite operation before computing gradients. The function additionally returns an Optimizer (opt) wrapped with a LossScaleOptimizer. This prevents underflow in the float16 tensors during the backward pass. An optimizer of type tf.train.Optimizer or tf.keras.optimizers.Optimizer must be passed to this function, which will then be wrapped to use loss scaling.
The graph rewrite operation changes the dtype of certain operations in the graph from float32 to float16. There are several categories of operations that are either included or excluded by this rewrite operation. The following categories of Ops are defined inside corresponding functions under the class AutoMixedPrecisionLists in  auto_mixed_precision_lists.h:
When this function is used, gradients should only be computed and applied with the returned optimizer, either by calling opt.minimize() or opt.compute_gradients() followed by opt.apply_gradients(). Gradients should not be computed with tf.gradients or tf.GradientTape. This is because the returned optimizer will apply loss scaling, and tf.gradients or tf.GradientTape will not. If you do directly use tf.gradients or tf.GradientTape, your model may not converge due to float16 underflow problems.
When eager execution is enabled, the mixed precision graph rewrite is only enabled within tf.functions, as outside tf.functions, there is no graph.
For NVIDIA GPUs with Tensor cores, as a general performance guide, dimensions (such as batch size, input size, output size, and channel counts) should be powers of two if under 256, or otherwise divisible by 8 if above 256. For more information, check out the NVIDIA Deep Learning Performance Guide.
Currently, mixed precision is only enabled on NVIDIA Tensor Core GPUs with Compute Capability 7.0 and above (Volta, Turing, or newer architectures). The parts of the graph on CPUs and TPUs are untouched by the graph rewrite.


Public API for tf.mixed_precision.experimental namespace.
class DynamicLossScale: Loss scale that dynamically adjusts itself.
class FixedLossScale: Loss scale with a fixed value.
class LossScale: Base class for all TF1 loss scales.


An optimizer that applies loss scaling.
Inherits From: Optimizer
Compat aliases for migration
See Migration guide for more details.
tf.compat.v1.train.experimental.MixedPrecisionLossScaleOptimizer
Loss scaling is a process that multiplies the loss by a multiplier called the loss scale, and divides each gradient by the same multiplier. The pseudocode for this process is:
Mathematically, loss scaling has no effect, but can help avoid numerical underflow in intermediate gradients when float16 tensors are used for mixed precision training. By multiplying the loss, each intermediate gradient will have the same multiplier applied.
The loss scale can either be a fixed constant, chosen by the user, or be dynamically determined. Dynamically determining the loss scale is convenient as a loss scale does not have to be explicitly chosen. However it reduces performance.
This optimizer wraps another optimizer and applies loss scaling to it via a LossScale. Loss scaling is applied whenever gradients are computed, such as through minimize().
View source
Apply gradients to variables.
This is the second part of minimize(). It returns an Operation that conditionally applies gradients if all gradient values are finite. Otherwise no update is performed (nor is global_step incremented).
View source
Compute gradients of loss for the variables in var_list.
This adjusts the dynamic range of the gradient evaluation by scaling up the loss value. The gradient values are then scaled back down by the reciprocal of the loss scale. This is useful in reduced precision training where small gradient values would otherwise underflow the representable range.
View source
View source
Return a slot named name created for var by the Optimizer.
Some Optimizer subclasses use additional variables. For example Momentum and Adagrad use variables to accumulate updates. This method gives access to these Variable objects if for some reason you need them.
Use get_slot_names() to get the list of slot names created by the Optimizer.
View source
Return a list of the names of slots created by the Optimizer.
See get_slot().
View source
Add operations to minimize loss by updating var_list.
This method simply combines calls compute_gradients() and apply_gradients(). If you want to process the gradient before applying them call compute_gradients() and apply_gradients() explicitly instead of using this function.
When eager execution is enabled, loss should be a Python function that takes no arguments and computes the value to be minimized. Minimization (and gradient computation) is done with respect to the elements of var_list if not None, else with respect to any trainable variables created during the execution of the loss function. gate_gradients, aggregation_method, colocate_gradients_with_ops and grad_loss are ignored when eager execution is enabled.
View source
Returns the variables of the Optimizer.


Public API for tf.mlir namespace.
experimental module: Public API for tf.mlir.experimental namespace.


Public API for tf.mlir.experimental namespace.
convert_function(...): Import a ConcreteFunction and convert it to a textual MLIR module.
convert_graph_def(...): Import a GraphDef and convert it to a textual MLIR module.


Returns all variables in the MODEL_VARIABLES collection.


Returns all variables that maintain their moving averages.
If an ExponentialMovingAverage object is created and the apply() method is called on a list of variables, these variables will be added to the GraphKeys.MOVING_AVERAGE_VARIABLES collection. This convenience function returns the contents of that collection.


Draws samples from a multinomial distribution. (deprecated)
Compat aliases for migration
See Migration guide for more details.
tf.compat.v1.random.multinomial


Public API for tf.nest namespace.
assert_same_structure(...): Asserts that two structures are nested in the same way.
flatten(...): Returns a flat list from a given nested structure.
is_nested(...): Returns true if its input is a collections.abc.Sequence (except strings).
map_structure(...): Applies func to each entry in structure and returns a new structure.
pack_sequence_as(...): Returns a given flattened sequence packed into a given structure.


Wrappers for primitive Neural Net (NN) Operations.
rnn_cell module: Module for constructing RNN Cells.
all_candidate_sampler(...): Generate the set of all classes.
atrous_conv2d(...): Atrous convolution (a.k.a. convolution with holes or dilated convolution).
atrous_conv2d_transpose(...): The transpose of atrous_conv2d.
avg_pool(...): Performs the average pooling on the input.
avg_pool1d(...): Performs the average pooling on the input.
avg_pool2d(...): Performs the average pooling on the input.
avg_pool3d(...): Performs the average pooling on the input.
avg_pool_v2(...): Performs the avg pooling on the input.
batch_norm_with_global_normalization(...): Batch normalization.
batch_normalization(...): Batch normalization.
bias_add(...): Adds bias to value.
bidirectional_dynamic_rnn(...): Creates a dynamic version of bidirectional recurrent neural network. (deprecated)
collapse_repeated(...): Merge repeated labels into single labels.
compute_accidental_hits(...): Compute the position ids in sampled_candidates matching true_classes.
compute_average_loss(...): Scales per-example losses with sample_weights and computes their average.
conv1d(...): Computes a 1-D convolution of input with rank >=3 and a 3-D filter. (deprecated argument values) (deprecated argument values)
conv1d_transpose(...): The transpose of conv1d.
conv2d(...): Computes a 2-D convolution given 4-D input and filter tensors.
conv2d_backprop_filter(...): Computes the gradients of convolution with respect to the filter.
conv2d_backprop_input(...): Computes the gradients of convolution with respect to the input.
conv2d_transpose(...): The transpose of conv2d.
conv3d(...): Computes a 3-D convolution given 5-D input and filter tensors.
conv3d_backprop_filter(...): Computes the gradients of 3-D convolution with respect to the filter.
conv3d_backprop_filter_v2(...): Computes the gradients of 3-D convolution with respect to the filter.
conv3d_transpose(...): The transpose of conv3d.
conv_transpose(...): The transpose of convolution.
convolution(...): Computes sums of N-D convolutions (actually cross-correlation).
crelu(...): Computes Concatenated ReLU.
ctc_beam_search_decoder(...): Performs beam search decoding on the logits given in input.
ctc_beam_search_decoder_v2(...): Performs beam search decoding on the logits given in input.
ctc_greedy_decoder(...): Performs greedy decoding on the logits given in input (best path).
ctc_loss(...): Computes the CTC (Connectionist Temporal Classification) Loss.
ctc_loss_v2(...): Computes CTC (Connectionist Temporal Classification) loss.
ctc_unique_labels(...): Get unique labels and indices for batched labels for tf.nn.ctc_loss.
depth_to_space(...): DepthToSpace for tensors of type T.
depthwise_conv2d(...): Depthwise 2-D convolution.
depthwise_conv2d_backprop_filter(...): Computes the gradients of depthwise convolution with respect to the filter.
depthwise_conv2d_backprop_input(...): Computes the gradients of depthwise convolution with respect to the input.
depthwise_conv2d_native(...): Computes a 2-D depthwise convolution.
depthwise_conv2d_native_backprop_filter(...): Computes the gradients of depthwise convolution with respect to the filter.
depthwise_conv2d_native_backprop_input(...): Computes the gradients of depthwise convolution with respect to the input.
dilation2d(...): Computes the grayscale dilation of 4-D input and 3-D filter tensors.
dropout(...): Computes dropout. (deprecated arguments)
dynamic_rnn(...): Creates a recurrent neural network specified by RNNCell cell. (deprecated)
elu(...): Computes exponential linear: exp(features) - 1 if < 0, features otherwise.
embedding_lookup(...): Looks up embeddings for the given ids from a list of tensors.
embedding_lookup_sparse(...): Looks up embeddings for the given ids and weights from a list of tensors.
erosion2d(...): Computes the grayscale erosion of 4-D value and 3-D kernel tensors.
fixed_unigram_candidate_sampler(...): Samples a set of classes using the provided (fixed) base distribution.
fractional_avg_pool(...): Performs fractional average pooling on the input. (deprecated)
fractional_max_pool(...): Performs fractional max pooling on the input. (deprecated)
fused_batch_norm(...): Batch normalization.
in_top_k(...): Says whether the targets are in the top K predictions.
l2_loss(...): L2 Loss.
l2_normalize(...): Normalizes along dimension axis using an L2 norm. (deprecated arguments)
leaky_relu(...): Compute the Leaky ReLU activation function.
learned_unigram_candidate_sampler(...): Samples a set of classes from a distribution learned during training.
local_response_normalization(...): Local Response Normalization.
log_poisson_loss(...): Computes log Poisson loss given log_input.
log_softmax(...): Computes log softmax activations. (deprecated arguments)
log_uniform_candidate_sampler(...): Samples a set of classes using a log-uniform (Zipfian) base distribution.
lrn(...): Local Response Normalization.
max_pool(...): Performs the max pooling on the input.
max_pool1d(...): Performs the max pooling on the input.
max_pool2d(...): Performs the max pooling on the input.
max_pool3d(...): Performs the max pooling on the input.
max_pool_v2(...): Performs the max pooling on the input.
max_pool_with_argmax(...): Performs max pooling on the input and outputs both max values and indices.
moments(...): Calculate the mean and variance of x.
nce_loss(...): Computes and returns the noise-contrastive estimation training loss.
normalize_moments(...): Calculate the mean and variance of based on the sufficient statistics.
pool(...): Performs an N-D pooling operation.
quantized_avg_pool(...): Produces the average pool of the input tensor for quantized types.
quantized_conv2d(...): Computes a 2D convolution given quantized 4D input and filter tensors.
quantized_max_pool(...): Produces the max pool of the input tensor for quantized types.
quantized_relu_x(...): Computes Quantized Rectified Linear X: min(max(features, 0), max_value)
raw_rnn(...): Creates an RNN specified by RNNCell cell and loop function loop_fn.
relu(...): Computes rectified linear: max(features, 0).
relu6(...): Computes Rectified Linear 6: min(max(features, 0), 6).
relu_layer(...): Computes Relu(x * weight + biases).
safe_embedding_lookup_sparse(...): Lookup embedding results, accounting for invalid IDs and empty features.
sampled_softmax_loss(...): Computes and returns the sampled softmax training loss.
scale_regularization_loss(...): Scales the sum of the given regularization losses by number of replicas.
selu(...): Computes scaled exponential linear: scale * alpha * (exp(features) - 1)
separable_conv2d(...): 2-D convolution with separable filters.
sigmoid(...): Computes sigmoid of x element-wise.
sigmoid_cross_entropy_with_logits(...): Computes sigmoid cross entropy given logits.
silu(...): Computes the SiLU or Swish activation function: x * sigmoid(x).
softmax(...): Computes softmax activations. (deprecated arguments)
softmax_cross_entropy_with_logits(...): Computes softmax cross entropy between logits and labels. (deprecated)
softmax_cross_entropy_with_logits_v2(...): Computes softmax cross entropy between logits and labels. (deprecated arguments)
softplus(...): Computes softplus: log(exp(features) + 1).
softsign(...): Computes softsign: features / (abs(features) + 1).
space_to_batch(...): SpaceToBatch for 4-D tensors of type T.
space_to_depth(...): SpaceToDepth for tensors of type T.
sparse_softmax_cross_entropy_with_logits(...): Computes sparse softmax cross entropy between logits and labels.
static_bidirectional_rnn(...): Creates a bidirectional recurrent neural network. (deprecated)
static_rnn(...): Creates a recurrent neural network specified by RNNCell cell. (deprecated)
static_state_saving_rnn(...): RNN that accepts a state saver for time-truncated RNN calculation. (deprecated)
sufficient_statistics(...): Calculate the sufficient statistics for the mean and variance of x.
swish(...): Computes the SiLU or Swish activation function: x * sigmoid(x).
tanh(...): Computes hyperbolic tangent of x element-wise.
top_k(...): Finds values and indices of the k largest entries for the last dimension.
uniform_candidate_sampler(...): Samples a set of classes using a uniform base distribution.
weighted_cross_entropy_with_logits(...): Computes a weighted cross entropy. (deprecated arguments)
weighted_moments(...): Returns the frequency-weighted mean and variance of x.
with_space_to_batch(...): Performs op on the space-to-batch representation of input.
xw_plus_b(...): Computes matmul(x, weights) + biases.
zero_fraction(...): Returns the fraction of zeros in value.


Performs the average pooling on the input.
Compat aliases for migration
See Migration guide for more details.
tf.compat.v1.nn.avg_pool2d
Each entry in output is the mean of the corresponding size ksize window in value.


Batch normalization.
This op is deprecated. See tf.nn.batch_normalization.
Batch Normalization - Accelerating Deep Network Training by Reducing Internal Covariate Shift: Ioffe et al., 2015 (pdf)


Creates a dynamic version of bidirectional recurrent neural network. (deprecated)
Takes input and builds independent forward and backward RNNs. The input_size of forward and backward cell must match. The initial state for both directions is zero by default (but can be set optionally) and no intermediate states are ever returned -- the network is fully unrolled for the given (passed in) length(s) of the sequence(s) or completely unrolled if length(s) is not given.


Computes a 1-D convolution of input with rank >=3 and a 3-D filter. (deprecated argument values) (deprecated argument values)
Given an input tensor of shape batch_shape + [in_width, in_channels] if data_format is "NWC", or batch_shape + [in_channels, in_width] if data_format is "NCW", and a filter / kernel tensor of shape [filter_width, in_channels, out_channels], this op reshapes the arguments to pass them to conv2d to perform the equivalent convolution operation.
Internally, this op reshapes the input tensors and invokes tf.nn.conv2d. For example, if data_format does not start with "NC", a tensor of shape batch_shape + [in_width, in_channels] is reshaped to batch_shape + [1, in_width, in_channels], and the filter is reshaped to [1, filter_width, in_channels, out_channels]. The result is then reshaped back to batch_shape + [out_width, out_channels] (where out_width is a function of the stride and padding as in conv2d) and returned to the caller.


Computes a 2-D convolution given 4-D input and filter tensors.
Given an input tensor of shape [batch, in_height, in_width, in_channels] and a filter / kernel tensor of shape [filter_height, filter_width, in_channels, out_channels], this op performs the following:
In detail, with the default NHWC format,
Must have strides[0] = strides[3] = 1. For the most common case of the same horizontal and vertical strides, strides = [1, stride, stride, 1].


Computes the gradients of convolution with respect to the filter.


Computes the gradients of convolution with respect to the input.


The transpose of conv2d.
This operation is sometimes called "deconvolution" after (Zeiler et al., 2010), but is really the transpose (gradient) of conv2d rather than an actual deconvolution.
Deconvolutional Networks: Zeiler et al., 2010 (pdf)


Computes a 3-D convolution given 5-D input and filter tensors.
In signal processing, cross-correlation is a measure of similarity of two waveforms as a function of a time-lag applied to one of them. This is also known as a sliding dot product or sliding inner-product.
Our Conv3D implements a form of cross-correlation.


Computes the gradients of 3-D convolution with respect to the filter.
Compat aliases for migration
See Migration guide for more details.
tf.compat.v1.nn.conv3d_backprop_filter_v2


The transpose of conv3d.
This operation is sometimes called "deconvolution" after (Zeiler et al., 2010), but is really the transpose (gradient) of conv3d rather than an actual deconvolution.
Deconvolutional Networks: Zeiler et al., 2010 (pdf)


Computes sums of N-D convolutions (actually cross-correlation).
This also supports either output striding via the optional strides parameter or atrous convolution (also known as convolution with holes or dilated convolution, based on the French word "trous" meaning holes in English) via the optional dilation_rate parameter. Currently, however, output striding is not supported for atrous convolutions.
Specifically, in the case that data_format does not start with "NC", given a rank (N+2) input Tensor of shape
[num_batches, input_spatial_shape[0], ..., input_spatial_shape[N-1], num_input_channels],
a rank (N+2) filter Tensor of shape
[spatial_filter_shape[0], ..., spatial_filter_shape[N-1], num_input_channels, num_output_channels],
an optional dilation_rate tensor of shape N specifying the filter upsampling/input downsampling rate, and an optional list of N strides (defaulting [1]*N), this computes for each N-D spatial output position (x[0], ..., x[N-1]):
where b is the index into the batch, k is the output channel number, q is the input channel number, and z is the N-D spatial offset within the filter. Here, padded_input is obtained by zero padding the input using an effective spatial filter shape of (spatial_filter_shape-1) * dilation_rate + 1 and output striding strides as described in the comment here.
In the case that data_format does start with "NC", the input and output (but not the filter) are simply transposed as follows:
convolution(input, data_format, **kwargs) = tf.transpose(convolution(tf.transpose(input, [0] + range(2,N+2) + [1]), **kwargs), [0, N+1] + range(1, N+1))
It is required that 1 <= N <= 3.
[batch_size] + output_spatial_shape + [out_channels]
if data_format is None or does not start with "NC", or
[batch_size, out_channels] + output_spatial_shape
if data_format starts with "NC", where output_spatial_shape depends on the value of padding.
If padding == "SAME": output_spatial_shape[i] = ceil(input_spatial_shape[i] / strides[i])
If padding == "VALID": output_spatial_shape[i] = ceil((input_spatial_shape[i] - (spatial_filter_shape[i]-1) * dilation_rate[i]) / strides[i]).


Computes Concatenated ReLU.
Concatenates a ReLU which selects only the positive part of the activation with a ReLU which selects only the negative part of the activation. Note that as a result this non-linearity doubles the depth of the activations. Source: Understanding and Improving Convolutional Neural Networks via Concatenated Rectified Linear Units. W. Shang, et al.
Understanding and Improving Convolutional Neural Networks via Concatenated Rectified Linear Units: Shang et al., 2016 (pdf)


Performs beam search decoding on the logits given in input.
If merge_repeated is True, merge repeated classes in the output beams. This means that if consecutive entries in a beam are the same, only the first of these is emitted. That is, when the sequence is A B B * B * B (where '*' is the blank label), the return value is:
decoded[j].indices: Indices matrix (total_decoded_outputs[j] x 2) The rows store: [batch, time].
decoded[j].values: Values vector, size (total_decoded_outputs[j]). The vector stores the decoded classes for beam j.
decoded[j].dense_shape: Shape vector, size (2). The shape values are: [batch_size, max_decoded_length[j]].


Computes the CTC (Connectionist Temporal Classification) Loss.
This op implements the CTC loss as presented in (Graves et al., 2006).
This class performs the softmax operation for you, so inputs should be e.g. linear projections of outputs by an LSTM.
The inputs Tensor's innermost dimension size, num_classes, represents num_labels + 1 classes, where num_labels is the number of true labels, and the largest value (num_classes - 1) is reserved for the blank label.
For example, for a vocabulary containing 3 labels [a, b, c], num_classes = 4 and the labels indexing is {a: 0, b: 1, c: 2, blank: 3}.
Regarding the arguments preprocess_collapse_repeated and ctc_merge_repeated:
If preprocess_collapse_repeated is True, then a preprocessing step runs before loss calculation, wherein repeated labels passed to the loss are merged into single labels. This is useful if the training labels come from, e.g., forced alignments and therefore have unnecessary repetitions.
If ctc_merge_repeated is set False, then deep within the CTC calculation, repeated non-blank labels will not be merged and are interpreted as individual labels. This is a simplified (non-standard) version of CTC.
Here is a table of the (roughly) expected first order behavior:
preprocess_collapse_repeated=False, ctc_merge_repeated=True
Classical CTC behavior: Outputs true repeated classes with blanks in between, and can also output repeated classes with no blanks in between that need to be collapsed by the decoder.
preprocess_collapse_repeated=True, ctc_merge_repeated=False
Never learns to output repeated classes, as they are collapsed in the input labels before training.
preprocess_collapse_repeated=False, ctc_merge_repeated=False
Outputs repeated classes with blanks in between, but generally does not require the decoder to collapse/merge repeated classes.
preprocess_collapse_repeated=True, ctc_merge_repeated=True
Untested. Very likely will not learn to output repeated classes.
The ignore_longer_outputs_than_inputs option allows to specify the behavior of the CTCLoss when dealing with sequences that have longer outputs than inputs. If true, the CTCLoss will simply return zero gradient for those items, otherwise an InvalidArgument error is returned, stopping training.
Connectionist Temporal Classification - Labeling Unsegmented Sequence Data with Recurrent Neural Networks: Graves et al., 2006 (pdf)


Computes CTC (Connectionist Temporal Classification) loss.
This op implements the CTC loss as presented in (Graves et al., 2006).
Connectionist Temporal Classification - Labeling Unsegmented Sequence Data with Recurrent Neural Networks: Graves et al., 2006 (pdf)


Depthwise 2-D convolution.
Given a 4D input tensor ('NHWC' or 'NCHW' data formats) and a filter tensor of shape [filter_height, filter_width, in_channels, channel_multiplier] containing in_channels convolutional filters of depth 1, depthwise_conv2d applies a different filter to each input channel (expanding from 1 channel to channel_multiplier channels for each), then concatenates the results together. The output has in_channels * channel_multiplier channels.
In detail, with the default NHWC format,
Must have strides[0] = strides[3] = 1. For the most common case of the same horizontal and vertical strides, strides = [1, stride, stride, 1]. If any value in rate is greater than 1, we perform atrous depthwise convolution, in which case all values in the strides tensor must be equal to 1.


Computes a 2-D depthwise convolution.
Given an input tensor of shape [batch, in_height, in_width, in_channels] and a filter / kernel tensor of shape [filter_height, filter_width, in_channels, channel_multiplier], containing in_channels convolutional filters of depth 1, depthwise_conv2d applies a different filter to each input channel (expanding from 1 channel to channel_multiplier channels for each), then concatenates the results together. Thus, the output has in_channels * channel_multiplier channels.
Must have strides[0] = strides[3] = 1. For the most common case of the same horizontal and vertices strides, strides = [1, stride, stride, 1].


Computes the grayscale dilation of 4-D input and 3-D filter tensors.
The input tensor has shape [batch, in_height, in_width, depth] and the filter tensor has shape [filter_height, filter_width, depth], i.e., each input channel is processed independently of the others with its own structuring function. The output tensor has shape [batch, out_height, out_width, depth]. The spatial dimensions of the output tensor depend on the padding algorithm. We currently only support the default "NHWC" data_format.
In detail, the grayscale morphological 2-D dilation is the max-sum correlation (for consistency with conv2d, we use unmirrored filters):
Max-pooling is a special case when the filter has size equal to the pooling kernel size and contains all zeros.
Note on duality: The dilation of input by the filter is equal to the negation of the erosion of -input by the reflected filter.


Computes dropout. (deprecated arguments)
For each element of x, with probability rate, outputs 0, and otherwise scales up the input by 1 / (1-rate). The scaling is such that the expected sum is unchanged.
By default, each element is kept or dropped independently. If noise_shape is specified, it must be broadcastable to the shape of x, and only dimensions with noise_shape[i] == shape(x)[i] will make independent decisions. For example, if shape(x) = [k, l, m, n] and noise_shape = [k, 1, 1, n], each batch and channel component will be kept independently and each row and column will be kept or not kept together.


Creates a recurrent neural network specified by RNNCell cell. (deprecated)
Performs fully dynamic unrolling of inputs.
If time_major == False (default), this will be a Tensor shaped: [batch_size, max_time, cell.output_size].
If time_major == True, this will be a Tensor shaped: [max_time, batch_size, cell.output_size].
Note, if cell.output_size is a (possibly nested) tuple of integers or TensorShape objects, then outputs will be a tuple having the same structure as cell.output_size, containing Tensors having shapes corresponding to the shape data in cell.output_size.


Looks up embeddings for the given ids from a list of tensors.
This function is used to perform parallel lookups on the list of tensors in params. It is a generalization of tf.gather, where params is interpreted as a partitioning of a large embedding tensor. params may be a PartitionedVariable as returned by using tf.compat.v1.get_variable() with a partitioner.
If len(params) > 1, each element id of ids is partitioned between the elements of params according to the partition_strategy. In all strategies, if the id space does not evenly divide the number of partitions, each of the first (max_id + 1) % len(params) partitions will be assigned one more id.
If partition_strategy is "mod", we assign each id to partition p = id % len(params). For instance, 13 ids are split across 5 partitions as: [[0, 5, 10], [1, 6, 11], [2, 7, 12], [3, 8], [4, 9]]
If partition_strategy is "div", we assign ids to partitions in a contiguous manner. In this case, 13 ids are split across 5 partitions as: [[0, 1, 2], [3, 4, 5], [6, 7, 8], [9, 10], [11, 12]]
If the input ids are ragged tensors, partition variables are not supported and the partition strategy and the max_norm are ignored. The results of the lookup are concatenated into a dense tensor. The returned tensor has shape shape(ids) + shape(params)[1:].


Looks up embeddings for the given ids and weights from a list of tensors.
This op assumes that there is at least one id for each row in the dense tensor represented by sp_ids (i.e. there are no rows with empty features), and that all the indices of sp_ids are in canonical row-major order.
sp_ids and sp_weights (if not None) are SparseTensors with rank of 2. Embeddings are always aggregated along the last dimension.
It also assumes that all id values lie in the range [0, p0), where p0 is the sum of the size of params along dimension 0.
In other words, if
shape(combined params) = [p0, p1, ..., pm]
and
shape(sp_ids) = shape(sp_weights) = [d0, d1]
then
shape(output) = [d0, p1, ..., pm].
For instance, if params is a 10x20 matrix, and sp_ids / sp_weights are
with combiner="mean", then the output will be a 3x20 matrix where


Computes the grayscale erosion of 4-D value and 3-D kernel tensors.
The value tensor has shape [batch, in_height, in_width, depth] and the kernel tensor has shape [kernel_height, kernel_width, depth], i.e., each input channel is processed independently of the others with its own structuring function. The output tensor has shape [batch, out_height, out_width, depth]. The spatial dimensions of the output tensor depend on the padding algorithm. We currently only support the default "NHWC" data_format.
In detail, the grayscale morphological 2-D erosion is given by:
Duality: The erosion of value by the kernel is equal to the negation of the dilation of -value by the reflected kernel.


Performs fractional average pooling on the input. (deprecated)
This is a deprecated version of fractional_avg_pool.
Fractional average pooling is similar to Fractional max pooling in the pooling region generation step. The only difference is that after pooling regions are generated, a mean operation is performed instead of a max operation in each pooling region.
A tuple of Tensor objects (output, row_pooling_sequence, col_pooling_sequence). output: Output Tensor after fractional avg pooling. Has the same type as value. row_pooling_sequence: A Tensor of type int64. col_pooling_sequence: A Tensor of type int64.
Fractional Max-Pooling: Graham, 2015 (pdf)


Performs fractional max pooling on the input. (deprecated)
This is a deprecated version of fractional_max_pool.
Fractional max pooling is slightly different than regular max pooling. In regular max pooling, you downsize an input set by taking the maximum value of smaller N x N subsections of the set (often 2x2), and try to reduce the set by a factor of N, where N is an integer. Fractional max pooling, as you might expect from the word "fractional", means that the overall reduction ratio N does not have to be an integer.
The sizes of the pooling regions are generated randomly but are fairly uniform. For example, let's look at the height dimension, and the constraints on the list of rows that will be pool boundaries.
First we define the following:
Then, row_pooling_sequence should satisfy:
A tuple of Tensor objects (output, row_pooling_sequence, col_pooling_sequence). output: Output Tensor after fractional max pooling. Has the same type as value. row_pooling_sequence: A Tensor of type int64. col_pooling_sequence: A Tensor of type int64.
Fractional Max-Pooling: Graham, 2015 (pdf)


Batch normalization.
See Source: Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift; S. Ioffe, C. Szegedy.
Batch Normalization - Accelerating Deep Network Training by Reducing Internal Covariate Shift: Ioffe et al., 2015 (pdf)


Performs max pooling on the input and outputs both max values and indices.
The indices in argmax are flattened, so that a maximum value at position [b, y, x, c] becomes flattened index: (y * width + x) * channels + c if include_batch_in_index is False; ((b * height + y) * width + x) * channels + c if include_batch_in_index is True.
The indices returned are always in [0, height) x [0, width) before flattening, even if padding is involved and the mathematically correct answer is outside (either negative or too large). This is a bug, but fixing it is difficult to do in a safe backwards compatible way, especially due to flattening.


Calculate the mean and variance of x.
The mean and variance are calculated by aggregating the contents of x across axes. If x is 1-D and axes = [0] this is just the mean and variance of a vector.
When using these moments for batch normalization (see tf.nn.batch_normalization):


Computes and returns the noise-contrastive estimation training loss.
A common use case is to use this method for training, and calculate the full sigmoid loss for evaluation or inference. In this case, you must set partition_strategy="div" for the two losses to be consistent, as in the following example:
Noise-contrastive estimation - A new estimation principle for unnormalized statistical models: Gutmann et al., 2010 (pdf)


Performs an N-D pooling operation.
In the case that data_format does not start with "NC", computes for 0 <= b < batch_size, 0 <= x[i] < output_spatial_shape[i], 0 <= c < num_channels:
where the reduction function REDUCE depends on the value of pooling_type, and pad_before is defined based on the value of padding as described in the "returns" section of tf.nn.convolution for details. The reduction never includes out-of-bounds positions.
In the case that data_format starts with "NC", the input and output are simply transposed as follows:
if data_format is None or does not start with "NC", or
[batch_size, num_channels] + output_spatial_shape
if data_format starts with "NC", where output_spatial_shape depends on the value of padding:
If padding = "SAME": output_spatial_shape[i] = ceil(input_spatial_shape[i] / strides[i])
If padding = "VALID": output_spatial_shape[i] = ceil((input_spatial_shape[i] - (window_shape[i] - 1) * dilation_rate[i]) / strides[i]).


Produces the average pool of the input tensor for quantized types.


Computes a 2D convolution given quantized 4D input and filter tensors.
The inputs are quantized tensors where the lowest value represents the real number of the associated minimum, and the highest represents the maximum. This means that you can only interpret the quantized output in the same way, by taking the returned minimum and maximum values into account.


Produces the max pool of the input tensor for quantized types.


Computes Quantized Rectified Linear X: min(max(features, 0), max_value)


Creates an RNN specified by RNNCell cell and loop function loop_fn.
This function is a more primitive version of dynamic_rnn that provides more direct access to the inputs each iteration. It also provides more control over when to start and finish reading the sequence, and what to emit for the output.
For example, it can be used to implement the dynamic decoder of a seq2seq model.
Instead of working with Tensor objects, most operations work with TensorArray objects directly.
The operation of raw_rnn, in pseudo-code, is basically the following:
with the additional properties that output and state may be (possibly nested) tuples, as determined by cell.output_size and cell.state_size, and as a result the final state and emit_ta may themselves be tuples.
A simple implementation of dynamic_rnn via raw_rnn looks like this:
emit_ta: The RNN output TensorArray. If loop_fn returns a (possibly nested) set of Tensors for emit_output during initialization, (inputs time = 0, cell_output = None, and loop_state = None), then emit_ta will have the same structure, dtypes, and shapes as emit_output instead. If loop_fn returns emit_output = None during this call, the structure of cell.output_size is used: If cell.output_size is a (possibly nested) tuple of integers or TensorShape objects, then emit_ta will be a tuple having the same structure as cell.output_size, containing TensorArrays whose elements' shapes correspond to the shape data in cell.output_size.
final_state: The final cell state. If cell.state_size is an int, this will be shaped [batch_size, cell.state_size]. If it is a TensorShape, this will be shaped [batch_size] + cell.state_size. If it is a (possibly nested) tuple of ints or TensorShape, this will be a tuple having the corresponding shapes.
final_loop_state: The final loop state as returned by loop_fn.


Module for constructing RNN Cells.
class BasicLSTMCell: DEPRECATED: Please use tf.compat.v1.nn.rnn_cell.LSTMCell instead.
class BasicRNNCell: The most basic RNN cell.
class DeviceWrapper: Operator that ensures an RNNCell runs on a particular device.
class DropoutWrapper: Operator adding dropout to inputs and outputs of the given cell.
class GRUCell: Gated Recurrent Unit cell.
class LSTMCell: Long short-term memory unit (LSTM) recurrent network cell.
class LSTMStateTuple: Tuple used by LSTM Cells for state_size, zero_state, and output state.
class MultiRNNCell: RNN cell composed sequentially of multiple simple cells.
class RNNCell: Abstract object representing an RNN cell.
class ResidualWrapper: RNNCell wrapper that ensures cell inputs are added to the outputs.


DEPRECATED: Please use tf.compat.v1.nn.rnn_cell.LSTMCell instead.
Inherits From: RNNCell, Layer, Layer, Module
Basic LSTM recurrent network cell.
The implementation is based on
We add forget_bias (default: 1) to the biases of the forget gate in order to reduce the scale of forgetting in the beginning of the training.
It does not allow cell clipping, a projection layer, and does not use peep-hole connections: it is the basic baseline.
For advanced models, please use the full tf.compat.v1.nn.rnn_cell.LSTMCell that follows.
Note that this cell is not optimized for performance. Please use tf.contrib.cudnn_rnn.CudnnLSTM for better performance on GPU, or tf.contrib.rnn.LSTMBlockCell and tf.contrib.rnn.LSTMBlockFusedCell for better performance on CPU.
It can be represented by an Integer, a TensorShape or a tuple of Integers or TensorShapes.
View source
View source
Return zero-filled state tensor(s).
If state_size is a nested list or tuple, then the return value is a nested list or tuple (of the same structure) of 2-D tensors with the shapes [batch_size, s] for each s in state_size.


The most basic RNN cell.
Inherits From: RNNCell, Layer, Layer, Module
Note that this cell is not optimized for performance. Please use tf.contrib.cudnn_rnn.CudnnRNNTanh for better performance on GPU.
It can be represented by an Integer, a TensorShape or a tuple of Integers or TensorShapes.
View source
View source
Return zero-filled state tensor(s).
If state_size is a nested list or tuple, then the return value is a nested list or tuple (of the same structure) of 2-D tensors with the shapes [batch_size, s] for each s in state_size.


Operator that ensures an RNNCell runs on a particular device.
Inherits From: RNNCell, Layer, Layer, Module
View source
View source


Operator adding dropout to inputs and outputs of the given cell.
Inherits From: RNNCell, Layer, Layer, Module
View source
View source


Gated Recurrent Unit cell.
Inherits From: RNNCell, Layer, Layer, Module
Note that this cell is not optimized for performance. Please use tf.contrib.cudnn_rnn.CudnnGRU for better performance on GPU, or tf.contrib.rnn.GRUBlockCellV2 for better performance on CPU.
References: Learning Phrase Representations using RNN Encoder Decoder for Statistical Machine Translation: Cho et al., 2014 (pdf)
It can be represented by an Integer, a TensorShape or a tuple of Integers or TensorShapes.
View source
View source
Return zero-filled state tensor(s).
If state_size is a nested list or tuple, then the return value is a nested list or tuple (of the same structure) of 2-D tensors with the shapes [batch_size, s] for each s in state_size.


Long short-term memory unit (LSTM) recurrent network cell.
Inherits From: RNNCell, Layer, Layer, Module
The default non-peephole implementation is based on (Gers et al., 1999). The peephole implementation is based on (Sak et al., 2014).
The class uses optional peep-hole connections, optional cell clipping, and an optional projection layer.
Note that this cell is not optimized for performance. Please use tf.contrib.cudnn_rnn.CudnnLSTM for better performance on GPU, or tf.contrib.rnn.LSTMBlockCell and tf.contrib.rnn.LSTMBlockFusedCell for better performance on CPU. References: Long short-term memory recurrent neural network architectures for large scale acoustic modeling: Sak et al., 2014 (pdf) Learning to forget: Gers et al., 1999 (pdf) Long Short-Term Memory: Hochreiter et al., 1997 (pdf)
It can be represented by an Integer, a TensorShape or a tuple of Integers or TensorShapes.
View source
View source
Return zero-filled state tensor(s).
If state_size is a nested list or tuple, then the return value is a nested list or tuple (of the same structure) of 2-D tensors with the shapes [batch_size, s] for each s in state_size.


Tuple used by LSTM Cells for state_size, zero_state, and output state.
Stores two elements: (c, h), in that order. Where c is the hidden state and h is the output.
Only used when state_is_tuple=True.


RNN cell composed sequentially of multiple simple cells.
Inherits From: RNNCell, Layer, Layer, Module
It can be represented by an Integer, a TensorShape or a tuple of Integers or TensorShapes.
View source
View source
Return zero-filled state tensor(s).
If state_size is a nested list or tuple, then the return value is a nested list or tuple (of the same structure) of 2-D tensors with the shapes [batch_size, s] for each s in state_size.


RNNCell wrapper that ensures cell inputs are added to the outputs.
Inherits From: RNNCell, Layer, Layer, Module
View source
View source


Abstract object representing an RNN cell.
Inherits From: Layer, Layer, Module
Every RNNCell must have the properties below and implement call with the signature (output, next_state) = call(input, state). The optional third input argument, scope, is allowed for backwards compatibility purposes; but should be left off for new subclasses.
This definition of cell differs from the definition used in the literature. In the literature, 'cell' refers to an object with a single scalar output. This definition refers to a horizontal array of such units.
An RNN cell, in the most abstract setting, is anything that has a state and performs some operation that takes a matrix of inputs. This operation results in an output matrix with self.output_size columns. If self.state_size is an integer, this operation also results in a new state matrix with self.state_size columns. If self.state_size is a (possibly nested tuple of) TensorShape object(s), then it should return a matching structure of Tensors having shape [batch_size].concatenate(s) for each s in self.batch_size.
It can be represented by an Integer, a TensorShape or a tuple of Integers or TensorShapes.
View source
View source
Return zero-filled state tensor(s).
If state_size is a nested list or tuple, then the return value is a nested list or tuple (of the same structure) of 2-D tensors with the shapes [batch_size, s] for each s in state_size.


Lookup embedding results, accounting for invalid IDs and empty features.
The partitioned embedding in embedding_weights must all be the same shape except for the first dimension. The first dimension is allowed to vary as the vocabulary size is not necessarily a multiple of P. embedding_weights may be a PartitionedVariable as returned by using tf.compat.v1.get_variable() with a partitioner.
Invalid IDs (< 0) are pruned from input IDs and weights, as well as any IDs with non-positive weight. For an entry with no features, the embedding vector for default_id is returned, or the 0-vector if default_id is not supplied.
The ids and weights may be multi-dimensional. Embeddings are always aggregated along the last dimension.
In other words, if
shape(combined embedding_weights) = [p0, p1, ..., pm]
and
shape(sparse_ids) = shape(sparse_weights) = [d0, d1, ..., dn]
then
shape(output) = [d0, d1, ... dn-1, p1, ..., pm].
For instance, if params is a 10x20 matrix, and sp_ids / sp_weights are
default_id is 0.
with combiner="mean", then the output will be a 3x20 matrix where


Computes and returns the sampled softmax training loss.
This is a faster way to train a softmax classifier over a huge number of classes.
This operation is for training only. It is generally an underestimate of the full softmax loss.
A common use case is to use this method for training, and calculate the full softmax loss for evaluation or inference. In this case, you must set partition_strategy="div" for the two losses to be consistent, as in the following example:
See our Candidate Sampling Algorithms Reference (pdf). Also see Section 3 of (Jean et al., 2014) for the math.
On Using Very Large Target Vocabulary for Neural Machine Translation: Jean et al., 2014 (pdf)


2-D convolution with separable filters.
Performs a depthwise convolution that acts separately on channels followed by a pointwise convolution that mixes channels. Note that this is separability between dimensions [1, 2] and 3, not spatial separability between dimensions 1 and 2.
In detail, with the default NHWC format,
strides controls the strides for the depthwise convolution only, since the pointwise convolution has implicit strides of [1, 1, 1, 1]. Must have strides[0] = strides[3] = 1. For the most common case of the same horizontal and vertical strides, strides = [1, stride, stride, 1]. If any value in rate is greater than 1, we perform atrous depthwise convolution, in which case all values in the strides tensor must be equal to 1.


Computes sigmoid cross entropy given logits.
Measures the probability error in discrete classification tasks in which each class is independent and not mutually exclusive. For instance, one could perform multilabel classification where a picture can contain both an elephant and a dog at the same time.
For brevity, let x = logits, z = labels. The logistic loss is
For x < 0, to avoid overflow in exp(-x), we reformulate the above
Hence, to ensure stability and avoid overflow, the implementation uses this equivalent formulation
logits and labels must have the same type and shape.


Computes softmax cross entropy between logits and labels. (deprecated)
Future major versions of TensorFlow will allow gradients to flow into the labels input on backprop by default.
See tf.nn.softmax_cross_entropy_with_logits_v2.
Measures the probability error in discrete classification tasks in which the classes are mutually exclusive (each entry is in exactly one class). For example, each CIFAR-10 image is labeled with one and only one label: an image can be a dog or a truck, but not both.
If using exclusive labels (wherein one and only one class is true at a time), see sparse_softmax_cross_entropy_with_logits.
A common use case is to have logits and labels of shape [batch_size, num_classes], but higher dimensions are supported, with the dim argument specifying the class dimension.
Backpropagation will happen only into logits. To calculate a cross entropy loss that allows backpropagation into both logits and labels, see tf.nn.softmax_cross_entropy_with_logits_v2.
Note that to avoid confusion, it is required to pass only named arguments to this function.


Computes softmax cross entropy between logits and labels. (deprecated arguments)
Measures the probability error in discrete classification tasks in which the classes are mutually exclusive (each entry is in exactly one class). For example, each CIFAR-10 image is labeled with one and only one label: an image can be a dog or a truck, but not both.
If using exclusive labels (wherein one and only one class is true at a time), see sparse_softmax_cross_entropy_with_logits.
A common use case is to have logits and labels of shape [batch_size, num_classes], but higher dimensions are supported, with the axis argument specifying the class dimension.
logits and labels must have the same dtype (either float16, float32, or float64).
Backpropagation will happen into both logits and labels. To disallow backpropagation into labels, pass label tensors through tf.stop_gradient before feeding it to this function.
Note that to avoid confusion, it is required to pass only named arguments to this function.


Computes sparse softmax cross entropy between logits and labels.
Measures the probability error in discrete classification tasks in which the classes are mutually exclusive (each entry is in exactly one class). For example, each CIFAR-10 image is labeled with one and only one label: an image can be a dog or a truck, but not both.
A common use case is to have logits of shape [batch_size, num_classes] and have labels of shape [batch_size], but higher dimensions are supported, in which case the dim-th dimension is assumed to be of size num_classes. logits must have the dtype of float16, float32, or float64, and labels must have the dtype of int32 or int64.
Note that to avoid confusion, it is required to pass only named arguments to this function.


Creates a bidirectional recurrent neural network. (deprecated)
Similar to the unidirectional case above (rnn) but takes input and builds independent forward and backward RNNs with the final forward and backward outputs depth-concatenated, such that the output will have the format [time][batch][cell_fw.output_size + cell_bw.output_size]. The input_size of forward and backward cell must match. The initial state for both directions is zero by default (but can be set optionally) and no intermediate states are ever returned -- the network is fully unrolled for the given (passed in) length(s) of the sequence(s) or completely unrolled if length(s) is not given.


Creates a recurrent neural network specified by RNNCell cell. (deprecated)
The simplest form of RNN network generated is:
However, a few other options are available:
An initial state can be provided. If the sequence_length vector is provided, dynamic calculation is performed. This method of calculation does not compute the RNN steps past the maximum sequence length of the minibatch (thus saving computational time), and properly propagates the state at an example's sequence length to the final state output.
The dynamic calculation performed is, at time t for batch row b,


RNN that accepts a state saver for time-truncated RNN calculation. (deprecated)


Calculate the sufficient statistics for the mean and variance of x.
These sufficient statistics are computed using the one pass algorithm on an input that's optionally shifted. See: https://en.wikipedia.org/wiki/Algorithms_for_calculating_variance#Computing_shifted_data


Computes a weighted cross entropy. (deprecated arguments)
This is like sigmoid_cross_entropy_with_logits() except that pos_weight, allows one to trade off recall and precision by up- or down-weighting the cost of a positive error relative to a negative error.
The usual cross-entropy cost is defined as:
A value pos_weight > 1 decreases the false negative count, hence increasing the recall. Conversely setting pos_weight < 1 decreases the false positive count and increases the precision. This can be seen from the fact that pos_weight is introduced as a multiplicative coefficient for the positive labels term in the loss expression:
For brevity, let x = logits, z = labels, q = pos_weight. The loss is:
Setting l = (1 + (q - 1) * z), to ensure stability and avoid overflow, the implementation uses
logits and labels must have the same type and shape.


Returns the frequency-weighted mean and variance of x.


A ProtocolMessage
class AttrEntry
class ExperimentalDebugInfo


Computes the norm of vectors, matrices, and tensors. (deprecated arguments)
Compat aliases for migration
See Migration guide for more details.
tf.compat.v1.linalg.norm
This function can compute several different vector norms (the 1-norm, the Euclidean or 2-norm, the inf-norm, and in general the p-norm for p > 0) and matrix norms (Frobenius, 1-norm, 2-norm and inf-norm).
Mostly equivalent to numpy.linalg.norm. Not supported: ord <= 0, 2-norm for matrices, nuclear norm. Other differences: a) If axis is None, treats the flattened tensor as a vector regardless of rank. b) Explicitly supports 'euclidean' norm as the default, including for higher order tensors.


Use this function to prevent regularization of variables.


Creates a tensor with all elements set to 1.
See also tf.ones.
Given a single tensor (tensor), this operation returns a tensor of the same type and shape as tensor with all elements set to 1. Optionally, you can specify a new type (dtype) for the returned tensor.


DEPRECATED. Same as name_scope above, just different argument order.


Pads a tensor.
This operation pads a tensor according to the paddings you specify. paddings is an integer tensor with shape [n, 2], where n is the rank of tensor. For each dimension D of input, paddings[D, 0] indicates how many values to add before the contents of tensor in that dimension, and paddings[D, 1] indicates how many values to add after the contents of tensor in that dimension. If mode is "REFLECT" then both paddings[D, 0] and paddings[D, 1] must be no greater than tensor.dim_size(D) - 1. If mode is "SYMMETRIC" then both paddings[D, 0] and paddings[D, 1] must be no greater than tensor.dim_size(D).
The padded size of each dimension D of the output is:
paddings[D, 0] + tensor.dim_size(D) + paddings[D, 1]


Parses Example protos into a dict of tensors.
Compat aliases for migration
See Migration guide for more details.
tf.compat.v1.io.parse_example
Parses a number of serialized Example protos given in serialized. We refer to serialized as a batch with batch_size many entries of individual Example protos.
example_names may contain descriptive names for the corresponding serialized protos. These may be useful for debugging purposes, but they have no effect on the output. If not None, example_names must be the same length as serialized.
This op parses serialized examples into a dictionary mapping keys to Tensor SparseTensor, and RaggedTensor objects. features is a dict from keys to VarLenFeature, SparseFeature, RaggedFeature, and FixedLenFeature objects. Each VarLenFeature and SparseFeature is mapped to a SparseTensor; each FixedLenFeature is mapped to a Tensor; and each RaggedFeature is mapped to a RaggedTensor.
Each VarLenFeature maps to a SparseTensor of the specified type representing a ragged matrix. Its indices are [batch, index] where batch identifies the example in serialized, and index is the value's index in the list of values associated with that feature and example.
Each SparseFeature maps to a SparseTensor of the specified type representing a Tensor of dense_shape [batch_size] + SparseFeature.size. Its values come from the feature in the examples with key value_key. A values[i] comes from a position k in the feature of an example at batch entry batch. This positional information is recorded in indices[i] as [batch, index_0, index_1, ...] where index_j is the k-th value of the feature in the example at with key SparseFeature.index_key[j]. In other words, we split the indices (except the first index indicating the batch entry) of a SparseTensor by dimension into different features of the Example. Due to its complexity a VarLenFeature should be preferred over a SparseFeature whenever possible.
Each FixedLenFeature df maps to a Tensor of the specified type (or tf.float32 if not specified) and shape (serialized.size(),) + df.shape.
FixedLenFeature entries with a default_value are optional. With no default value, we will fail if that Feature is missing from any example in serialized.
Each FixedLenSequenceFeature df maps to a Tensor of the specified type (or tf.float32 if not specified) and shape (serialized.size(), None) + df.shape. All examples in serialized will be padded with default_value along the second dimension.
Each RaggedFeature maps to a RaggedTensor of the specified type. It is formed by stacking the RaggedTensor for each example, where the RaggedTensor for each individual example is constructed using the tensors specified by RaggedTensor.values_key and RaggedTensor.partition. See the tf.io.RaggedFeature documentation for details and examples.
For example, if one expects a tf.float32 VarLenFeature ft and three serialized Examples are provided:
then the output will look like:
If instead a FixedLenSequenceFeature with default_value = -1.0 and shape=[] is used then the output will look like:
Given two Example input protos in serialized:
And arguments
Then the output is a dictionary:
For dense results in two serialized Examples:
And the expected output is:
An alternative to VarLenFeature to obtain a SparseTensor is SparseFeature. For example, given two Example input protos in serialized:
And arguments
Then the output is a dictionary:
See the tf.io.RaggedFeature documentation for examples showing how RaggedFeature can be used to obtain RaggedTensors.


Parses a single Example proto.
Compat aliases for migration
See Migration guide for more details.
tf.compat.v1.io.parse_single_example
Similar to parse_example, except:
For dense tensors, the returned Tensor is identical to the output of parse_example, except there is no batch dimension, the output shape is the same as the shape given in dense_shape.
For SparseTensors, the first (batch) column of the indices matrix is removed (the indices matrix is a column vector), the values vector is unchanged, and the first (batch_size) entry of the shape vector is removed (it is now a single element vector).
One might see performance advantages by batching Example protos with parse_example instead of using this function directly.


Inserts a placeholder for a tensor that will be always fed.
Placeholders are not compatible with eager execution.


A placeholder op that passes through input when its output is not fed.


Prints a list of tensors. (deprecated)
This is an identity op (behaves like tf.identity) with the side effect of printing data when evaluating.


Public API for tf.profiler namespace.
class AdviceProto: A ProtocolMessage
class GraphNodeProto: A ProtocolMessage
class MultiGraphNodeProto: A ProtocolMessage
class OpLogProto: A ProtocolMessage
class ProfileOptionBuilder: Option Builder for Profiling API.
class Profiler: TensorFlow multi-step profiler.
advise(...): Auto profile and advise.
profile(...): Profile model.
write_op_log(...): Log provided 'op_log', and add additional model information below.


A ProtocolMessage
class Checker
class CheckersEntry


Auto profile and advise.
Builds profiles and automatically check anomalies of various aspects. For more details: https://github.com/tensorflow/tensorflow/tree/master/tensorflow/core/profiler/README.md


A ProtocolMessage
class InputShapesEntry


Profile model.
Tutorials and examples can be found in: https://github.com/tensorflow/tensorflow/blob/master/tensorflow/core/profiler/g3doc/python_api.md


Option Builder for Profiling API.
For tutorial on the options, see https://github.com/tensorflow/tensorflow/tree/master/tensorflow/core/profiler/g3doc/options.md
View source
Whether only account the statistics of displayed profiler nodes.
View source
Build a profiling option.
View source
Options used to profile float operations.
Please see https://github.com/tensorflow/tensorflow/tree/master/tensorflow/core/profiler/g3doc/profile_model_architecture.md on the caveats of calculating float operations.
View source
Order the displayed profiler nodes based on a attribute.
Supported attribute includes micros, bytes, occurrence, params, etc. https://github.com/tensorflow/tensorflow/tree/master/tensorflow/core/profiler/g3doc/options.md
View source
Select the attributes to display.
See https://github.com/tensorflow/tensorflow/tree/master/tensorflow/core/profiler/g3doc/options.md for supported attributes.
View source
Show operation time and memory consumptions.
View source
Options used to profile trainable variable parameters.
Normally used together with 'scope' view.
View source
Selectively counting statistics based on node types.
Here, 'types' means the profiler nodes' properties. Profiler by default consider device name (e.g. /job:xx/.../device:GPU:0) and operation type (e.g. MatMul) as profiler nodes' properties. User can also associate customized 'types' to profiler nodes through OpLogProto proto.
For example, user can select profiler nodes placed on gpu:0 with: account_type_regexes=['.*gpu:0.*']
If none of a node's properties match the specified regexes, the node is not displayed nor accounted.
View source
Do not generate side-effect outputs.
View source
Print the result to a file.
View source
Set the maximum depth of display.
The depth depends on profiling view. For 'scope' view, it's the depth of name scope hierarchy (tree), for 'op' view, it's the number of operation types (list), etc.
View source
Only show profiler nodes consuming no less than 'min_micros'.
View source
Only show profiler nodes consuming no less than 'min_float_ops'.
Please see https://github.com/tensorflow/tensorflow/tree/master/tensorflow/core/profiler/g3doc/profile_model_architecture.md on the caveats of calculating float operations.
View source
Only show profiler nodes consuming no less than 'min_bytes'.
View source
Only show profiler nodes including no less than 'min_occurrence' graph nodes.
A "node" means a profiler output node, which can be a python line (code view), an operation type (op view), or a graph node (graph/scope view). A python line includes all graph nodes created by that line, while an operation type includes all graph nodes of that type.
View source
Only show profiler nodes holding no less than 'min_params' parameters.
'Parameters' normally refers the weights of in TensorFlow variables. It reflects the 'capacity' of models.
View source
Regular expressions used to select profiler nodes to display.
After 'with_accounted_types' is evaluated, 'with_node_names' are evaluated as follows:
For a profile data structure, profiler first finds the profiler nodes matching 'start_name_regexes', and starts displaying profiler nodes from there. Then, if a node matches 'show_name_regexes' and doesn't match 'hide_name_regexes', it's displayed. If a node matches 'trim_name_regexes', profiler stops further searching that branch.
View source
Generate a pprof profile gzip file.
pprof -png --nodecount=100 --sample_index=1
View source
Print the result to stdout.
View source
Which profile step to use for profiling.
The 'step' here refers to the step defined by Profiler.add_step() API.
View source
Generate a timeline json file.


TensorFlow multi-step profiler.
https://github.com/tensorflow/tensorflow/tree/master/tensorflow/core/profiler/README.md
View source
Add statistics of a step.
View source
Automatically detect problems and generate reports.
View source
Profile the statistics of graph nodes, organized by dataflow graph.
View source
Profile the statistics of graph nodes, organized by name scope.
View source
Profile the statistics of the Operation types (e.g. MatMul, Conv2D).
View source
Profile the statistics of the Python codes.
By default, it shows the call stack from root. To avoid redundant output, you may use options to filter as below options['show_name_regexes'] = ['.my_code.py.']
View source
Serialize the ProfileProto to a binary string.
Users can write it to file for offline analysis by tfprof commandline or graphical interface.


Log provided 'op_log', and add additional model information below.
The API also assigns ops in tf.compat.v1.trainable_variables() an op type called '_trainable_variables'. The API also logs 'flops' statistics for ops with op.RegisterStatistics() defined. flops calculation depends on Tensor shapes defined in 'graph', which might not be complete. 'run_meta', if provided, completes the shape information with best effort.


Python functions for directly manipulating TFRecord-formatted files.
class TFRecordCompressionType: The type of compression for the record.
class TFRecordOptions: Options used for manipulating TFRecord files.
class TFRecordWriter: A class to write records to a TFRecords file.
tf_record_iterator(...): An iterator that read the records from a TFRecords file. (deprecated)


Wraps a python function and uses it as a TensorFlow op.
Given a python function func, which takes numpy arrays as its arguments and returns numpy arrays as its outputs, wrap this function as an operation in a TensorFlow graph. The following snippet constructs a simple TensorFlow graph that invokes the np.sinh() NumPy function as a operation in the graph:
The body of the function (i.e. func) will not be serialized in a GraphDef. Therefore, you should not use this function if you need to serialize your model and restore it in a different environment.
The operation must run in the same address space as the Python program that calls tf.compat.v1.py_func(). If you are using distributed TensorFlow, you must run a tf.distribute.Server in the same process as the program that calls tf.compat.v1.py_func() and you must pin the created operation to a device in that server (e.g. using with tf.device():).


Public API for tf.quantization namespace.
dequantize(...): Dequantize the 'input' tensor into a float or bfloat16 Tensor.
fake_quant_with_min_max_args(...): Fake-quantize the 'inputs' tensor, type float to 'outputs' tensor of same type.
fake_quant_with_min_max_args_gradient(...): Compute gradients for a FakeQuantWithMinMaxArgs operation.
fake_quant_with_min_max_vars(...): Fake-quantize the 'inputs' tensor of type float via global float scalars
fake_quant_with_min_max_vars_gradient(...): Compute gradients for a FakeQuantWithMinMaxVars operation.
fake_quant_with_min_max_vars_per_channel(...): Fake-quantize the 'inputs' tensor of type float via per-channel floats
fake_quant_with_min_max_vars_per_channel_gradient(...): Compute gradients for a FakeQuantWithMinMaxVarsPerChannel operation.
quantize(...): Quantize the 'input' tensor of type float to 'output' tensor of type 'T'.
quantize_and_dequantize(...): Quantizes then dequantizes a tensor. (deprecated)
quantize_and_dequantize_v2(...): Quantizes then dequantizes a tensor.
quantized_concat(...): Concatenates quantized tensors along one dimension.


Please use tf.quantization.quantize instead.


Public API for tf.queue namespace.
class FIFOQueue: A queue implementation that dequeues elements in first-in first-out order.
class PaddingFIFOQueue: A FIFOQueue that supports batching variable-sized tensors by padding.
class PriorityQueue: A queue implementation that dequeues elements in prioritized order.
class QueueBase: Base class for queue implementations.
class RandomShuffleQueue: A queue implementation that dequeues elements in a random order.


Ragged Tensors.
This package defines ops for manipulating ragged tensors (tf.RaggedTensor), which are tensors with non-uniform shapes. In particular, each RaggedTensor has one or more ragged dimensions, which are dimensions whose slices may have different lengths. For example, the inner (column) dimension of rt=[[3, 1, 4, 1], [], [5, 9, 2], [6], []] is ragged, since the column slices (rt[0, :], ..., rt[4, :]) have different lengths. For a more detailed description of ragged tensors, see the tf.RaggedTensor class documentation and the Ragged Tensor Guide.
Arguments that accept RaggedTensors are marked in bold.
class RaggedTensorValue: Represents the value of a RaggedTensor.
boolean_mask(...): Applies a boolean mask to data without flattening the mask dimensions.
constant(...): Constructs a constant RaggedTensor from a nested Python list.
constant_value(...): Constructs a RaggedTensorValue from a nested Python list.
cross(...): Generates feature cross from a list of tensors.
cross_hashed(...): Generates hashed feature cross from a list of tensors.
map_flat_values(...): Applies op to the values of one or more RaggedTensors.
placeholder(...): Creates a placeholder for a tf.RaggedTensor that will always be fed.
range(...): Returns a RaggedTensor containing the specified sequences of numbers.
row_splits_to_segment_ids(...): Generates the segmentation corresponding to a RaggedTensor row_splits.
segment_ids_to_row_splits(...): Generates the RaggedTensor row_splits corresponding to a segmentation.
stack(...): Stacks a list of rank-R tensors into one rank-(R+1) RaggedTensor.
stack_dynamic_partitions(...): Stacks dynamic partitions of a Tensor or RaggedTensor.


Constructs a RaggedTensorValue from a nested Python list.
All scalar values in pylist must have the same nesting depth K, and the returned RaggedTensorValue will have rank K. If pylist contains no scalar values, then K is one greater than the maximum depth of empty lists in pylist. All scalar values in pylist must be compatible with dtype.


Creates a placeholder for a tf.RaggedTensor that will always be fed.
@compatibility{eager} Placeholders are not compatible with eager execution.


Represents the value of a RaggedTensor.
See tf.RaggedTensor for a description of ragged tensors.
View source
Returns this ragged tensor value as a nested Python list.


Public API for tf.random namespace.
experimental module: Public API for tf.random.experimental namespace.
class Algorithm: An enumeration.
class Generator: Random-number generator.
all_candidate_sampler(...): Generate the set of all classes.
categorical(...): Draws samples from a categorical distribution.
create_rng_state(...): Creates a RNG state from an integer or a vector.
fixed_unigram_candidate_sampler(...): Samples a set of classes using the provided (fixed) base distribution.
gamma(...): Draws shape samples from each of the given Gamma distribution(s).
get_global_generator(...): Retrieves the global generator.
get_seed(...): Returns the local seeds an operation should use given an op-specific seed.
learned_unigram_candidate_sampler(...): Samples a set of classes from a distribution learned during training.
log_uniform_candidate_sampler(...): Samples a set of classes using a log-uniform (Zipfian) base distribution.
multinomial(...): Draws samples from a multinomial distribution. (deprecated)
normal(...): Outputs random values from a normal distribution.
poisson(...): Draws shape samples from each of the given Poisson distribution(s).
set_global_generator(...): Replaces the global generator with another Generator object.
set_random_seed(...): Sets the graph-level random seed for the default graph.
shuffle(...): Randomly shuffles a tensor along its first dimension.
stateless_binomial(...): Outputs deterministic pseudorandom values from a binomial distribution.
stateless_categorical(...): Draws deterministic pseudorandom samples from a categorical distribution.
stateless_gamma(...): Outputs deterministic pseudorandom values from a gamma distribution.
stateless_multinomial(...): Draws deterministic pseudorandom samples from a multinomial distribution. (deprecated)
stateless_normal(...): Outputs deterministic pseudorandom values from a normal distribution.
stateless_parameterized_truncated_normal(...): Outputs random values from a truncated normal distribution.
stateless_poisson(...): Outputs deterministic pseudorandom values from a Poisson distribution.
stateless_truncated_normal(...): Outputs deterministic pseudorandom values, truncated normally distributed.
stateless_uniform(...): Outputs deterministic pseudorandom values from a uniform distribution.
truncated_normal(...): Outputs random values from a truncated normal distribution.
uniform(...): Outputs random values from a uniform distribution.
uniform_candidate_sampler(...): Samples a set of classes using a uniform base distribution.


Public API for tf.random.experimental namespace.
class Algorithm: An enumeration.
class Generator: Random-number generator.
create_rng_state(...): Creates a RNG state from an integer or a vector.
get_global_generator(...): Retrieves the global generator.
set_global_generator(...): Replaces the global generator with another Generator object.
stateless_fold_in(...): Folds in data to an RNG seed to form a new RNG seed.
stateless_split(...): Splits an RNG seed into num new seeds by adding a leading axis.


Draws deterministic pseudorandom samples from a multinomial distribution. (deprecated)
This is a stateless version of tf.random.categorical: if run twice with the same seeds and shapes, it will produce the same pseudorandom numbers. The output is consistent across multiple runs on the same hardware (and between CPU and GPU), but may change between versions of TensorFlow or on non-CPU/GPU hardware.


Initializer that generates tensors with a normal distribution.
Compat aliases for migration
See Migration guide for more details.
tf.compat.v1.initializers.random_normal
View source
Instantiates an initializer from a configuration dictionary.
View source
Returns the configuration of the initializer as a JSON-serializable dict.
View source
Returns a tensor object initialized as specified by the initializer.


Draws shape samples from each of the given Poisson distribution(s).
Compat aliases for migration
See Migration guide for more details.
tf.compat.v1.random.poisson
lam is the rate parameter describing the distribution(s).


Initializer that generates tensors with a uniform distribution.
Compat aliases for migration
See Migration guide for more details.
tf.compat.v1.initializers.random_uniform
View source
Instantiates an initializer from a configuration dictionary.
View source
Returns the configuration of the initializer as a JSON-serializable dict.
View source
Returns a tensor object initialized as specified by the initializer.