tf.nn.pool     View source on GitHub    Performs an N-D pooling operation.
tf.nn.pool(
    input, window_shape, pooling_type, strides=None, padding='VALID',
    data_format=None, dilations=None, name=None
)
 In the case that data_format does not start with "NC", computes for 0 <= b < batch_size, 0 <= x[i] < output_spatial_shape[i], 0 <= c < num_channels: output[b, x[0], ..., x[N-1], c] =
  REDUCE_{z[0], ..., z[N-1]}
    input[b,
          x[0] * strides[0] - pad_before[0] + dilation_rate[0]*z[0],
          ...
          x[N-1]*strides[N-1] - pad_before[N-1] + dilation_rate[N-1]*z[N-1],
          c],
 where the reduction function REDUCE depends on the value of pooling_type, and pad_before is defined based on the value of padding as described in the "returns" section of tf.nn.convolution for details. The reduction never includes out-of-bounds positions. In the case that data_format starts with "NC", the input and output are simply transposed as follows: pool(input, data_format, **kwargs) =
  tf.transpose(pool(tf.transpose(input, [0] + range(2,N+2) + [1]),
                    **kwargs),
               [0, N+1] + range(1, N+1))

 Args
  input   Tensor of rank N+2, of shape [batch_size] + input_spatial_shape + [num_channels] if data_format does not start with "NC" (default), or [batch_size, num_channels] + input_spatial_shape if data_format starts with "NC". Pooling happens over the spatial dimensions only.
  window_shape   Sequence of N ints >= 1.
  pooling_type   Specifies pooling operation, must be "AVG" or "MAX".
  strides   Optional. Sequence of N ints >= 1. Defaults to [1]N. If any value of strides is > 1, then all values of dilation_rate must be 1.

  padding   The padding algorithm, must be "SAME" or "VALID". Defaults to "SAME". See the "returns" section of tf.nn.convolution for details.
  data_format   A string or None. Specifies whether the channel dimension of the input and output is the last dimension (default, or if data_format does not start with "NC"), or the second dimension (if data_format starts with "NC"). For N=1, the valid values are "NWC" (default) and "NCW". For N=2, the valid values are "NHWC" (default) and "NCHW". For N=3, the valid values are "NDHWC" (default) and "NCDHW".
  dilations   Optional. Dilation rate. List of N ints >= 1. Defaults to [1]N. If any value of dilation_rate is > 1, then all values of strides must be 1.
  name   Optional. Name of the op.

 Returns   Tensor of rank N+2, of shape [batch_size] + output_spatial_shape + [num_channels] if data_format is None or does not start with "NC", or [batch_size, num_channels] + output_spatial_shape if data_format starts with "NC", where output_spatial_shape depends on the value of padding: If padding = "SAME": output_spatial_shape[i] = ceil(input_spatial_shape[i] / strides[i]) If padding = "VALID": output_spatial_shape[i] = ceil((input_spatial_shape[i] - (window_shape[i] - 1) * dilation_rate[i]) / strides[i]).

 Raises
  ValueError   if arguments are invalid.


tf.nn.relu Computes rectified linear: max(features, 0).  View aliases  Compat aliases for migration
See Migration guide for more details. tf.compat.v1.nn.relu
tf.nn.relu(
    features, name=None
)
 See: https://en.wikipedia.org/wiki/Rectifier_(neural_networks) Example usage: >>> tf.nn.relu([-2., 0., -0., 3.]).numpy()
array([ 0.,  0., -0.,  3.], dtype=float32)

 Args
  features   A Tensor. Must be one of the following types: float32, float64, int32, uint8, int16, int8, int64, bfloat16, uint16, half, uint32, uint64, qint8.
  name   A name for the operation (optional).

 Returns   A Tensor. Has the same type as features.


tf.nn.relu6     View source on GitHub    Computes Rectified Linear 6: min(max(features, 0), 6).  View aliases  Compat aliases for migration
See Migration guide for more details. tf.compat.v1.nn.relu6
tf.nn.relu6(
    features, name=None
)

 Args
  features   A Tensor with type float, double, int32, int64, uint8, int16, or int8.
  name   A name for the operation (optional).

 Returns   A Tensor with the same type as features.
 References: Convolutional Deep Belief Networks on CIFAR-10: Krizhevsky et al., 2010 (pdf)


tf.nn.RNNCellDeviceWrapper Operator that ensures an RNNCell runs on a particular device. Inherits From: AbstractRNNCell, Layer, Module
tf.nn.RNNCellDeviceWrapper(
    *args, **kwargs
)

 Args
  cell   An instance of RNNCell.
  device   A device string or function, for passing to tf.device.
  **kwargs   dict of keyword arguments for base layer.

 Attributes
  output_size

  state_size
   Methods get_initial_state View source
get_initial_state(
    inputs=None, batch_size=None, dtype=None
)
 zero_state View source
zero_state(
    batch_size, dtype
)


tf.nn.RNNCellDropoutWrapper Operator adding dropout to inputs and outputs of the given cell. Inherits From: AbstractRNNCell, Layer, Module
tf.nn.RNNCellDropoutWrapper(
    *args, **kwargs
)

 Args
  cell   an RNNCell, a projection to output_size is added to it.
  input_keep_prob   unit Tensor or float between 0 and 1, input keep probability; if it is constant and 1, no input dropout will be added.
  output_keep_prob   unit Tensor or float between 0 and 1, output keep probability; if it is constant and 1, no output dropout will be added.
  state_keep_prob   unit Tensor or float between 0 and 1, output keep probability; if it is constant and 1, no output dropout will be added. State dropout is performed on the outgoing states of the cell. Note the state components to which dropout is applied when state_keep_prob is in (0, 1) are also determined by the argument dropout_state_filter_visitor (e.g. by default dropout is never applied to the c component of an LSTMStateTuple).
  variational_recurrent   Python bool. If True, then the same dropout pattern is applied across all time steps per run call. If this parameter is set, input_size must be provided.
  input_size   (optional) (possibly nested tuple of) TensorShape objects containing the depth(s) of the input tensors expected to be passed in to the DropoutWrapper. Required and used iff variational_recurrent = True and input_keep_prob < 1.
  dtype   (optional) The dtype of the input, state, and output tensors. Required and used iff variational_recurrent = True.
  seed   (optional) integer, the randomness seed.
  dropout_state_filter_visitor   (optional), default: (see below). Function that takes any hierarchical level of the state and returns a scalar or depth=1 structure of Python booleans describing which terms in the state should be dropped out. In addition, if the function returns True, dropout is applied across this sublevel. If the function returns False, dropout is not applied across this entire sublevel. Default behavior: perform dropout on all terms except the memory (c) state of LSTMCellState objects, and don't try to apply dropout to TensorArray objects: def dropout_state_filter_visitor(s): if isinstance(s, LSTMCellState): # Never perform dropout on the c state. return LSTMCellState(c=False, h=True) elif isinstance(s, TensorArray): return False return True
  **kwargs   dict of keyword arguments for base layer.

 Raises
  TypeError   if cell is not an RNNCell, or keep_state_fn is provided but not callable.
  ValueError   if any of the keep_probs are not between 0 and 1.

 Attributes
  output_size

  state_size

  wrapped_cell
   Methods get_initial_state View source
get_initial_state(
    inputs=None, batch_size=None, dtype=None
)
 zero_state View source
zero_state(
    batch_size, dtype
)


tf.nn.RNNCellResidualWrapper RNNCell wrapper that ensures cell inputs are added to the outputs. Inherits From: AbstractRNNCell, Layer, Module
tf.nn.RNNCellResidualWrapper(
    *args, **kwargs
)

 Args
  cell   An instance of RNNCell.
  residual_fn   (Optional) The function to map raw cell inputs and raw cell outputs to the actual cell outputs of the residual network. Defaults to calling nest.map_structure on (lambda i, o: i + o), inputs and outputs.
  **kwargs   dict of keyword arguments for base layer.

 Attributes
  output_size

  state_size
   Methods get_initial_state View source
get_initial_state(
    inputs=None, batch_size=None, dtype=None
)
 zero_state View source
zero_state(
    batch_size, dtype
)


tf.nn.safe_embedding_lookup_sparse     View source on GitHub    Lookup embedding results, accounting for invalid IDs and empty features.
tf.nn.safe_embedding_lookup_sparse(
    embedding_weights, sparse_ids, sparse_weights=None, combiner='mean',
    default_id=None, max_norm=None, name=None
)
 The partitioned embedding in embedding_weights must all be the same shape except for the first dimension. The first dimension is allowed to vary as the vocabulary size is not necessarily a multiple of num of shards. Invalid IDs (< 0) are pruned from input IDs and weights, as well as any IDs with non-positive weight. For an entry with no features, the embedding vector for default_id is returned, or the 0-vector if default_id is not supplied. The ids and weights may be multi-dimensional. Embeddings are always aggregated along the last dimension. If len(embedding_weights) > 1, each element id of ids is partitioned between the elements of embedding_weights according to the "div" partition strategy, which means we assign ids to partitions in a contiguous manner. For instance, 13 ids are split across 5 partitions as: [[0, 1, 2], [3, 4, 5], [6, 7, 8], [9, 10], [11, 12]]. If the id space does not evenly divide the number of partitions, each of the first (max_id + 1) % len(embedding_weights) partitions will be assigned one more id.

 Args
  embedding_weights   A single tensor representing the complete embedding tensor, or a list of tensors all of same shape except for the first dimension, representing sharded embedding tensors following "div" partition strategy.
  sparse_ids   SparseTensor of shape [d_0, d_1, ..., d_n] containing the ids. d_0 is typically batch size.
  sparse_weights   SparseTensor of same shape as sparse_ids, containing float weights corresponding to sparse_ids, or None if all weights are be assumed to be 1.0.
  combiner   A string specifying how to combine embedding results for each entry. Currently "mean", "sqrtn" and "sum" are supported, with "mean" the default.
  default_id   The id to use for an entry with no features. Defaults to 0-vector.
  max_norm   If not None, all embeddings are l2-normalized to max_norm before combining.
  name   A name for this operation (optional).

 Returns   A dense tensor representing the combined embeddings for the sparse ids. For each row in the dense tensor represented by sparse_ids, the op looks up the embeddings for all ids in that row, multiplies them by the corresponding weight, and combines these embeddings as specified. In other words, if shape(combined embedding_weights) = [p0, p1, ..., pm] and shape(sparse_ids) = shape(sparse_weights) = [d0, d1, ..., dn] then shape(output) = [d0, d1, ... dn-1, p1, ..., pm]. For instance, if params is a 10x20 matrix, and sp_ids / sp_weights are [0, 0]: id 1, weight 2.0
[0, 1]: id 3, weight 0.5
[1, 0]: id -1, weight 1.0
[2, 3]: id 1, weight 3.0
 default_id is 0. with combiner="mean", then the output will be a 3x20 matrix where output[0, :] = (params[1, :] * 2.0 + params[3, :] * 0.5) / (2.0 + 0.5)
output[1, :] = (params[0, :] * 1.0) / 1.0
output[2, :] = (params[1, :] * 3.0) / 3.0

 Raises
  ValueError   if embedding_weights is empty.


tf.nn.sampled_softmax_loss     View source on GitHub    Computes and returns the sampled softmax training loss.
tf.nn.sampled_softmax_loss(
    weights, biases, labels, inputs, num_sampled, num_classes, num_true=1,
    sampled_values=None, remove_accidental_hits=True, seed=None,
    name='sampled_softmax_loss'
)
 This is a faster way to train a softmax classifier over a huge number of classes. This operation is for training only. It is generally an underestimate of the full softmax loss. A common use case is to use this method for training, and calculate the full sigmoid loss for evaluation or inference as in the following example: if mode == "train":
  loss = tf.nn.sampled_softmax_loss(
      weights=weights,
      biases=biases,
      labels=labels,
      inputs=inputs,
      ...)
elif mode == "eval":
  logits = tf.matmul(inputs, tf.transpose(weights))
  logits = tf.nn.bias_add(logits, biases)
  labels_one_hot = tf.one_hot(labels, n_classes)
  loss = tf.nn.softmax_cross_entropy_with_logits(
      labels=labels_one_hot,
      logits=logits)
 See our Candidate Sampling Algorithms Reference Also see Section 3 of Jean et al., 2014 (pdf) for the math.
Note: when doing embedding lookup on weights and bias, "div" partition strategy will be used. Support for other partition strategy will be added later.

 Args
  weights   A Tensor of shape [num_classes, dim], or a list of Tensor objects whose concatenation along dimension 0 has shape [num_classes, dim]. The (possibly-sharded) class embeddings.
  biases   A Tensor of shape [num_classes]. The class biases.
  labels   A Tensor of type int64 and shape [batch_size, num_true]. The target classes. Note that this format differs from the labels argument of nn.softmax_cross_entropy_with_logits.
  inputs   A Tensor of shape [batch_size, dim]. The forward activations of the input network.
  num_sampled   An int. The number of classes to randomly sample per batch.
  num_classes   An int. The number of possible classes.
  num_true   An int. The number of target classes per training example.
  sampled_values   a tuple of (sampled_candidates, true_expected_count, sampled_expected_count) returned by a *_candidate_sampler function. (if None, we default to log_uniform_candidate_sampler)
  remove_accidental_hits   A bool. whether to remove "accidental hits" where a sampled class equals one of the target classes. Default is True.
  seed   random seed for candidate sampling. Default to None, which doesn't set the op-level random seed for candidate sampling.
  name   A name for the operation (optional).

 Returns   A batch_size 1-D tensor of per-example sampled softmax losses.


tf.nn.scale_regularization_loss     View source on GitHub    Scales the sum of the given regularization losses by number of replicas.  View aliases  Compat aliases for migration
See Migration guide for more details. tf.compat.v1.nn.scale_regularization_loss
tf.nn.scale_regularization_loss(
    regularization_loss
)
 Usage with distribution strategy and custom training loop: with strategy.scope():
  def compute_loss(self, label, predictions):
    per_example_loss = tf.keras.losses.sparse_categorical_crossentropy(
        labels, predictions)

    # Compute loss that is scaled by sample_weight and by global batch size.
    loss = tf.nn.compute_average_loss(
        per_example_loss,
        sample_weight=sample_weight,
        global_batch_size=GLOBAL_BATCH_SIZE)

    # Add scaled regularization losses.
    loss += tf.nn.scale_regularization_loss(tf.nn.l2_loss(weights))
    return loss

 Args
  regularization_loss   Regularization loss.

 Returns   Scalar loss value.


tf.nn.selu Computes scaled exponential linear: scale * alpha * (exp(features) - 1)  View aliases  Compat aliases for migration
See Migration guide for more details. tf.compat.v1.nn.selu
tf.nn.selu(
    features, name=None
)
 if < 0, scale * features otherwise. To be used together with initializer = tf.variance_scaling_initializer(factor=1.0, mode='FAN_IN'). For correct dropout, use tf.contrib.nn.alpha_dropout. See Self-Normalizing Neural Networks

 Args
  features   A Tensor. Must be one of the following types: half, bfloat16, float32, float64.
  name   A name for the operation (optional).

 Returns   A Tensor. Has the same type as features.


tf.nn.separable_conv2d     View source on GitHub    2-D convolution with separable filters.
tf.nn.separable_conv2d(
    input, depthwise_filter, pointwise_filter, strides, padding, data_format=None,
    dilations=None, name=None
)
 Performs a depthwise convolution that acts separately on channels followed by a pointwise convolution that mixes channels. Note that this is separability between dimensions [1, 2] and 3, not spatial separability between dimensions 1 and 2. In detail, with the default NHWC format, output[b, i, j, k] = sum_{di, dj, q, r}
    input[b, strides[1] * i + di, strides[2] * j + dj, q] *
    depthwise_filter[di, dj, q, r] *
    pointwise_filter[0, 0, q * channel_multiplier + r, k]
 strides controls the strides for the depthwise convolution only, since the pointwise convolution has implicit strides of [1, 1, 1, 1]. Must have strides[0] = strides[3] = 1. For the most common case of the same horizontal and vertical strides, strides = [1, stride, stride, 1]. If any value in rate is greater than 1, we perform atrous depthwise convolution, in which case all values in the strides tensor must be equal to 1.

 Args
  input   4-D Tensor with shape according to data_format.
  depthwise_filter   4-D Tensor with shape [filter_height, filter_width, in_channels, channel_multiplier]. Contains in_channels convolutional filters of depth 1.
  pointwise_filter   4-D Tensor with shape [1, 1, channel_multiplier * in_channels, out_channels]. Pointwise filter to mix channels after depthwise_filter has convolved spatially.
  strides   1-D of size 4. The strides for the depthwise convolution for each dimension of input.
  padding   Controls how to pad the image before applying the depthwise convolution. Can be the string "SAME" or "VALID" indicating the type of padding algorithm to use, or a Python list indicating the explicit paddings at the start and end of each dimension. When explicit padding is used and data_format is "NHWC", this should be in the form [[0, 0], [pad_top, pad_bottom], [pad_left, pad_right], [0, 0]]. When explicit padding used and data_format is "NCHW", this should be in the form [[0, 0], [0, 0], [pad_top, pad_bottom], [pad_left, pad_right]].
  data_format   The data format for input. Either "NHWC" (default) or "NCHW".
  dilations   1-D of size 2. The dilation rate in which we sample input values across the height and width dimensions in atrous convolution. If it is greater than 1, then all values of strides must be 1.
  name   A name for this operation (optional).

 Returns   A 4-D Tensor with shape according to 'data_format'. For example, with data_format="NHWC", shape is [batch, out_height, out_width, out_channels].


tf.nn.sigmoid_cross_entropy_with_logits     View source on GitHub    Computes sigmoid cross entropy given logits.
tf.nn.sigmoid_cross_entropy_with_logits(
    labels=None, logits=None, name=None
)
 Measures the probability error in discrete classification tasks in which each class is independent and not mutually exclusive. For instance, one could perform multilabel classification where a picture can contain both an elephant and a dog at the same time. For brevity, let x = logits, z = labels. The logistic loss is   z * -log(sigmoid(x)) + (1 - z) * -log(1 - sigmoid(x))
= z * -log(1 / (1 + exp(-x))) + (1 - z) * -log(exp(-x) / (1 + exp(-x)))
= z * log(1 + exp(-x)) + (1 - z) * (-log(exp(-x)) + log(1 + exp(-x)))
= z * log(1 + exp(-x)) + (1 - z) * (x + log(1 + exp(-x))
= (1 - z) * x + log(1 + exp(-x))
= x - x * z + log(1 + exp(-x))
 For x < 0, to avoid overflow in exp(-x), we reformulate the above   x - x * z + log(1 + exp(-x))
= log(exp(x)) - x * z + log(1 + exp(-x))
= - x * z + log(1 + exp(x))
 Hence, to ensure stability and avoid overflow, the implementation uses this equivalent formulation max(x, 0) - x * z + log(1 + exp(-abs(x)))
 logits and labels must have the same type and shape.

 Args
  labels   A Tensor of the same type and shape as logits.
  logits   A Tensor of type float32 or float64.
  name   A name for the operation (optional).

 Returns   A Tensor of the same shape as logits with the componentwise logistic losses.

 Raises
  ValueError   If logits and labels do not have the same shape.


tf.nn.silu Computes the SiLU or Swish activation function: x * sigmoid(x).  View aliases  Main aliases
tf.nn.swish Compat aliases for migration See Migration guide for more details. tf.compat.v1.nn.silu, tf.compat.v1.nn.swish
tf.nn.silu(
    features
)
 The SiLU activation function was introduced in "Gaussian Error Linear Units (GELUs)" Hendrycks et al. 2016 and "Sigmoid-Weighted Linear Units for Neural Network Function Approximation in Reinforcement Learning" Elfwing et al. 2017 and was independently discovered (and called swish) in "Searching for Activation Functions" Ramachandran et al. 2017

 Args
  features   A Tensor representing preactivation values.
  name   A name for the operation (optional).

 Returns   The activation value.


tf.nn.softmax     View source on GitHub    Computes softmax activations.  View aliases  Main aliases
tf.math.softmax
tf.nn.softmax(
    logits, axis=None, name=None
)
 This function performs the equivalent of softmax = tf.exp(logits) / tf.reduce_sum(tf.exp(logits), axis)

 Args
  logits   A non-empty Tensor. Must be one of the following types: half, float32, float64.
  axis   The dimension softmax would be performed on. The default is -1 which indicates the last dimension.
  name   A name for the operation (optional).

 Returns   A Tensor. Has the same type and shape as logits.

 Raises
  InvalidArgumentError   if logits is empty or axis is beyond the last dimension of logits.


tf.nn.softmax_cross_entropy_with_logits     View source on GitHub    Computes softmax cross entropy between logits and labels.
tf.nn.softmax_cross_entropy_with_logits(
    labels, logits, axis=-1, name=None
)
 Measures the probability error in discrete classification tasks in which the classes are mutually exclusive (each entry is in exactly one class). For example, each CIFAR-10 image is labeled with one and only one label: an image can be a dog or a truck, but not both.
Note: While the classes are mutually exclusive, their probabilities need not be. All that is required is that each row of labels is a valid probability distribution. If they are not, the computation of the gradient will be incorrect.
 If using exclusive labels (wherein one and only one class is true at a time), see sparse_softmax_cross_entropy_with_logits. Usage:
logits = [[4.0, 2.0, 1.0], [0.0, 5.0, 1.0]]
labels = [[1.0, 0.0, 0.0], [0.0, 0.8, 0.2]]
tf.nn.softmax_cross_entropy_with_logits(labels=labels, logits=logits)
<tf.Tensor: shape=(2,), dtype=float32,
numpy=array([0.16984604, 0.82474494], dtype=float32)>
 Warning: This op expects unscaled logits, since it performs a softmax on logits internally for efficiency. Do not call this op with the output of softmax, as it will produce incorrect results. A common use case is to have logits and labels of shape [batch_size, num_classes], but higher dimensions are supported, with the axis argument specifying the class dimension. logits and labels must have the same dtype (either float16, float32, or float64). Backpropagation will happen into both logits and labels. To disallow backpropagation into labels, pass label tensors through tf.stop_gradient before feeding it to this function. Note that to avoid confusion, it is required to pass only named arguments to this function.

 Args
  labels   Each vector along the class dimension should hold a valid probability distribution e.g. for the case in which labels are of shape [batch_size, num_classes], each row of labels[i] must be a valid probability distribution.
  logits   Per-label activations, typically a linear output. These activation energies are interpreted as unnormalized log probabilities.
  axis   The class dimension. Defaulted to -1 which is the last dimension.
  name   A name for the operation (optional).

 Returns   A Tensor that contains the softmax cross entropy loss. Its type is the same as logits and its shape is the same as labels except that it does not have the last dimension of labels.


tf.nn.softsign Computes softsign: features / (abs(features) + 1).  View aliases  Main aliases
tf.math.softsign Compat aliases for migration See Migration guide for more details. tf.compat.v1.math.softsign, tf.compat.v1.nn.softsign
tf.nn.softsign(
    features, name=None
)

 Args
  features   A Tensor. Must be one of the following types: half, bfloat16, float32, float64.
  name   A name for the operation (optional).

 Returns   A Tensor. Has the same type as features.


tf.nn.space_to_depth     View source on GitHub    SpaceToDepth for tensors of type T.
tf.nn.space_to_depth(
    input, block_size, data_format='NHWC', name=None
)
 Rearranges blocks of spatial data, into depth. More specifically, this op outputs a copy of the input tensor where values from the height and width dimensions are moved to the depth dimension. The attr block_size indicates the input block size.  Non-overlapping blocks of size block_size x block size are rearranged into depth at each location. The depth of the output tensor is block_size * block_size * input_depth. The Y, X coordinates within each block of the input become the high order component of the output channel index. The input tensor's height and width must be divisible by block_size.  The data_format attr specifies the layout of the input and output tensors with the following options: "NHWC": [ batch, height, width, channels ] "NCHW": [ batch, channels, height, width ] "NCHW_VECT_C": qint8 [ batch, channels / 4, height, width, 4 ] It is useful to consider the operation as transforming a 6-D Tensor. e.g. for data_format = NHWC, Each element in the input tensor can be specified via 6 coordinates, ordered by decreasing memory layout significance as: n,oY,bY,oX,bX,iC (where n=batch index, oX, oY means X or Y coordinates within the output image, bX, bY means coordinates within the input block, iC means input channels). The output would be a transpose to the following layout: n,oY,oX,bY,bX,iC This operation is useful for resizing the activations between convolutions (but keeping all data), e.g. instead of pooling. It is also useful for training purely convolutional models. For example, given an input of shape [1, 2, 2, 1], data_format = "NHWC" and block_size = 2: x = [[[[1], [2]],
      [[3], [4]]]]
 This operation will output a tensor of shape [1, 1, 1, 4]: [[[[1, 2, 3, 4]]]]
 Here, the input has a batch of 1 and each batch element has shape [2, 2, 1], the corresponding output will have a single element (i.e. width and height are both 1) and will have a depth of 4 channels (1 * block_size * block_size). The output element shape is [1, 1, 4]. For an input tensor with larger depth, here of shape [1, 2, 2, 3], e.g. x = [[[[1, 2, 3], [4, 5, 6]],
      [[7, 8, 9], [10, 11, 12]]]]
 This operation, for block_size of 2, will return the following tensor of shape [1, 1, 1, 12] [[[[1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12]]]]
 Similarly, for the following input of shape [1 4 4 1], and a block size of 2: x = [[[[1],   [2],  [5],  [6]],
      [[3],   [4],  [7],  [8]],
      [[9],  [10], [13],  [14]],
      [[11], [12], [15],  [16]]]]
 the operator will return the following tensor of shape [1 2 2 4]: x = [[[[1, 2, 3, 4],
       [5, 6, 7, 8]],
      [[9, 10, 11, 12],
       [13, 14, 15, 16]]]]

 Args
  input   A Tensor.
  block_size   An int that is >= 2. The size of the spatial block.
  data_format   An optional string from: "NHWC", "NCHW", "NCHW_VECT_C". Defaults to "NHWC".
  name   A name for the operation (optional).

 Returns   A Tensor. Has the same type as input.


tf.nn.sparse_softmax_cross_entropy_with_logits     View source on GitHub    Computes sparse softmax cross entropy between logits and labels.
tf.nn.sparse_softmax_cross_entropy_with_logits(
    labels, logits, name=None
)
 Measures the probability error in discrete classification tasks in which the classes are mutually exclusive (each entry is in exactly one class). For example, each CIFAR-10 image is labeled with one and only one label: an image can be a dog or a truck, but not both.
Note: For this operation, the probability of a given label is considered exclusive. That is, soft classes are not allowed, and the labels vector must provide a single specific index for the true class for each row of logits (each minibatch entry). For soft softmax classification with a probability distribution for each entry, see softmax_cross_entropy_with_logits_v2.

Warning: This op expects unscaled logits, since it performs a softmax on logits internally for efficiency. Do not call this op with the output of softmax, as it will produce incorrect results. A common use case is to have logits of shape [batch_size, num_classes] and have labels of shape [batch_size], but higher dimensions are supported, in which case the dim-th dimension is assumed to be of size num_classes. logits must have the dtype of float16, float32, or float64, and labels must have the dtype of int32 or int64. Note that to avoid confusion, it is required to pass only named arguments to this function.

 Args
  labels   Tensor of shape [d_0, d_1, ..., d_{r-1}] (where r is rank of labels and result) and dtype int32 or int64. Each entry in labels must be an index in [0, num_classes). Other values will raise an exception when this op is run on CPU, and return NaN for corresponding loss and gradient rows on GPU.
  logits   Unscaled log probabilities of shape [d_0, d_1, ..., d_{r-1}, num_classes] and dtype float16, float32, or float64.
  name   A name for the operation (optional).

 Returns   A Tensor of the same shape as labels and of the same type as logits with the softmax cross entropy loss.

 Raises
  ValueError   If logits are scalars (need to have rank >= 1) or if the rank of the labels is not equal to the rank of the logits minus one.


tf.nn.sufficient_statistics     View source on GitHub    Calculate the sufficient statistics for the mean and variance of x.
tf.nn.sufficient_statistics(
    x, axes, shift=None, keepdims=False, name=None
)
 These sufficient statistics are computed using the one pass algorithm on an input that's optionally shifted. See: https://en.wikipedia.org/wiki/Algorithms_for_calculating_variance#Computing_shifted_data

 Args
  x   A Tensor.
  axes   Array of ints. Axes along which to compute mean and variance.
  shift   A Tensor containing the value by which to shift the data for numerical stability, or None if no shift is to be performed. A shift close to the true mean provides the most numerically stable results.
  keepdims   produce statistics with the same dimensionality as the input.
  name   Name used to scope the operations that compute the sufficient stats.

 Returns   Four Tensor objects of the same type as x:  the count (number of elements to average over). the (possibly shifted) sum of the elements in the array. the (possibly shifted) sum of squares of the elements in the array. the shift by which the mean must be corrected or None if shift is None.


tf.nn.weighted_cross_entropy_with_logits     View source on GitHub    Computes a weighted cross entropy.
tf.nn.weighted_cross_entropy_with_logits(
    labels, logits, pos_weight, name=None
)
 This is like sigmoid_cross_entropy_with_logits() except that pos_weight, allows one to trade off recall and precision by up- or down-weighting the cost of a positive error relative to a negative error. The usual cross-entropy cost is defined as: labels * -log(sigmoid(logits)) +
    (1 - labels) * -log(1 - sigmoid(logits))
 A value pos_weight > 1 decreases the false negative count, hence increasing the recall. Conversely setting pos_weight < 1 decreases the false positive count and increases the precision. This can be seen from the fact that pos_weight is introduced as a multiplicative coefficient for the positive labels term in the loss expression: labels * -log(sigmoid(logits)) * pos_weight +
    (1 - labels) * -log(1 - sigmoid(logits))
 For brevity, let x = logits, z = labels, q = pos_weight. The loss is:   qz * -log(sigmoid(x)) + (1 - z) * -log(1 - sigmoid(x))
= qz * -log(1 / (1 + exp(-x))) + (1 - z) * -log(exp(-x) / (1 + exp(-x)))
= qz * log(1 + exp(-x)) + (1 - z) * (-log(exp(-x)) + log(1 + exp(-x)))
= qz * log(1 + exp(-x)) + (1 - z) * (x + log(1 + exp(-x))
= (1 - z) * x + (qz +  1 - z) * log(1 + exp(-x))
= (1 - z) * x + (1 + (q - 1) * z) * log(1 + exp(-x))
 Setting l = (1 + (q - 1) * z), to ensure stability and avoid overflow, the implementation uses (1 - z) * x + l * (log(1 + exp(-abs(x))) + max(-x, 0))
 logits and labels must have the same type and shape.

 Args
  labels   A Tensor of the same type and shape as logits.
  logits   A Tensor of type float32 or float64.
  pos_weight   A coefficient to use on the positive examples.
  name   A name for the operation (optional).

 Returns   A Tensor of the same shape as logits with the componentwise weighted logistic losses.

 Raises
  ValueError   If logits and labels do not have the same shape.


tf.nn.weighted_moments     View source on GitHub    Returns the frequency-weighted mean and variance of x.
tf.nn.weighted_moments(
    x, axes, frequency_weights, keepdims=False, name=None
)

 Args
  x   A tensor.
  axes   1-d tensor of int32 values; these are the axes along which to compute mean and variance.
  frequency_weights   A tensor of positive weights which can be broadcast with x.
  keepdims   Produce moments with the same dimensionality as the input.
  name   Name used to scope the operation.

 Returns   Two tensors: weighted_mean and weighted_variance.


tf.nn.with_space_to_batch     View source on GitHub    Performs op on the space-to-batch representation of input.  View aliases  Compat aliases for migration
See Migration guide for more details. tf.compat.v1.nn.with_space_to_batch
tf.nn.with_space_to_batch(
    input, dilation_rate, padding, op, filter_shape=None, spatial_dims=None,
    data_format=None
)
 This has the effect of transforming sliding window operations into the corresponding "atrous" operation in which the input is sampled at the specified dilation_rate. In the special case that dilation_rate is uniformly 1, this simply returns: op(input, num_spatial_dims, padding) Otherwise, it returns: batch_to_space_nd( op(space_to_batch_nd(input, adjusted_dilation_rate, adjusted_paddings), num_spatial_dims, "VALID") adjusted_dilation_rate, adjusted_crops), where: adjusted_dilation_rate is an int64 tensor of shape [max(spatialdims)], adjusted{paddings,crops} are int64 tensors of shape [max(spatial_dims), 2] defined as follows: We first define two int64 tensors paddings and crops of shape [num_spatial_dims, 2] based on the value of padding and the spatial dimensions of the input: If padding = "VALID", then: paddings, crops = required_space_to_batch_paddings( input_shape[spatial_dims], dilation_rate) If padding = "SAME", then: dilated_filter_shape = filter_shape + (filter_shape - 1) * (dilation_rate - 1) paddings, crops = required_space_to_batch_paddings( input_shape[spatial_dims], dilation_rate, [(dilated_filter_shape - 1) // 2, dilated_filter_shape - 1 - (dilated_filter_shape - 1) // 2]) Because space_to_batch_nd and batch_to_space_nd assume that the spatial dimensions are contiguous starting at the second dimension, but the specified spatial_dims may not be, we must adjust dilation_rate, paddings and crops in order to be usable with these operations. For a given dimension, if the block size is 1, and both the starting and ending padding and crop amounts are 0, then space_to_batch_nd effectively leaves that dimension alone, which is what is needed for dimensions not part of spatial_dims. Furthermore, space_to_batch_nd and batch_to_space_nd handle this case efficiently for any number of leading and trailing dimensions. For 0 <= i < len(spatial_dims), we assign: adjusted_dilation_rate[spatial_dims[i] - 1] = dilation_rate[i] adjusted_paddings[spatial_dims[i] - 1, :] = paddings[i, :] adjusted_crops[spatial_dims[i] - 1, :] = crops[i, :] All unassigned values of adjusted_dilation_rate default to 1, while all unassigned values of adjusted_paddings and adjusted_crops default to 0. Note in the case that dilation_rate is not uniformly 1, specifying "VALID" padding is equivalent to specifying padding = "SAME" with a filter_shape of [1]*N. Advanced usage. Note the following optimization: A sequence of with_space_to_batch operations with identical (not uniformly 1) dilation_rate parameters and "VALID" padding net = with_space_to_batch(net, dilation_rate, "VALID", op_1) ... net = with_space_to_batch(net, dilation_rate, "VALID", op_k) can be combined into a single with_space_to_batch operation as follows: def combined_op(converted_input, num_spatial_dims, _): result = op_1(converted_input, num_spatial_dims, "VALID") ... result = op_k(result, num_spatial_dims, "VALID") net = with_space_to_batch(net, dilation_rate, "VALID", combined_op) This eliminates the overhead of k-1 calls to space_to_batch_nd and batch_to_space_nd. Similarly, a sequence of with_space_to_batch operations with identical (not uniformly 1) dilation_rate parameters, "SAME" padding, and odd filter dimensions net = with_space_to_batch(net, dilation_rate, "SAME", op_1, filter_shape_1) ... net = with_space_to_batch(net, dilation_rate, "SAME", op_k, filter_shape_k) can be combined into a single with_space_to_batch operation as follows: def combined_op(converted_input, num_spatial_dims, _): result = op_1(converted_input, num_spatial_dims, "SAME") ... result = op_k(result, num_spatial_dims, "SAME") net = with_space_to_batch(net, dilation_rate, "VALID", combined_op)

 Args
  input   Tensor of rank > max(spatial_dims).
  dilation_rate   int32 Tensor of known shape [num_spatial_dims].
  padding   str constant equal to "VALID" or "SAME"
  op   Function that maps (input, num_spatial_dims, padding) -> output
  filter_shape   If padding = "SAME", specifies the shape of the convolution kernel/pooling window as an integer Tensor of shape [>=num_spatial_dims]. If padding = "VALID", filter_shape is ignored and need not be specified.
  spatial_dims   Monotonically increasing sequence of num_spatial_dims integers (which are >= 1) specifying the spatial dimensions of input and output. Defaults to: range(1, num_spatial_dims+1).
  data_format   A string or None. Specifies whether the channel dimension of the input and output is the last dimension (default, or if data_format does not start with "NC"), or the second dimension (if data_format starts with "NC"). For N=1, the valid values are "NWC" (default) and "NCW". For N=2, the valid values are "NHWC" (default) and "NCHW". For N=3, the valid values are "NDHWC" (default) and "NCDHW".

 Returns   The output Tensor as described above, dimensions will vary based on the op provided.

 Raises
  ValueError   if padding is invalid or the arguments are incompatible.
  ValueError   if spatial_dims are invalid.


tf.nondifferentiable_batch_function     View source on GitHub    Batches the computation done by the decorated function.  View aliases  Compat aliases for migration
See Migration guide for more details. tf.compat.v1.nondifferentiable_batch_function
tf.nondifferentiable_batch_function(
    num_batch_threads, max_batch_size, batch_timeout_micros,
    allowed_batch_sizes=None, max_enqueued_batches=10, autograph=True,
    enable_large_batch_splitting=True
)
 So, for example, in the following code @batch_function(1, 2, 3)
def layer(a):
  return tf.matmul(a, a)

b = layer(w)
 if more than one session.run call is simultaneously trying to compute b the values of w will be gathered, non-deterministically concatenated along the first axis, and only one thread will run the computation. See the documentation of the Batch op for more details. Assumes that all arguments of the decorated function are Tensors which will be batched along their first dimension. SparseTensor is not supported. The return value of the decorated function must be a Tensor or a list/tuple of Tensors.

 Args
  num_batch_threads   Number of scheduling threads for processing batches of work. Determines the number of batches processed in parallel.
  max_batch_size   Batch sizes will never be bigger than this.
  batch_timeout_micros   Maximum number of microseconds to wait before outputting an incomplete batch.
  allowed_batch_sizes   Optional list of allowed batch sizes. If left empty, does nothing. Otherwise, supplies a list of batch sizes, causing the op to pad batches up to one of those sizes. The entries must increase monotonically, and the final entry must equal max_batch_size.
  max_enqueued_batches   The maximum depth of the batch queue. Defaults to 10.
  autograph   Whether to use autograph to compile python and eager style code for efficient graph-mode execution.
  enable_large_batch_splitting   The value of this option doesn't affect processing output given the same input; it affects implementation details as stated below: 1. Improve batching efficiency by eliminating unnecessary adding. 2.max_batch_size specifies the limit of input and allowed_batch_sizes specifies the limit of a task to be processed. API user can give an input of size 128 when 'max_execution_batch_size' is 32 -> implementation can split input of 128 into 4 x 32, schedule concurrent processing, and then return concatenated results corresponding to 128.

 Returns   The decorated function will return the unbatched computation output Tensors.


tf.norm     View source on GitHub    Computes the norm of vectors, matrices, and tensors.  View aliases  Main aliases
tf.linalg.norm
tf.norm(
    tensor, ord='euclidean', axis=None, keepdims=None, name=None
)
 This function can compute several different vector norms (the 1-norm, the Euclidean or 2-norm, the inf-norm, and in general the p-norm for p > 0) and matrix norms (Frobenius, 1-norm, 2-norm and inf-norm).

 Args
  tensor   Tensor of types float32, float64, complex64, complex128
  ord   Order of the norm. Supported values are 'fro', 'euclidean', 1, 2, np.inf and any positive real number yielding the corresponding p-norm. Default is 'euclidean' which is equivalent to Frobenius norm if tensor is a matrix and equivalent to 2-norm for vectors. Some restrictions apply: a) The Frobenius norm 'fro' is not defined for vectors, b) If axis is a 2-tuple (matrix norm), only 'euclidean', 'fro', 1, 2, np.inf are supported. See the description of axis on how to compute norms for a batch of vectors or matrices stored in a tensor.
  axis   If axis is None (the default), the input is considered a vector and a single vector norm is computed over the entire set of values in the tensor, i.e. norm(tensor, ord=ord) is equivalent to norm(reshape(tensor, [-1]), ord=ord). If axis is a Python integer, the input is considered a batch of vectors, and axis determines the axis in tensor over which to compute vector norms. If axis is a 2-tuple of Python integers it is considered a batch of matrices and axis determines the axes in tensor over which to compute a matrix norm. Negative indices are supported. Example: If you are passing a tensor that can be either a matrix or a batch of matrices at runtime, pass axis=[-2,-1] instead of axis=None to make sure that matrix norms are computed.
  keepdims   If True, the axis indicated in axis are kept with size 1. Otherwise, the dimensions in axis are removed from the output shape.
  name   The name of the op.

 Returns
  output   A Tensor of the same type as tensor, containing the vector or matrix norms. If keepdims is True then the rank of output is equal to the rank of tensor. Otherwise, if axis is none the output is a scalar, if axis is an integer, the rank of output is one less than the rank of tensor, if axis is a 2-tuple the rank of output is two less than the rank of tensor.

 Raises
  ValueError   If ord or axis is invalid.    Numpy Compatibility Mostly equivalent to numpy.linalg.norm. Not supported: ord <= 0, 2-norm for matrices, nuclear norm. Other differences: a) If axis is None, treats the flattened tensor as a vector regardless of rank. b) Explicitly supports 'euclidean' norm as the default, including for higher order tensors.


tf.no_gradient     View source on GitHub    Specifies that ops of type op_type is not differentiable.  View aliases  Compat aliases for migration
See Migration guide for more details. tf.compat.v1.NoGradient, tf.compat.v1.NotDifferentiable, tf.compat.v1.no_gradient
tf.no_gradient(
    op_type
)
 This function should not be used for operations that have a well-defined gradient that is not yet implemented. This function is only used when defining a new op type. It may be used for ops such as tf.size() that are not differentiable. For example: tf.no_gradient("Size")
 The gradient computed for 'op_type' will then propagate zeros. For ops that have a well-defined gradient but are not yet implemented, no declaration should be made, and an error must be thrown if an attempt to request its gradient is made.

 Args
  op_type   The string type of an operation. This corresponds to the OpDef.name field for the proto that defines the operation.

 Raises
  TypeError   If op_type is not a string.


tf.no_op Does nothing. Only useful as a placeholder for control edges.  View aliases  Compat aliases for migration
See Migration guide for more details. tf.compat.v1.no_op
tf.no_op(
    name=None
)

 Args
  name   A name for the operation (optional).

 Returns   The created Operation.


tf.numpy_function     View source on GitHub    Wraps a python function and uses it as a TensorFlow op.  View aliases  Compat aliases for migration
See Migration guide for more details. tf.compat.v1.numpy_function
tf.numpy_function(
    func, inp, Tout, name=None
)
 Given a python function func wrap this function as an operation in a TensorFlow function. func must take numpy arrays as its arguments and return numpy arrays as its outputs. The following example creates a TensorFlow graph with np.sinh() as an operation in the graph:
def my_numpy_func(x):
  # x will be a numpy array with the contents of the input to the
  # tf.function
  return np.sinh(x)
@tf.function(input_signature=[tf.TensorSpec(None, tf.float32)])
def tf_function(input):
  y = tf.numpy_function(my_numpy_func, [input], tf.float32)
  return y * y
tf_function(tf.constant(1.))
<tf.Tensor: shape=(), dtype=float32, numpy=1.3810978>
 Comparison to tf.py_function: tf.py_function and tf.numpy_function are very similar, except that tf.numpy_function takes numpy arrays, and not tf.Tensors. If you want the function to contain tf.Tensors, and have any TensorFlow operations executed in the function be differentiable, please use tf.py_function.
Note: The tf.numpy_function operation has the following known limitations:
  The body of the function (i.e. func) will not be serialized in a tf.SavedModel. Therefore, you should not use this function if you need to serialize your model and restore it in a different environment. The operation must run in the same address space as the Python program that calls tf.numpy_function(). If you are using distributed TensorFlow, you must run a tf.distribute.Server in the same process as the program that calls tf.numpy_function you must pin the created operation to a device in that server (e.g. using with tf.device():). Since the function takes numpy arrays, you cannot take gradients through a numpy_function. If you require something that is differentiable, please consider using tf.py_function. The resulting function is assumed stateful and will never be optimized.

 Args
  func   A Python function, which accepts numpy.ndarray objects as arguments and returns a list of numpy.ndarray objects (or a single numpy.ndarray). This function must accept as many arguments as there are tensors in inp, and these argument types will match the corresponding tf.Tensor objects in inp. The returns numpy.ndarrays must match the number and types defined Tout. Important Note: Input and output numpy.ndarrays of func are not guaranteed to be copies. In some cases their underlying memory will be shared with the corresponding TensorFlow tensors. In-place modification or storing func input or return values in python datastructures without explicit (np.)copy can have non-deterministic consequences.
  inp   A list of tf.Tensor objects.
  Tout   A list or tuple of tensorflow data types or a single tensorflow data type if there is only one, indicating what func returns.
  name   (Optional) A name for the operation.

 Returns   Single or list of tf.Tensor which func computes.


tf.ones     View source on GitHub    Creates a tensor with all elements set to one (1).  View aliases  Compat aliases for migration
See Migration guide for more details. tf.compat.v1.ones
tf.ones(
    shape, dtype=tf.dtypes.float32, name=None
)
 See also tf.ones_like, tf.zeros, tf.fill, tf.eye. This operation returns a tensor of type dtype with shape shape and all elements set to one.
tf.ones([3, 4], tf.int32)
<tf.Tensor: shape=(3, 4), dtype=int32, numpy=
array([[1, 1, 1, 1],
       [1, 1, 1, 1],
       [1, 1, 1, 1]], dtype=int32)>

 Args
  shape   A list of integers, a tuple of integers, or a 1-D Tensor of type int32.
  dtype   Optional DType of an element in the resulting Tensor. Default is tf.float32.
  name   Optional string. A name for the operation.

 Returns   A Tensor with all elements set to one (1).


tf.ones_initializer Initializer that generates tensors initialized to 1. Initializers allow you to pre-specify an initialization strategy, encoded in the Initializer object, without knowing the shape and dtype of the variable being initialized. Examples:
def make_variables(k, initializer):
  return (tf.Variable(initializer(shape=[k], dtype=tf.float32)),
          tf.Variable(initializer(shape=[k, k], dtype=tf.float32)))
v1, v2 = make_variables(3, tf.ones_initializer())
v1
<tf.Variable ... shape=(3,) ... numpy=array([1., 1., 1.], dtype=float32)>
v2
<tf.Variable ... shape=(3, 3) ... numpy=
array([[1., 1., 1.],
       [1., 1., 1.],
       [1., 1., 1.]], dtype=float32)>
make_variables(4, tf.random_uniform_initializer(minval=-1., maxval=1.))
(<tf.Variable...shape=(4,) dtype=float32...>, <tf.Variable...shape=(4, 4) ...
 Methods from_config View source
@classmethod
from_config(
    config
)
 Instantiates an initializer from a configuration dictionary. Example: initializer = RandomUniform(-1, 1)
config = initializer.get_config()
initializer = RandomUniform.from_config(config)

 Args
  config   A Python dictionary. It will typically be the output of get_config.

 Returns   An Initializer instance.
 get_config View source
get_config()
 Returns the configuration of the initializer as a JSON-serializable dict.

 Returns   A JSON-serializable Python dict.
 __call__ View source
__call__(
    shape, dtype=tf.dtypes.float32, **kwargs
)
 Returns a tensor object initialized as specified by the initializer.

 Args
  shape   Shape of the tensor.
  dtype   Optional dtype of the tensor. Only numeric or boolean dtypes are supported.
  **kwargs   Additional keyword arguments.

 Raises
  ValuesError   If the dtype is not numeric or boolean.


tf.ones_like     View source on GitHub    Creates a tensor of all ones that has the same shape as the input.
tf.ones_like(
    input, dtype=None, name=None
)
 See also tf.ones. Given a single tensor (tensor), this operation returns a tensor of the same type and shape as tensor with all elements set to 1. Optionally, you can use dtype to specify a new type for the returned tensor. For example:
tensor = tf.constant([[1, 2, 3], [4, 5, 6]])
tf.ones_like(tensor)
<tf.Tensor: shape=(2, 3), dtype=int32, numpy=
  array([[1, 1, 1],
         [1, 1, 1]], dtype=int32)>

 Args
  input   A Tensor.
  dtype   A type for the returned Tensor. Must be float16, float32, float64, int8, uint8, int16, uint16, int32, int64, complex64, complex128, bool or string.
  name   A name for the operation (optional).

 Returns   A Tensor with all elements set to one.


tf.one_hot     View source on GitHub    Returns a one-hot tensor.  View aliases  Compat aliases for migration
See Migration guide for more details. tf.compat.v1.one_hot
tf.one_hot(
    indices, depth, on_value=None, off_value=None, axis=None, dtype=None, name=None
)
 See also tf.fill, tf.eye. The locations represented by indices in indices take value on_value, while all other locations take value off_value. on_value and off_value must have matching data types. If dtype is also provided, they must be the same data type as specified by dtype. If on_value is not provided, it will default to the value 1 with type dtype If off_value is not provided, it will default to the value 0 with type dtype If the input indices is rank N, the output will have rank N+1. The new axis is created at dimension axis (default: the new axis is appended at the end). If indices is a scalar the output shape will be a vector of length depth If indices is a vector of length features, the output shape will be: features x depth if axis == -1
depth x features if axis == 0
 If indices is a matrix (batch) with shape [batch, features], the output shape will be: batch x features x depth if axis == -1
batch x depth x features if axis == 1
depth x batch x features if axis == 0
 If indices is a RaggedTensor, the 'axis' argument must be positive and refer to a non-ragged axis. The output will be equivalent to applying 'one_hot' on the values of the RaggedTensor, and creating a new RaggedTensor from the result. If dtype is not provided, it will attempt to assume the data type of on_value or off_value, if one or both are passed in. If none of on_value, off_value, or dtype are provided, dtype will default to the value tf.float32.
Note: If a non-numeric data type output is desired (tf.string, tf.bool, etc.), both on_value and off_value must be provided to one_hot.
 For example: indices = [0, 1, 2]
depth = 3
tf.one_hot(indices, depth)  # output: [3 x 3]
# [[1., 0., 0.],
#  [0., 1., 0.],
#  [0., 0., 1.]]

indices = [0, 2, -1, 1]
depth = 3
tf.one_hot(indices, depth,
           on_value=5.0, off_value=0.0,
           axis=-1)  # output: [4 x 3]
# [[5.0, 0.0, 0.0],  # one_hot(0)
#  [0.0, 0.0, 5.0],  # one_hot(2)
#  [0.0, 0.0, 0.0],  # one_hot(-1)
#  [0.0, 5.0, 0.0]]  # one_hot(1)

indices = [[0, 2], [1, -1]]
depth = 3
tf.one_hot(indices, depth,
           on_value=1.0, off_value=0.0,
           axis=-1)  # output: [2 x 2 x 3]
# [[[1.0, 0.0, 0.0],   # one_hot(0)
#   [0.0, 0.0, 1.0]],  # one_hot(2)
#  [[0.0, 1.0, 0.0],   # one_hot(1)
#   [0.0, 0.0, 0.0]]]  # one_hot(-1)

indices = tf.ragged.constant([[0, 1], [2]])
depth = 3
tf.one_hot(indices, depth)  # output: [2 x None x 3]
# [[[1., 0., 0.],
#   [0., 1., 0.]],
#  [[0., 0., 1.]]]

 Args
  indices   A Tensor of indices.
  depth   A scalar defining the depth of the one hot dimension.
  on_value   A scalar defining the value to fill in output when indices[j] = i. (default: 1)
  off_value   A scalar defining the value to fill in output when indices[j] != i. (default: 0)
  axis   The axis to fill (default: -1, a new inner-most axis).
  dtype   The data type of the output tensor.
  name   A name for the operation (optional).

 Returns
  output   The one-hot tensor.

 Raises
  TypeError   If dtype of either on_value or off_value don't match dtype
  TypeError   If dtype of on_value and off_value don't match one another


tf.Operation     View source on GitHub    Represents a graph node that performs computation on tensors.  View aliases  Compat aliases for migration
See Migration guide for more details. tf.compat.v1.Operation
tf.Operation(
    node_def, g, inputs=None, output_types=None, control_inputs=None,
    input_types=None, original_op=None, op_def=None
)
 An Operation is a node in a tf.Graph that takes zero or more Tensor objects as input, and produces zero or more Tensor objects as output. Objects of type Operation are created by calling a Python op constructor (such as tf.matmul) within a tf.function or under a tf.Graph.as_default context manager. For example, within a tf.function, c = tf.matmul(a, b) creates an Operation of type "MatMul" that takes tensors a and b as input, and produces c as output. If a tf.compat.v1.Session is used, an Operation of a tf.Graph can be executed by passing it to tf.Session.run. op.run() is a shortcut for calling tf.compat.v1.get_default_session().run(op).

 Args
  node_def   node_def_pb2.NodeDef. NodeDef for the Operation. Used for attributes of node_def_pb2.NodeDef, typically name, op, and device. The input attribute is irrelevant here as it will be computed when generating the model.
  g   Graph. The parent graph.
  inputs   list of Tensor objects. The inputs to this Operation.
  output_types   list of DType objects. List of the types of the Tensors computed by this operation. The length of this list indicates the number of output endpoints of the Operation.
  control_inputs   list of operations or tensors from which to have a control dependency.
  input_types   List of DType objects representing the types of the tensors accepted by the Operation. By default uses [x.dtype.base_dtype for x in inputs]. Operations that expect reference-typed inputs must specify these explicitly.
  original_op   Optional. Used to associate the new Operation with an existing Operation (for example, a replica with the op that was replicated).
  op_def   Optional. The op_def_pb2.OpDef proto that describes the op type that this Operation represents.

 Raises
  TypeError   if control inputs are not Operations or Tensors, or if node_def is not a NodeDef, or if g is not a Graph, or if inputs are not tensors, or if inputs and input_types are incompatible.
  ValueError   if the node_def name is not valid.

 Attributes
  control_inputs   The Operation objects on which this op has a control dependency. Before this op is executed, TensorFlow will ensure that the operations in self.control_inputs have finished executing. This mechanism can be used to run ops sequentially for performance reasons, or to ensure that the side effects of an op are observed in the correct order.

  device   The name of the device to which this op has been assigned, if any.
  graph   The Graph that contains this operation.
  inputs   The sequence of Tensor objects representing the data inputs of this op.
  name   The full name of this operation.
  node_def   Returns the NodeDef representation of this operation.
  op_def   Returns the OpDef proto that represents the type of this op.
  outputs   The list of Tensor objects representing the outputs of this op.
  traceback   Returns the call stack from when this operation was constructed.
  type   The type of the op (e.g. "MatMul").    Methods colocation_groups View source
colocation_groups()
 Returns the list of colocation groups of the op. get_attr View source
get_attr(
    name
)
 Returns the value of the attr of this op with the given name.

 Args
  name   The name of the attr to fetch.

 Returns   The value of the attr, as a Python object.

 Raises
  ValueError   If this op does not have an attr with the given name.    run View source
run(
    feed_dict=None, session=None
)
 Runs this operation in a Session. Calling this method will execute all preceding operations that produce the inputs needed for this operation.
Note: Before invoking Operation.run(), its graph must have been launched in a session, and either a default session must be available, or session must be specified explicitly.

 Args
  feed_dict   A dictionary that maps Tensor objects to feed values. See tf.Session.run for a description of the valid feed values.
  session   (Optional.) The Session to be used to run to this operation. If none, the default session will be used.    values View source
values()
 DEPRECATED: Use outputs.


tf.OptionalSpec     View source on GitHub    Type specification for tf.experimental.Optional. Inherits From: TypeSpec  View aliases  Compat aliases for migration
See Migration guide for more details. tf.compat.v1.OptionalSpec, tf.compat.v1.data.experimental.OptionalStructure
tf.OptionalSpec(
    element_spec
)
 For instance, tf.OptionalSpec can be used to define a tf.function that takes tf.experimental.Optional as an input argument:
@tf.function(input_signature=[tf.OptionalSpec(
  tf.TensorSpec(shape=(), dtype=tf.int32, name=None))])
def maybe_square(optional):
  if optional.has_value():
    x = optional.get_value()
    return x * x
  return -1
optional = tf.experimental.Optional.from_value(5)
print(maybe_square(optional))
tf.Tensor(25, shape=(), dtype=int32)

 Attributes
  element_spec   A nested structure of TypeSpec objects that represents the type specification of the optional element.
  value_type   The Python type for values that are compatible with this TypeSpec. In particular, all values that are compatible with this TypeSpec must be an instance of this type.
   Methods from_value View source
@staticmethod
from_value(
    value
)
 is_compatible_with View source
is_compatible_with(
    spec_or_value
)
 Returns true if spec_or_value is compatible with this TypeSpec. most_specific_compatible_type View source
most_specific_compatible_type(
    other
)
 Returns the most specific TypeSpec compatible with self and other.

 Args
  other   A TypeSpec.

 Raises
  ValueError   If there is no TypeSpec that is compatible with both self and other.    __eq__ View source
__eq__(
    other
)
 Return self==value. __ne__ View source
__ne__(
    other
)
 Return self!=value.


tf.pad     View source on GitHub    Pads a tensor.
tf.pad(
    tensor, paddings, mode='CONSTANT', constant_values=0, name=None
)
 This operation pads a tensor according to the paddings you specify. paddings is an integer tensor with shape [n, 2], where n is the rank of tensor. For each dimension D of input, paddings[D, 0] indicates how many values to add before the contents of tensor in that dimension, and paddings[D, 1] indicates how many values to add after the contents of tensor in that dimension. If mode is "REFLECT" then both paddings[D, 0] and paddings[D, 1] must be no greater than tensor.dim_size(D) - 1. If mode is "SYMMETRIC" then both paddings[D, 0] and paddings[D, 1] must be no greater than tensor.dim_size(D). The padded size of each dimension D of the output is: paddings[D, 0] + tensor.dim_size(D) + paddings[D, 1] For example: t = tf.constant([[1, 2, 3], [4, 5, 6]])
paddings = tf.constant([[1, 1,], [2, 2]])
# 'constant_values' is 0.
# rank of 't' is 2.
tf.pad(t, paddings, "CONSTANT")  # [[0, 0, 0, 0, 0, 0, 0],
                                 #  [0, 0, 1, 2, 3, 0, 0],
                                 #  [0, 0, 4, 5, 6, 0, 0],
                                 #  [0, 0, 0, 0, 0, 0, 0]]

tf.pad(t, paddings, "REFLECT")  # [[6, 5, 4, 5, 6, 5, 4],
                                #  [3, 2, 1, 2, 3, 2, 1],
                                #  [6, 5, 4, 5, 6, 5, 4],
                                #  [3, 2, 1, 2, 3, 2, 1]]

tf.pad(t, paddings, "SYMMETRIC")  # [[2, 1, 1, 2, 3, 3, 2],
                                  #  [2, 1, 1, 2, 3, 3, 2],
                                  #  [5, 4, 4, 5, 6, 6, 5],
                                  #  [5, 4, 4, 5, 6, 6, 5]]

 Args
  tensor   A Tensor.
  paddings   A Tensor of type int32.
  mode   One of "CONSTANT", "REFLECT", or "SYMMETRIC" (case-insensitive)
  constant_values   In "CONSTANT" mode, the scalar pad value to use. Must be same type as tensor.
  name   A name for the operation (optional).

 Returns   A Tensor. Has the same type as tensor.

 Raises
  ValueError   When mode is not one of "CONSTANT", "REFLECT", or "SYMMETRIC".


tf.parallel_stack     View source on GitHub    Stacks a list of rank-R tensors into one rank-(R+1) tensor in parallel.  View aliases  Compat aliases for migration
See Migration guide for more details. tf.compat.v1.parallel_stack
tf.parallel_stack(
    values, name='parallel_stack'
)
 Requires that the shape of inputs be known at graph construction time. Packs the list of tensors in values into a tensor with rank one higher than each tensor in values, by packing them along the first dimension. Given a list of length N of tensors of shape (A, B, C); the output tensor will have the shape (N, A, B, C). For example: x = tf.constant([1, 4])
y = tf.constant([2, 5])
z = tf.constant([3, 6])
tf.parallel_stack([x, y, z])  # [[1, 4], [2, 5], [3, 6]]
 The difference between stack and parallel_stack is that stack requires all the inputs be computed before the operation will begin but doesn't require that the input shapes be known during graph construction. parallel_stack will copy pieces of the input into the output as they become available, in some situations this can provide a performance benefit. Unlike stack, parallel_stack does NOT support backpropagation. This is the opposite of unstack. The numpy equivalent is tf.parallel_stack([x, y, z]) = np.asarray([x, y, z])

 Args
  values   A list of Tensor objects with the same shape and type.
  name   A name for this operation (optional).

 Returns
  output   A stacked Tensor with the same type as values.


tf.print     View source on GitHub    Print the specified inputs.  View aliases  Compat aliases for migration
See Migration guide for more details. tf.compat.v1.print
tf.print(
    *inputs, **kwargs
)
 A TensorFlow operator that prints the specified inputs to a desired output stream or logging level. The inputs may be dense or sparse Tensors, primitive python objects, data structures that contain tensors, and printable Python objects. Printed tensors will recursively show the first and last elements of each dimension to summarize. Example: Single-input usage: tensor = tf.range(10)
tf.print(tensor, output_stream=sys.stderr)
 (This prints "[0 1 2 ... 7 8 9]" to sys.stderr) Multi-input usage: tensor = tf.range(10)
tf.print("tensors:", tensor, {2: tensor * 2}, output_stream=sys.stdout)
 (This prints "tensors: [0 1 2 ... 7 8 9] {2: [0 2 4 ... 14 16 18]}" to sys.stdout) Changing the input separator: tensor_a = tf.range(2)
tensor_b = tensor_a * 2
tf.print(tensor_a, tensor_b, output_stream=sys.stderr, sep=',')
 (This prints "[0 1],[0 2]" to sys.stderr) Usage in a tf.function: @tf.function
def f():
    tensor = tf.range(10)
    tf.print(tensor, output_stream=sys.stderr)
    return tensor

range_tensor = f()
 (This prints "[0 1 2 ... 7 8 9]" to sys.stderr) @compatibility(TF 1.x Graphs and Sessions) In graphs manually created outside of tf.function, this method returns the created TF operator that prints the data. To make sure the operator runs, users need to pass the produced op to tf.compat.v1.Session's run method, or to use the op as a control dependency for executed ops by specifying with tf.compat.v1.control_dependencies([print_op]). @end_compatibility Compatibility usage in TF 1.x graphs: sess = tf.compat.v1.Session()
with sess.as_default():
    tensor = tf.range(10)
    print_op = tf.print("tensors:", tensor, {2: tensor * 2},
                        output_stream=sys.stdout)
    with tf.control_dependencies([print_op]):
      tripled_tensor = tensor * 3
    sess.run(tripled_tensor)
 (This prints "tensors: [0 1 2 ... 7 8 9] {2: [0 2 4 ... 14 16 18]}" to sys.stdout)
Note: In Jupyter notebooks and colabs, tf.print prints to the notebook cell outputs. It will not write to the notebook kernel's console logs.

 Args
  *inputs   Positional arguments that are the inputs to print. Inputs in the printed output will be separated by spaces. Inputs may be python primitives, tensors, data structures such as dicts and lists that may contain tensors (with the data structures possibly nested in arbitrary ways), and printable python objects.
  output_stream   The output stream, logging level, or file to print to. Defaults to sys.stderr, but sys.stdout, tf.compat.v1.logging.info, tf.compat.v1.logging.warning, tf.compat.v1.logging.error, absl.logging.info, absl.logging.warning and absl.logging.error are also supported. To print to a file, pass a string started with "file://" followed by the file path, e.g., "file:///tmp/foo.out".
  summarize   The first and last summarize elements within each dimension are recursively printed per Tensor. If None, then the first 3 and last 3 elements of each dimension are printed for each tensor. If set to -1, it will print all elements of every tensor.
  sep   The string to use to separate the inputs. Defaults to " ".
  end   End character that is appended at the end the printed string. Defaults to the newline character.
  name   A name for the operation (optional).

 Returns   None when executing eagerly. During graph tracing this returns a TF operator that prints the specified inputs in the specified output stream or logging level. This operator will be automatically executed except inside of tf.compat.v1 graphs and sessions.

 Raises
  ValueError   If an unsupported output stream is specified.


Module: tf.profiler Public API for tf.profiler namespace. Modules experimental module: Public API for tf.profiler.experimental namespace.


Module: tf.profiler.experimental Public API for tf.profiler.experimental namespace. Modules client module: Public API for tf.profiler.experimental.client namespace. server module: Public API for tf.profiler.experimental.server namespace. Classes class Profile: Context-manager profile API. class ProfilerOptions: Options for finer control over the profiler. class Trace: Context manager that generates a trace event in the profiler. Functions start(...): Start profiling TensorFlow performance. stop(...): Stops the current profiling session.


Module: tf.profiler.experimental.client Public API for tf.profiler.experimental.client namespace. Functions monitor(...): Sends grpc requests to profiler server to perform on-demand monitoring. trace(...): Sends gRPC requests to one or more profiler servers to perform on-demand profiling.


tf.profiler.experimental.client.monitor Sends grpc requests to profiler server to perform on-demand monitoring.
tf.profiler.experimental.client.monitor(
    service_addr, duration_ms, level=1
)
 The monitoring result is a light weight performance summary of your model execution. This method will block the caller thread until it receives the monitoring result. This method currently supports Cloud TPU only.

 Args
  service_addr   gRPC address of profiler service e.g. grpc://10.0.0.2:8466.
  duration_ms   Duration of monitoring in ms.
  level   Choose a monitoring level between 1 and 2 to monitor your job. Level 2 is more verbose than level 1 and shows more metrics.

 Returns   A string of monitoring output.
 Example usage: # Continuously send gRPC requests to the Cloud TPU to monitor the model
# execution.

for query in range(0, 100):
  print(
    tf.profiler.experimental.client.monitor('grpc://10.0.0.2:8466', 1000))


tf.profiler.experimental.client.trace   Sends gRPC requests to one or more profiler servers to perform on-demand profiling.
tf.profiler.experimental.client.trace(
    service_addr, logdir, duration_ms, worker_list='',
    num_tracing_attempts=3, options=None
)
 This method will block the calling thread until it receives responses from all servers or until deadline expiration. Both single host and multiple host profiling are supported on CPU, GPU, and TPU. The profiled results will be saved by each server to the specified TensorBoard log directory (i.e. the directory you save your model checkpoints). Use the TensorBoard profile plugin to view the visualization and analysis results.

 Args
  service_addr   A comma delimited string of gRPC addresses of the workers to profile. e.g. service_addr='grpc://localhost:6009' service_addr='grpc://10.0.0.2:8466,grpc://10.0.0.3:8466' service_addr='grpc://localhost:12345,grpc://localhost:23456'
  logdir   Path to save profile data to, typically a TensorBoard log directory. This path must be accessible to both the client and server. e.g. logdir='gs://your_tb_dir'
  duration_ms   Duration of tracing or monitoring in milliseconds. Must be greater than zero.
  worker_list   An optional TPU only configuration. The list of workers to profile in the current session.
  num_tracing_attempts   Optional. Automatically retry N times when no trace event is collected (default 3).
  options   profiler.experimental.ProfilerOptions namedtuple for miscellaneous profiler options.

 Raises
  InvalidArgumentError   For when arguments fail validation checks.
  UnavailableError   If no trace event was collected.    Example usage (CPU/GPU): # Start a profiler server before your model runs.
tf.profiler.experimental.server.start(6009)
# (Model code goes here).
# Send gRPC request to the profiler server to collect a trace of your model.
tf.profiler.experimental.client.trace('grpc://localhost:6009',
                                      '/nfs/tb_log', 2000)
 Example usage (Multiple GPUs): # E.g. your worker IP addresses are 10.0.0.2, 10.0.0.3, 10.0.0.4, and you
# would like to schedule start of profiling 1 second from now, for a
# duration of 2 seconds.
options['delay_ms'] = 1000
tf.profiler.experimental.client.trace(
    'grpc://10.0.0.2:8466,grpc://10.0.0.3:8466,grpc://10.0.0.4:8466',
    'gs://your_tb_dir',
    2000,
    options=options)
 Example usage (TPU): # Send gRPC request to a TPU worker to collect a trace of your model. A
# profiler service has been started in the TPU worker at port 8466.
# E.g. your TPU IP address is 10.0.0.2 and you want to profile for 2 seconds
# .
tf.profiler.experimental.client.trace('grpc://10.0.0.2:8466',
                                      'gs://your_tb_dir', 2000)
 Example usage (Multiple TPUs): # Send gRPC request to a TPU pod to collect a trace of your model on
# multipleTPUs. A profiler service has been started in all the TPU workers
# at theport 8466.
# E.g. your TPU IP addresses are 10.0.0.2, 10.0.0.3, 10.0.0.4, and you want
# to profile for 2 seconds.
tf.profiler.experimental.client.trace('grpc://10.0.0.2:8466',
                                      'gs://your_tb_dir',
                                      2000, '10.0.0.2,10.0.0.3,10.0.0.4')
 Launch TensorBoard and point it to the same logdir you provided to this API. # logdir can be gs://your_tb_dir as in the above examples.
$ tensorboard --logdir=/tmp/tb_log
 Open your browser and go to localhost:6006/#profile to view profiling results.


tf.profiler.experimental.Profile Context-manager profile API.
tf.profiler.experimental.Profile(
    logdir, options=None
)
 Profiling will start when entering the scope, and stop and save the results to the logdir when exits the scope. Open TensorBoard profile tab to view results. Example usage: with tf.profiler.experimental.Profile("/path/to/logdir"):
  # do some work

 Args
  logdir   profile data will save to this directory.
  options   An optional tf.profiler.ProfilerOptions can be provided to fine tune the profiler's behavior.    Methods __enter__ View source
__enter__()
 __exit__ View source
__exit__(
    typ, value, tb
)


tf.profiler.experimental.ProfilerOptions Options for finer control over the profiler.
tf.profiler.experimental.ProfilerOptions(
    host_tracer_level=2, python_tracer_level=0, device_tracer_level=1, delay_ms=None
)
 Use tf.profiler.ProfilerOptions to control tf.profiler behavior. Fields:
host_tracer_level: Adjust CPU tracing level. Values are: 1 - critical info only, 2 - info, 3 - verbose. [default value is 2]
python_tracer_level: Toggle tracing of Python function calls. Values are: 1 enabled, 0 - disabled [default value is 0]
device_tracer_level: Adjust device (TPU/GPU) tracing level. Values are: 1 - enabled, 0 - disabled [default value is 1]
delay_ms: Requests for all hosts to start profiling at a timestamp that is delay_ms away from the current time. delay_ms is in milliseconds. If zero, each host will start profiling immediately upon receiving the request. Default value is None, allowing the profiler guess the best value.

 Attributes
  host_tracer_level

  python_tracer_level

  device_tracer_level

  delay_ms


Module: tf.profiler.experimental.server Public API for tf.profiler.experimental.server namespace. Functions start(...): Start a profiler grpc server that listens to given port.


tf.profiler.experimental.server.start Start a profiler grpc server that listens to given port.
tf.profiler.experimental.server.start(
    port
)
 The profiler server will exit when the process finishes. The service is defined in tensorflow/core/profiler/profiler_service.proto.

 Args
  port   port profiler server listens to.    Example usage: ```python tf.profiler.experimental.server.start('6009') # do your training here.


tf.profiler.experimental.start Start profiling TensorFlow performance.
tf.profiler.experimental.start(
    logdir, options=None
)

 Args
  logdir   Profiling results log directory.
  options   ProfilerOptions namedtuple to specify miscellaneous profiler options. See example usage below.

 Raises
  AlreadyExistsError   If a profiling session is already running.    Example usage: options = tf.profiler.experimental.ProfilerOptions(host_tracer_level = 3,
                                                   python_tracer_level = 1,
                                                   device_tracer_level = 1)
tf.profiler.experimental.start('logdir_path', options = options)
# Training code here
tf.profiler.experimental.stop()
 To view the profiling results, launch TensorBoard and point it to logdir. Open your browser and go to localhost:6006/#profile to view profiling results.


tf.profiler.experimental.stop Stops the current profiling session.
tf.profiler.experimental.stop(
    save=True
)
 The profiler session will be stopped and profile results can be saved.

 Args
  save   An optional variable to save the results to TensorBoard. Default True.

 Raises
  UnavailableError   If there is no active profiling session.


tf.profiler.experimental.Trace Context manager that generates a trace event in the profiler.
tf.profiler.experimental.Trace(
    name, **kwargs
)
 A trace event will start when entering the context, and stop and save the result to the profiler when exiting the context. Open TensorBoard Profile tab and choose trace viewer to view the trace event in the timeline. Trace events are created only when the profiler is enabled. More information on how to use the profiler can be found at https://tensorflow.org/guide/profiler Example usage: tf.profiler.experimental.start('logdir')
for step in range(num_steps):
  # Creates a trace event for each training step with the step number.
  with tf.profiler.experimental.Trace("Train", step_num=step):
    train_fn()
tf.profiler.experimental.stop()

 Args
  name   The name of the trace event.
  **kwargs   Keyword arguments added to the trace event. Both the key and value are of types that can be converted to strings, which will be interpreted by the profiler according to the traceme name. Example usage:
tf.profiler.experimental.start('logdir')
for step in range(num_steps):
# Creates a trace event for each training step with the
# step number.
with tf.profiler.experimental.Trace("Train", step_num=step):
train_fn()
tf.profiler.experimental.stop()

 The example above uses the keyword argument "step_num" to specify the training step being traced.
   Methods set_metadata View source
set_metadata(
    **kwargs
)
 Sets metadata in this trace event.

 Args
  **kwargs   metadata in key-value pairs.    This method enables setting metadata in a trace event after it is created. Example usage:
def call(function):
  with tf.profiler.experimental.Trace("call",
       function_name=function.name) as tm:
    binary, in_cache = jit_compile(function)
    tm.set_metadata(in_cache=in_cache)
    execute(binary)

 In this example, we want to trace how much time spent on calling a function, which includes compilation and execution. The compilation can be either getting a cached copy of the binary or actually generating the binary, which is indicated by the boolean "in_cache" returned by jit_compile(). We need to use set_metadata() to pass in_cache because we did not know the in_cache value when the trace was created (and we cannot create the trace after jit_compile(), because we want to measure the entire duration of call()). __enter__ View source
__enter__()
 __exit__ View source
__exit__(
    exc_type, exc_val, exc_tb
)


tf.py_function     View source on GitHub    Wraps a python function into a TensorFlow op that executes it eagerly.  View aliases  Compat aliases for migration
See Migration guide for more details. tf.compat.v1.py_function
tf.py_function(
    func, inp, Tout, name=None
)
 This function allows expressing computations in a TensorFlow graph as Python functions. In particular, it wraps a Python function func in a once-differentiable TensorFlow operation that executes it with eager execution enabled. As a consequence, tf.py_function makes it possible to express control flow using Python constructs (if, while, for, etc.), instead of TensorFlow control flow constructs (tf.cond, tf.while_loop). For example, you might use tf.py_function to implement the log huber function: def log_huber(x, m):
  if tf.abs(x) <= m:
    return x**2
  else:
    return m**2 * (1 - 2 * tf.math.log(m) + tf.math.log(x**2))

x = tf.compat.v1.placeholder(tf.float32)
m = tf.compat.v1.placeholder(tf.float32)

y = tf.py_function(func=log_huber, inp=[x, m], Tout=tf.float32)
dy_dx = tf.gradients(y, x)[0]

with tf.compat.v1.Session() as sess:
  # The session executes `log_huber` eagerly. Given the feed values below,
  # it will take the first branch, so `y` evaluates to 1.0 and
  # `dy_dx` evaluates to 2.0.
  y, dy_dx = sess.run([y, dy_dx], feed_dict={x: 1.0, m: 2.0})
 You can also use tf.py_function to debug your models at runtime using Python tools, i.e., you can isolate portions of your code that you want to debug, wrap them in Python functions and insert pdb tracepoints or print statements as desired, and wrap those functions in tf.py_function. For more information on eager execution, see the Eager guide. tf.py_function is similar in spirit to tf.compat.v1.py_func, but unlike the latter, the former lets you use TensorFlow operations in the wrapped Python function. In particular, while tf.compat.v1.py_func only runs on CPUs and wraps functions that take NumPy arrays as inputs and return NumPy arrays as outputs, tf.py_function can be placed on GPUs and wraps functions that take Tensors as inputs, execute TensorFlow operations in their bodies, and return Tensors as outputs. Like tf.compat.v1.py_func, tf.py_function has the following limitations with respect to serialization and distribution:  The body of the function (i.e. func) will not be serialized in a GraphDef. Therefore, you should not use this function if you need to serialize your model and restore it in a different environment. The operation must run in the same address space as the Python program that calls tf.py_function(). If you are using distributed TensorFlow, you must run a tf.distribute.Server in the same process as the program that calls tf.py_function() and you must pin the created operation to a device in that server (e.g. using with tf.device():).

 Args
  func   A Python function which accepts a list of Tensor objects having element types that match the corresponding tf.Tensor objects in inp and returns a list of Tensor objects (or a single Tensor, or None) having element types that match the corresponding values in Tout.
  inp   A list of Tensor objects.
  Tout   A list or tuple of tensorflow data types or a single tensorflow data type if there is only one, indicating what func returns; an empty list if no value is returned (i.e., if the return value is None).
  name   A name for the operation (optional).

 Returns   A list of Tensor or a single Tensor which func computes; an empty list if func returns None.


Module: tf.quantization Public API for tf.quantization namespace. Functions dequantize(...): Dequantize the 'input' tensor into a float or bfloat16 Tensor. fake_quant_with_min_max_args(...): Fake-quantize the 'inputs' tensor, type float to 'outputs' tensor of same type. fake_quant_with_min_max_args_gradient(...): Compute gradients for a FakeQuantWithMinMaxArgs operation. fake_quant_with_min_max_vars(...): Fake-quantize the 'inputs' tensor of type float via global float scalars fake_quant_with_min_max_vars_gradient(...): Compute gradients for a FakeQuantWithMinMaxVars operation. fake_quant_with_min_max_vars_per_channel(...): Fake-quantize the 'inputs' tensor of type float via per-channel floats fake_quant_with_min_max_vars_per_channel_gradient(...): Compute gradients for a FakeQuantWithMinMaxVarsPerChannel operation. quantize(...): Quantize the 'input' tensor of type float to 'output' tensor of type 'T'. quantize_and_dequantize(...): Quantizes then dequantizes a tensor. (deprecated) quantize_and_dequantize_v2(...): Quantizes then dequantizes a tensor. quantized_concat(...): Concatenates quantized tensors along one dimension.


tf.quantization.dequantize     View source on GitHub    Dequantize the 'input' tensor into a float or bfloat16 Tensor.  View aliases  Compat aliases for migration
See Migration guide for more details. tf.compat.v1.dequantize, tf.compat.v1.quantization.dequantize
tf.quantization.dequantize(
    input, min_range, max_range, mode='MIN_COMBINED', name=None,
    axis=None, narrow_range=False, dtype=tf.dtypes.float32
)
 [min_range, max_range] are scalar floats that specify the range for the output. The 'mode' attribute controls exactly which calculations are used to convert the float values to their quantized equivalents. In 'MIN_COMBINED' mode, each value of the tensor will undergo the following: if T == qint8: in[i] += (range(T) + 1)/ 2.0
out[i] = min_range + (in[i]* (max_range - min_range) / range(T))
 here range(T) = numeric_limits<T>::max() - numeric_limits<T>::min() MIN_COMBINED Mode Example If the input comes from a QuantizedRelu6, the output type is quint8 (range of 0-255) but the possible range of QuantizedRelu6 is 0-6. The min_range and max_range values are therefore 0.0 and 6.0. Dequantize on quint8 will take each value, cast to float, and multiply by 6 / 255. Note that if quantizedtype is qint8, the operation will additionally add each value by 128 prior to casting. If the mode is 'MIN_FIRST', then this approach is used: num_discrete_values = 1 << (# of bits in T)
range_adjust = num_discrete_values / (num_discrete_values - 1)
range = (range_max - range_min) * range_adjust
range_scale = range / num_discrete_values
const double offset_input = static_cast<double>(input) - lowest_quantized;
result = range_min + ((input - numeric_limits<T>::min()) * range_scale)
 If the mode is SCALED, dequantization is performed by multiplying each input value by a scaling_factor. (Thus an input of 0 always maps to 0.0). The scaling_factor is determined from min_range, max_range, and narrow_range in a way that is compatible with QuantizeAndDequantize{V2|V3} and QuantizeV2, using the following algorithm:
const int min_expected_T = std::numeric_limits<T>::min() +
  (narrow_range ? 1 : 0);
const int max_expected_T = std::numeric_limits<T>::max();
const float max_expected_T = std::numeric_limits<float>::max();

const float scale_factor =
  (std::numeric_limits<T>::min() == 0) ? (max_range / max_expected_T)
                                       : std::max(min_range / min_expected_T,
                                                  max_range / max_expected_T);

 Args
  input   A Tensor. Must be one of the following types: qint8, quint8, qint32, qint16, quint16.
  min_range   A Tensor of type float32. The minimum scalar value possibly produced for the input.
  max_range   A Tensor of type float32. The maximum scalar value possibly produced for the input.
  mode   An optional string from: "MIN_COMBINED", "MIN_FIRST", "SCALED". Defaults to "MIN_COMBINED".
  narrow_range   An optional bool. Defaults to False.
  axis   An optional int. Defaults to -1.
  dtype   An optional tf.DType from: tf.bfloat16, tf.float32. Defaults to tf.float32. Type of the output tensor. Currently Dequantize supports float and bfloat16. If 'dtype' is 'bfloat16', it only supports 'MIN_COMBINED' mode.
  name   A name for the operation (optional).

 Returns   A Tensor of type dtype.


tf.quantization.fake_quant_with_min_max_args Fake-quantize the 'inputs' tensor, type float to 'outputs' tensor of same type.  View aliases  Compat aliases for migration
See Migration guide for more details. tf.compat.v1.fake_quant_with_min_max_args, tf.compat.v1.quantization.fake_quant_with_min_max_args
tf.quantization.fake_quant_with_min_max_args(
    inputs, min=-6, max=6, num_bits=8, narrow_range=False, name=None
)
 Attributes
[min; max] define the clamping range for the inputs data.
inputs values are quantized into the quantization range ( [0; 2^num_bits - 1] when narrow_range is false and [1; 2^num_bits - 1] when it is true) and then de-quantized and output as floats in [min; max] interval.
num_bits is the bitwidth of the quantization; between 2 and 16, inclusive.  Before quantization, min and max values are adjusted with the following logic. It is suggested to have min <= 0 <= max. If 0 is not in the range of values, the behavior can be unexpected:  If 0 < min < max: min_adj = 0 and max_adj = max - min. If min < max < 0: min_adj = min - max and max_adj = 0. If min <= 0 <= max: scale = (max - min) / (2^num_bits - 1), min_adj = scale * round(min / scale) and max_adj = max + min_adj - min.  Quantization is called fake since the output is still in floating point.

 Args
  inputs   A Tensor of type float32.
  min   An optional float. Defaults to -6.
  max   An optional float. Defaults to 6.
  num_bits   An optional int. Defaults to 8.
  narrow_range   An optional bool. Defaults to False.
  name   A name for the operation (optional).

 Returns   A Tensor of type float32.


tf.quantization.fake_quant_with_min_max_args_gradient Compute gradients for a FakeQuantWithMinMaxArgs operation.  View aliases  Compat aliases for migration
See Migration guide for more details. tf.compat.v1.fake_quant_with_min_max_args_gradient, tf.compat.v1.quantization.fake_quant_with_min_max_args_gradient
tf.quantization.fake_quant_with_min_max_args_gradient(
    gradients, inputs, min=-6, max=6, num_bits=8, narrow_range=False, name=None
)

 Args
  gradients   A Tensor of type float32. Backpropagated gradients above the FakeQuantWithMinMaxArgs operation.
  inputs   A Tensor of type float32. Values passed as inputs to the FakeQuantWithMinMaxArgs operation.
  min   An optional float. Defaults to -6.
  max   An optional float. Defaults to 6.
  num_bits   An optional int. Defaults to 8.
  narrow_range   An optional bool. Defaults to False.
  name   A name for the operation (optional).

 Returns   A Tensor of type float32.


tf.quantization.fake_quant_with_min_max_vars Fake-quantize the 'inputs' tensor of type float via global float scalars  View aliases  Compat aliases for migration
See Migration guide for more details. tf.compat.v1.fake_quant_with_min_max_vars, tf.compat.v1.quantization.fake_quant_with_min_max_vars
tf.quantization.fake_quant_with_min_max_vars(
    inputs, min, max, num_bits=8, narrow_range=False, name=None
)
 Fake-quantize the inputs tensor of type float via global float scalars min and max to outputs tensor of same shape as inputs. Attributes
[min; max] define the clamping range for the inputs data.
inputs values are quantized into the quantization range ( [0; 2^num_bits - 1] when narrow_range is false and [1; 2^num_bits - 1] when it is true) and then de-quantized and output as floats in [min; max] interval.
num_bits is the bitwidth of the quantization; between 2 and 16, inclusive.  Before quantization, min and max values are adjusted with the following logic. It is suggested to have min <= 0 <= max. If 0 is not in the range of values, the behavior can be unexpected:  If 0 < min < max: min_adj = 0 and max_adj = max - min. If min < max < 0: min_adj = min - max and max_adj = 0. If min <= 0 <= max: scale = (max - min) / (2^num_bits - 1), min_adj = scale * round(min / scale) and max_adj = max + min_adj - min.  This operation has a gradient and thus allows for training min and max values.

 Args
  inputs   A Tensor of type float32.
  min   A Tensor of type float32.
  max   A Tensor of type float32.
  num_bits   An optional int. Defaults to 8.
  narrow_range   An optional bool. Defaults to False.
  name   A name for the operation (optional).

 Returns   A Tensor of type float32.


tf.quantization.fake_quant_with_min_max_vars_gradient Compute gradients for a FakeQuantWithMinMaxVars operation.  View aliases  Compat aliases for migration
See Migration guide for more details. tf.compat.v1.fake_quant_with_min_max_vars_gradient, tf.compat.v1.quantization.fake_quant_with_min_max_vars_gradient
tf.quantization.fake_quant_with_min_max_vars_gradient(
    gradients, inputs, min, max, num_bits=8, narrow_range=False, name=None
)

 Args
  gradients   A Tensor of type float32. Backpropagated gradients above the FakeQuantWithMinMaxVars operation.
  inputs   A Tensor of type float32. Values passed as inputs to the FakeQuantWithMinMaxVars operation. min, max: Quantization interval, scalar floats.
  min   A Tensor of type float32.
  max   A Tensor of type float32.
  num_bits   An optional int. Defaults to 8. The bitwidth of the quantization; between 2 and 8, inclusive.
  narrow_range   An optional bool. Defaults to False. Whether to quantize into 2^num_bits - 1 distinct values.
  name   A name for the operation (optional).

 Returns   A tuple of Tensor objects (backprops_wrt_input, backprop_wrt_min, backprop_wrt_max).     backprops_wrt_input   A Tensor of type float32.
  backprop_wrt_min   A Tensor of type float32.
  backprop_wrt_max   A Tensor of type float32.


tf.quantization.fake_quant_with_min_max_vars_per_channel Fake-quantize the 'inputs' tensor of type float via per-channel floats  View aliases  Compat aliases for migration
See Migration guide for more details. tf.compat.v1.fake_quant_with_min_max_vars_per_channel, tf.compat.v1.quantization.fake_quant_with_min_max_vars_per_channel
tf.quantization.fake_quant_with_min_max_vars_per_channel(
    inputs, min, max, num_bits=8, narrow_range=False, name=None
)
 Fake-quantize the inputs tensor of type float per-channel and one of the shapes: [d], [b, d] [b, h, w, d] via per-channel floats min and max of shape [d] to outputs tensor of same shape as inputs. Attributes
[min; max] define the clamping range for the inputs data.
inputs values are quantized into the quantization range ( [0; 2^num_bits - 1] when narrow_range is false and [1; 2^num_bits - 1] when it is true) and then de-quantized and output as floats in [min; max] interval.
num_bits is the bitwidth of the quantization; between 2 and 16, inclusive.  Before quantization, min and max values are adjusted with the following logic. It is suggested to have min <= 0 <= max. If 0 is not in the range of values, the behavior can be unexpected:  If 0 < min < max: min_adj = 0 and max_adj = max - min. If min < max < 0: min_adj = min - max and max_adj = 0. If min <= 0 <= max: scale = (max - min) / (2^num_bits - 1), min_adj = scale * round(min / scale) and max_adj = max + min_adj - min.  This operation has a gradient and thus allows for training min and max values.

 Args
  inputs   A Tensor of type float32.
  min   A Tensor of type float32.
  max   A Tensor of type float32.
  num_bits   An optional int. Defaults to 8.
  narrow_range   An optional bool. Defaults to False.
  name   A name for the operation (optional).

 Returns   A Tensor of type float32.


tf.quantization.fake_quant_with_min_max_vars_per_channel_gradient Compute gradients for a FakeQuantWithMinMaxVarsPerChannel operation.  View aliases  Compat aliases for migration
See Migration guide for more details. tf.compat.v1.fake_quant_with_min_max_vars_per_channel_gradient, tf.compat.v1.quantization.fake_quant_with_min_max_vars_per_channel_gradient
tf.quantization.fake_quant_with_min_max_vars_per_channel_gradient(
    gradients, inputs, min, max, num_bits=8, narrow_range=False, name=None
)

 Args
  gradients   A Tensor of type float32. Backpropagated gradients above the FakeQuantWithMinMaxVars operation, shape one of: [d], [b, d], [b, h, w, d].
  inputs   A Tensor of type float32. Values passed as inputs to the FakeQuantWithMinMaxVars operation, shape same as gradients. min, max: Quantization interval, floats of shape [d].
  min   A Tensor of type float32.
  max   A Tensor of type float32.
  num_bits   An optional int. Defaults to 8. The bitwidth of the quantization; between 2 and 16, inclusive.
  narrow_range   An optional bool. Defaults to False. Whether to quantize into 2^num_bits - 1 distinct values.
  name   A name for the operation (optional).

 Returns   A tuple of Tensor objects (backprops_wrt_input, backprop_wrt_min, backprop_wrt_max).     backprops_wrt_input   A Tensor of type float32.
  backprop_wrt_min   A Tensor of type float32.
  backprop_wrt_max   A Tensor of type float32.