Manages saving/restoring trackable values to disk.
TensorFlow objects may contain trackable state, such as tf.Variables, tf.keras.optimizers.Optimizer implementations, tf.data.Dataset iterators, tf.keras.Layer implementations, or tf.keras.Model implementations. These are called trackable objects.
A Checkpoint object can be constructed to save either a single or group of trackable objects to a checkpoint file. It maintains a save_counter for numbering checkpoints.
Checkpoint.save() and Checkpoint.restore() write and read object-based checkpoints, in contrast to TensorFlow 1.x's tf.compat.v1.train.Saver which writes and reads variable.name based checkpoints. Object-based checkpointing saves a graph of dependencies between Python objects (Layers, Optimizers, Variables, etc.) with named edges, and this graph is used to match variables when restoring a checkpoint. It can be more robust to changes in the Python program, and helps to support restore-on-create for variables.
Checkpoint objects have dependencies on the objects passed as keyword arguments to their constructors, and each dependency is given a name that is identical to the name of the keyword argument for which it was created. TensorFlow classes like Layers and Optimizers will automatically add dependencies on their own variables (e.g. "kernel" and "bias" for tf.keras.layers.Dense). Inheriting from tf.keras.Model makes managing dependencies easy in user-defined classes, since Model hooks into attribute assignment. For example:
This Model has a dependency named "input_transform" on its Dense layer, which in turn depends on its variables. As a result, saving an instance of Regress using tf.train.Checkpoint will also save all the variables created by the Dense layer.
When variables are assigned to multiple workers, each worker writes its own section of the checkpoint. These sections are then merged/re-indexed to behave as a single checkpoint. This avoids copying all variables to one worker, but does require that all workers see a common filesystem.
This function differs slightly from the Keras Model save_weights function. tf.keras.Model.save_weights creates a checkpoint file with the name specified in filepath, while tf.train.Checkpoint numbers the checkpoints, using filepath as the prefix for the checkpoint file names. Aside from this, model.save_weights() and tf.train.Checkpoint(model).save() are equivalent.
See the guide to training checkpoints for details.
View source
Reads a training checkpoint written with write.
Reads this Checkpoint and any objects it depends on.
This method is just like restore() but does not expect the save_counter variable in the checkpoint. It only restores the objects that the checkpoint already depends on.
The method is primarily intended for use by higher level checkpoint management utilities that use write() instead of save() and have their own mechanisms to number and track checkpoints.
View source
Restores a training checkpoint.
Restores this Checkpoint and any objects it depends on.
This method is intended to be used to load checkpoints created by save(). For checkpoints created by write() use the read() method which does not expect the save_counter variable added by save().
restore() either assigns values immediately if variables to restore have been created already, or defers restoration until the variables are created. Dependencies added after this call will be matched if they have a corresponding object in the checkpoint (the restore request will queue in any trackable object waiting for the expected dependency to be added).
To ensure that loading is complete and no more assignments will take place, use the assert_consumed() method of the status object returned by restore():
An exception will be raised if any Python objects in the dependency graph were not found in the checkpoint, or if any checkpointed values do not have a matching Python object.
Name-based tf.compat.v1.train.Saver checkpoints from TensorFlow 1.x can be loaded using this method. Names are used to match variables. Re-encode name-based checkpoints using tf.train.Checkpoint.save as soon as possible.
Loading from SavedModel checkpoints
To load values from a SavedModel, just pass the SavedModel directory to checkpoint.restore:
This example calls expect_partial() on the loaded status, since SavedModels saved from Keras often generates extra keys in the checkpoint. Otherwise, the program prints a lot of warnings about unused keys at exit time.
The returned status object has the following methods:
assert_consumed(): Raises an exception if any variables are unmatched: either checkpointed values which don't have a matching Python object or Python objects in the dependency graph with no values in the checkpoint. This method returns the status object, and so may be chained with other assertions.
assert_existing_objects_matched(): Raises an exception if any existing Python objects in the dependency graph are unmatched. Unlike assert_consumed, this assertion will pass if values in the checkpoint have no corresponding Python objects. For example a tf.keras.Layer object which has not yet been built, and so has not created any variables, will pass this assertion but fail assert_consumed. Useful when loading part of a larger checkpoint into a new Python program, e.g. a training checkpoint with a tf.compat.v1.train.Optimizer was saved but only the state required for inference is being loaded. This method returns the status object, and so may be chained with other assertions.
assert_nontrivial_match(): Asserts that something aside from the root object was matched. This is a very weak assertion, but is useful for sanity checking in library code where objects may exist in the checkpoint which haven't been created in Python and some Python objects may not have a checkpointed value.
expect_partial(): Silence warnings about incomplete checkpoint restores. Warnings are otherwise printed for unused parts of the checkpoint file or object when the Checkpoint object is deleted (often at program shutdown).
View source
Saves a training checkpoint and provides basic checkpoint management.
The saved checkpoint includes variables created by this object and any trackable objects it depends on at the time Checkpoint.save() is called.
save is a basic convenience wrapper around the write method, sequentially numbering checkpoints using save_counter and updating the metadata used by tf.train.latest_checkpoint. More advanced checkpoint management, for example garbage collection and custom numbering, may be provided by other utilities which also wrap write and read. (tf.train.CheckpointManager for example).
View source
Writes a training checkpoint.
The checkpoint includes variables created by this object and any trackable objects it depends on at the time Checkpoint.write() is called.
write does not number checkpoints, increment save_counter, or update the metadata used by tf.train.latest_checkpoint. It is primarily intended for use by higher level checkpoint management utilities. save provides a very basic implementation of these features.
Checkpoints written with write must be read with read.


Manages multiple checkpoints by keeping some and deleting unneeded ones.
Compat aliases for migration
See Migration guide for more details.
tf.compat.v1.train.CheckpointManager
CheckpointManager preserves its own state across instantiations (see the __init__ documentation for details). Only one should be active in a particular directory at a time.
Note that checkpoints saved due to keep_checkpoint_every_n_hours will not show up in this list (to avoid ever-growing filename lists).
Equivalent to tf.train.latest_checkpoint(directory) where directory is the constructor argument to CheckpointManager.
Suitable for passing to tf.train.Checkpoint.restore to resume training.
View source
Restore items in checkpoint from the latest checkpoint file.
This method will first try to restore from the most recent checkpoint in directory. If no checkpoints exist in directory, and init_fn is specified, this method will call init_fn to do customized initialization. This can be used to support initialization from pretrained models.
Note that unlike tf.train.Checkpoint.restore(), this method doesn't return a load status object that users can run assertions on (e.g. assert_consumed()). Thus to run assertions, users should directly use tf.train.Checkpoint.restore() method.
View source
Creates a new checkpoint and manages it.


Options for constructing a Checkpoint.
Compat aliases for migration
See Migration guide for more details.
tf.compat.v1.train.CheckpointOptions
Used as the _options argument to the tf.Checkpoint constructor to adjust how variables are saved.
Example: Run IO ops on "localhost" while saving a checkpoint:
This is for example useful if you want to save to a local directory, such as "/tmp" when running in a distributed setting. In that case pass a device for the host where the "/tmp" directory is accessible.


Continuously yield new checkpoint files as they appear.
Compat aliases for migration
See Migration guide for more details.
tf.compat.v1.train.checkpoints_iterator
The iterator only checks for new checkpoints when control flow has been reverted to it. This means it can miss checkpoints if your code takes longer to run between iterations than min_interval_secs or the interval at which new checkpoints are written.
The timeout argument is the maximum number of seconds to block waiting for a new checkpoint. It is used in combination with the timeout_fn as follows:
This behavior gives control to callers on what to do if checkpoints do not come fast enough or stop being generated. For example, if callers have a way to detect that the training has stopped and know that no new checkpoints will be generated, they can provide a timeout_fn that returns True when the training has stopped. If they know that the training is still going on they return False instead.


A ProtocolMessage
Compat aliases for migration
See Migration guide for more details.
tf.compat.v1.train.ClusterDef


Represents a cluster as a set of "tasks", organized into "jobs".
Compat aliases for migration
See Migration guide for more details.
tf.compat.v1.train.ClusterSpec
A tf.train.ClusterSpec represents the set of processes that participate in a distributed TensorFlow computation. Every tf.distribute.Server is constructed in a particular cluster.
To create a cluster with two jobs and five tasks, you specify the mapping from job names to lists of network addresses (typically hostname-port pairs).
Each job may also be specified as a sparse mapping from task indices to network addresses. This enables a server to be configured without needing to know the identity of (for example) all other worker tasks:
View source
Returns a tf.train.ClusterDef protocol buffer based on this cluster.
View source
Returns a dictionary from job names to their tasks.
For each job, if the task index space is dense, the corresponding value will be a list of network addresses; otherwise it will be a dictionary mapping (sparse) task indices to the corresponding addresses.
View source
Returns a mapping from task ID to address in the given job.
View source
Returns the number of tasks defined in the given job.
View source
Returns the address of the given task in the given job.
View source
Returns a list of valid task indices in the given job.
View source
View source
Return self==value.
View source
Return self!=value.
View source


A coordinator for threads.
Compat aliases for migration
See Migration guide for more details.
tf.compat.v1.train.Coordinator
This class implements a simple mechanism to coordinate the termination of a set of threads.
Any of the threads can call coord.request_stop() to ask for all the threads to stop. To cooperate with the requests, each thread must check for coord.should_stop() on a regular basis. coord.should_stop() returns True as soon as coord.request_stop() has been called.
A typical thread running with a coordinator will do something like:
A thread can report an exception to the coordinator as part of the request_stop() call. The exception will be re-raised from the coord.join() call.
To simplify the thread implementation, the Coordinator provides a context handler stop_on_exception() that automatically requests a stop if an exception is raised. Using the context handler the thread code above can be written as:
After a thread has called coord.request_stop() the other threads have a fixed time to stop, this is called the 'stop grace period' and defaults to 2 minutes. If any of the threads is still alive after the grace period expires coord.join() raises a RuntimeError reporting the laggards.
View source
Clears the stop flag.
After this is called, calls to should_stop() will return False.
View source
Wait for threads to terminate.
This call blocks until a set of threads have terminated. The set of thread is the union of the threads passed in the threads argument and the list of threads that registered with the coordinator by calling Coordinator.register_thread().
After the threads stop, if an exc_info was passed to request_stop, that exception is re-raised.
Grace period handling: When request_stop() is called, threads are given 'stop_grace_period_secs' seconds to terminate. If any of them is still alive after that period expires, a RuntimeError is raised. Note that if an exc_info was passed to request_stop() then it is raised instead of that RuntimeError.
View source
If an exception has been passed to request_stop, this raises it.
View source
Register a thread to join.
View source
Request that the threads stop.
After this is called, calls to should_stop() will return True.
View source
Check if stop was requested.
View source
Context manager to request stop when an Exception is raised.
Code that uses a coordinator must catch exceptions and pass them to the request_stop() method to stop the other threads managed by the coordinator.
This context handler simplifies the exception handling. Use it as follows:
This is completely equivalent to the slightly longer code:
View source
Wait till the Coordinator is told to stop.


A ProtocolMessage
Compat aliases for migration
See Migration guide for more details.
tf.compat.v1.train.Example


Public API for tf.train.experimental namespace.
class DynamicLossScale: Loss scale that dynamically adjusts itself.
class FixedLossScale: Loss scale with a fixed value.
class LossScale: Base class for all TF1 loss scales.
class PythonState: A mixin for putting Python state in an object-based checkpoint.
disable_mixed_precision_graph_rewrite(...): Disables the mixed precision graph rewrite. (deprecated)
enable_mixed_precision_graph_rewrite(...): Enable mixed precision via a graph rewrite. (deprecated)


Disables the mixed precision graph rewrite. (deprecated)
After this is called, the mixed precision graph rewrite will no longer run for tf.functions, and so float32 operations will no longer be converted to float16.
This does not undo the effects of loss scaling. Any optimizers wrapped with a LossScaleOptimizer will continue to do loss scaling, although this loss scaling will no longer be useful, as the graph rewrite no longer converts tf.functions to use float16.
This function is useful for unit testing. A unit test can test using the mixed precision graph rewrite, then disable it so future unit tests continue using float32.


Enable mixed precision via a graph rewrite. (deprecated)
Mixed precision is the use of both float32 and float16 data types when training a model to improve performance. This is achieved via a graph rewrite operation and a loss-scale optimizer.
Performing arithmetic operations in float16 takes advantage of specialized processing units, such as NVIDIA Tensor Cores, for much higher arithmetic throughput. However, due to the smaller representable range, performing the entire training with float16 can result in gradient underflow, that is, small gradient values becoming zeroes. Instead, performing only select arithmetic operations in float16 results in higher throughput and decreased training time when using compatible hardware accelerators while also reducing memory usage, typically without sacrificing model accuracy.
Calling enable_mixed_precision_graph_rewrite(opt) enables the graph rewrite operation before computing gradients. The function additionally returns an Optimizer (opt) wrapped with a LossScaleOptimizer. This prevents underflow in the float16 tensors during the backward pass. An optimizer of type tf.keras.optimizers.Optimizer or tf.compat.v1.train.Optimizer must be passed to this function, which will then be wrapped to use loss scaling.
The graph rewrite operation changes the dtype of certain operations in the graph from float32 to float16. There are several categories of operations that are either included or excluded by this rewrite operation. The following categories of Ops are defined inside corresponding functions under the class AutoMixedPrecisionLists in  auto_mixed_precision_lists.h:
When this function is used, gradients should be computed and applied with the returned optimizer, either by calling opt.minimize() or opt.compute_gradients() followed by opt.apply_gradients(). If gradients are instead computed with tf.gradients or tf.GradientTape, loss scaling will not be applied, which will likely cause your model not to converge due to float16 underflow problems. To apply lossing scaling with tf.gradients or tf.GradientTape, LossScaleOptimizer.get_scaled_loss and LossScaleOptimizer.get_unscaled_gradients. See keras.mixed_precision.experimental.LossScaleOptimizer for details how to do this.
When eager execution is enabled, the mixed precision graph rewrite is only enabled within tf.functions, as outside tf.functions, there is no graph.
For NVIDIA GPUs with Tensor cores, as a general performance guide, dimensions (such as batch size, input size, output size, and channel counts) should be powers of two if under 256, or otherwise divisible by 8 if above 256. For more information, check out the NVIDIA Deep Learning Performance Guide.
Currently, mixed precision is only enabled on NVIDIA Tensor Core GPUs with Compute Capability 7.0 and above (Volta, Turing, or newer architectures). The parts of the graph on CPUs and TPUs are untouched by the graph rewrite.
Both this function and the Keras mixed precision API enable the use of mixed precision in a model. Therefore, only one of the two APIs can be used. We recommend using the Keras mixed precision API, as it is more customizable and supports Eager execution. However, it only supports models which use Keras layers, while the graph rewrite works in any model that uses tf.functions.
The core difference between the two APIs is that this function is a graph rewrite, and so it changes the graph to use mixed precision under the hood. You still build your graph in float32, and the graph rewrite will change certain ops to float16. The Keras mixed precision API directly builds the Keras Model using a mix of float16 and float32.
One core advantage of the Keras API is it supports mixed precision with Eager execution, i.e. mixed precision outside tf.functions. The graph rewrite will only affect ops within tf.functions, making it harder to debug if issues occur with mixed precision. The Keras API is also more customizable, as you can override any layer to run in float32 by passing dtype="float32" to the layer constructor. Additionally, you can query the dtype of tensors in the model by checking tensor.dtype. With the graph rewrite, all tensors appear to be float32 since the dtype is only changed under the hood.
The main advantage of the graph rewrite (this function) is that it works even if you do not use Keras layers or any other part of Keras. The Keras mixed precision API requires models which use Keras layers, as it only inserts casts inside Keras layers and models. Another advantage is that the graph rewrite never results in a TypeError, which the Keras API may introduce if you do certain operations outside Keras. For example, the following will result in a TypeError if the Keras mixed precision API is enabled, as a float16 and float32 tensor will be added: tf.keras.layers.Dense(2)(x) + tf.keras.layers.Dense(2, dtype="float32")(x)


A mixin for putting Python state in an object-based checkpoint.
Compat aliases for migration
See Migration guide for more details.
tf.compat.v1.train.experimental.PythonState
This is an abstract class which allows extensions to TensorFlow's object-based checkpointing (see tf.train.Checkpoint). For example a wrapper for NumPy arrays:
Instances of NumpyWrapper are checkpointable objects, and will be saved and restored from checkpoints along with TensorFlow state like variables.
View source
Callback to deserialize the object.
View source
Callback to serialize the object. Returns a string.


Maintains moving averages of variables by employing an exponential decay.
Compat aliases for migration
See Migration guide for more details.
tf.compat.v1.train.ExponentialMovingAverage
When training a model, it is often beneficial to maintain moving averages of the trained parameters. Evaluations that use averaged parameters sometimes produce significantly better results than the final trained values.
The apply() method adds shadow copies of trained variables and add ops that maintain a moving average of the trained variables in their shadow copies. It is used when building the training model. The ops that maintain moving averages are typically run after each training step. The average() and average_name() methods give access to the shadow variables and their names. They are useful when building an evaluation model, or when restoring a model from a checkpoint file. They help use the moving averages in place of the last trained values for evaluations.
The moving averages are computed using exponential decay. You specify the decay value when creating the ExponentialMovingAverage object. The shadow variables are initialized with the same initial values as the trained variables. When you run the ops to maintain the moving averages, each shadow variable is updated with the formula:
shadow_variable -= (1 - decay) * (shadow_variable - variable)
This is mathematically equivalent to the classic formula below, but the use of an assign_sub op (the "-=" in the formula) allows concurrent lockless updates to the variables:
shadow_variable = decay * shadow_variable + (1 - decay) * variable
Reasonable values for decay are close to 1.0, typically in the multiple-nines range: 0.999, 0.9999, etc.
Example usage when creating a training model:
There are two ways to use the moving averages for evaluations:
Example of restoring the shadow variable values:
View source
Maintains moving averages of variables.
var_list must be a list of Variable or Tensor objects. This method creates shadow variables for all elements of var_list. Shadow variables for Variable objects are initialized to the variable's initial value. They will be added to the GraphKeys.MOVING_AVERAGE_VARIABLES collection. For Tensor objects, the shadow variables are initialized to 0 and zero debiased (see docstring in assign_moving_average for more details).
shadow variables are created with trainable=False and added to the GraphKeys.ALL_VARIABLES collection. They will be returned by calls to tf.compat.v1.global_variables().
Returns an op that updates all shadow variables from the current value of their associated variables.
Note that apply() can be called multiple times. When eager execution is enabled each call to apply will update the variables once, so this needs to be called in a loop.
View source
Returns the Variable holding the average of var.
View source
Returns the name of the Variable holding the average for var.
The typical scenario for ExponentialMovingAverage is to compute moving averages of variables during training, and restore the variables from the computed moving averages during evaluations.
To restore variables, you have to know the name of the shadow variables. That name and the original variable can then be passed to a Saver() object to restore the variable from the moving average value with: saver = tf.compat.v1.train.Saver({ema.average_name(var): var})
average_name() can be called whether or not apply() has been called.
View source
Returns a map of names to Variables to restore.
If a variable has a moving average, use the moving average variable name as the restore name; otherwise, use the variable name.
For example,
Below is an example of such mapping:


A ProtocolMessage
Compat aliases for migration
See Migration guide for more details.
tf.compat.v1.train.Feature


A ProtocolMessage
Compat aliases for migration
See Migration guide for more details.
tf.compat.v1.train.FeatureList


A ProtocolMessage
Compat aliases for migration
See Migration guide for more details.
tf.compat.v1.train.FeatureLists
class FeatureListEntry


A ProtocolMessage
Compat aliases for migration
See Migration guide for more details.
tf.compat.v1.train.FeatureLists.FeatureListEntry


A ProtocolMessage
Compat aliases for migration
See Migration guide for more details.
tf.compat.v1.train.Features
class FeatureEntry


A ProtocolMessage
Compat aliases for migration
See Migration guide for more details.
tf.compat.v1.train.Features.FeatureEntry


A ProtocolMessage
Compat aliases for migration
See Migration guide for more details.
tf.compat.v1.train.FloatList


Returns CheckpointState proto from the "checkpoint" file.
Compat aliases for migration
See Migration guide for more details.
tf.compat.v1.train.get_checkpoint_state
If the "checkpoint" file contains a valid CheckpointState proto, returns it.


A ProtocolMessage
Compat aliases for migration
See Migration guide for more details.
tf.compat.v1.train.Int64List


A ProtocolMessage
Compat aliases for migration
See Migration guide for more details.
tf.compat.v1.train.JobDef
class TasksEntry


A ProtocolMessage
Compat aliases for migration
See Migration guide for more details.
tf.compat.v1.train.JobDef.TasksEntry


Finds the filename of latest saved checkpoint file.
Compat aliases for migration
See Migration guide for more details.
tf.compat.v1.train.latest_checkpoint
Gets the checkpoint state given the provided checkpoint_dir and looks for a corresponding TensorFlow 2 (preferred) or TensorFlow 1.x checkpoint path. The latest_filename argument is only applicable if you are saving checkpoint using v1.train.Saver.save
See the Training Checkpoints Guide for more details and examples.`


Lists the checkpoint keys and shapes of variables in a checkpoint.
Compat aliases for migration
See Migration guide for more details.
tf.compat.v1.train.list_variables
Checkpoint keys are paths in a checkpoint graph.


Returns CheckpointReader for checkpoint found in ckpt_dir_or_file.
Compat aliases for migration
See Migration guide for more details.
tf.compat.v1.train.load_checkpoint
If ckpt_dir_or_file resolves to a directory with multiple checkpoints, reader for the latest checkpoint is returned.


Returns the tensor value of the given variable in the checkpoint.
Compat aliases for migration
See Migration guide for more details.
tf.compat.v1.train.load_variable


A ProtocolMessage
Compat aliases for migration
See Migration guide for more details.
tf.compat.v1.train.SequenceExample


A ProtocolMessage
Compat aliases for migration
See Migration guide for more details.
tf.compat.v1.train.ServerDef


Transposes a, where a is a Tensor.
Permutes the dimensions according to the value of perm.
The returned tensor's dimension i will correspond to the input dimension perm[i]. If perm is not given, it is set to (n-1...0), where n is the rank of the input tensor. Hence by default, this operation performs a regular matrix transpose on 2-D input Tensors.
If conjugate is True and a.dtype is either complex64 or complex128 then the values of a are conjugated and transposed.
Equivalently, you could call tf.transpose(x, perm=[1, 0]).
If x is complex, setting conjugate=True gives the conjugate transpose:
'perm' is more useful for n-dimensional tensors where n > 2:
As above, simply calling tf.transpose will default to perm=[2,1,0].
To take the transpose of the matrices in dimension-0 (such as when you are transposing matrices where 0 is the batch dimesnion), you would set perm=[0,2,1].
In numpy transposes are memory-efficient constant time operations as they simply return a new view of the same data with adjusted strides.
TensorFlow does not support strides, so transpose returns a new tensor with the items permuted.


Returns x / y element-wise for integer types.
Compat aliases for migration
See Migration guide for more details.
tf.compat.v1.truncatediv
Truncation designates that negative numbers will round fractional quantities toward zero. I.e. -7 / 5 = -1. This matches C semantics but it is different than Python semantics. See FloorDiv for a division function that matches Python Semantics.


Returns element-wise remainder of division. This emulates C semantics in that
Compat aliases for migration
See Migration guide for more details.
tf.compat.v1.truncatemod
the result here is consistent with a truncating divide. E.g. truncate(x / y) * y + truncate_mod(x, y) = x.


Group tensors together.
This creates a tuple of tensors with the same values as the tensors argument, except that the value of each tensor is only returned after the values of all tensors have been computed.
control_inputs contains additional ops that have to finish before this op finishes, but whose outputs are not returned.
This can be used as a "join" mechanism for parallel computations: all the argument tensors can be computed in parallel, but the values of any tensor returned by tuple are only available after all the parallel computations are done.
See also tf.group and tf.control_dependencies.


Public TensorFlow type definitions.
For details, see https://github.com/tensorflow/community/blob/master/rfcs/20200211-tf-types.md
experimental module: Public API for tf.types.experimental namespace.


Public API for tf.types.experimental namespace.
TensorLike: Union of all types that can be converted to a tf.Tensor by tf.convert_to_tensor.


This symbol is a type alias.
Union of all types that can be converted to a tf.Tensor by tf.convert_to_tensor.
This definition may be used in user code. Additional types may be added in the future as more input types are supported.
This definition passes static type verification for:


Specifies a TensorFlow value type.
Compat aliases for migration
See Migration guide for more details.
tf.compat.v1.TypeSpec, tf.compat.v1.data.experimental.Structure
A tf.TypeSpec provides metadata describing an object accepted or returned by TensorFlow APIs. Concrete subclasses, such as tf.TensorSpec and tf.RaggedTensorSpec, are used to describe different value types.
For example, tf.function's input_signature argument accepts a list (or nested structure) of TypeSpecs.
Creating new subclasses of TypeSpec (outside of TensorFlow core) is not currently supported. In particular, we may make breaking changes to the private methods and properties defined by this base class.
In particular, all values that are compatible with this TypeSpec must be an instance of this type.
View source
Returns true if spec_or_value is compatible with this TypeSpec.
View source
Returns the most specific TypeSpec compatible with self and other.
View source
Return self==value.
View source
Return self!=value.


Returns a tf.TypeSpec that represents the given value.
Compat aliases for migration
See Migration guide for more details.
tf.compat.v1.type_spec_from_value


Controls how gradient computation behaves when y does not depend on x.
Compat aliases for migration
See Migration guide for more details.
tf.compat.v1.UnconnectedGradients
The gradient of y with respect to x can be zero in two different ways: there could be no differentiable path in the graph connecting x to y (and so we can statically prove that the gradient is zero) or it could be that runtime values of tensors in a particular execution lead to a gradient of zero (say, if a relu unit happens to not be activated). To allow you to distinguish between these two cases you can choose what value gets returned for the gradient when there is no path in the graph from x to y:


Finds unique elements in a 1-D tensor.
Compat aliases for migration
See Migration guide for more details.
tf.compat.v1.unique
This operation returns a tensor y containing all of the unique elements of x sorted in the same order that they occur in x; x does not need to be sorted. This operation also returns a tensor idx the same size as x that contains the index of each value of x in the unique output y. In other words:
y[idx[i]] = x[i] for i in [0, 1,...,rank(x) - 1]


Finds unique elements in a 1-D tensor.
Compat aliases for migration
See Migration guide for more details.
tf.compat.v1.unique_with_counts
This operation returns a tensor y containing all of the unique elements of x sorted in the same order that they occur in x. This operation also returns a tensor idx the same size as x that contains the index of each value of x in the unique output y. Finally, it returns a third tensor count that contains the count of each element of y in x. In other words:
y[idx[i]] = x[i] for i in [0, 1,...,rank(x) - 1]


Converts an array of flat indices into a tuple of coordinate arrays.
Compat aliases for migration
See Migration guide for more details.
tf.compat.v1.unravel_index
Equivalent to np.unravel_index


Unpacks the given dimension of a rank-R tensor into rank-(R-1) tensors.
Compat aliases for migration
See Migration guide for more details.
tf.compat.v1.unstack
Unpacks num tensors from value by chipping it along the axis dimension. If num is not specified (the default), it is inferred from value's shape. If value.shape[axis] is not known, ValueError is raised.
For example, given a tensor of shape (A, B, C, D);
If axis == 0 then the i'th tensor in output is the slice value[i, :, :, :] and each tensor in output will have shape (B, C, D). (Note that the dimension unpacked along is gone, unlike split).
If axis == 1 then the i'th tensor in output is the slice value[:, i, :, :] and each tensor in output will have shape (A, C, D). Etc.
This is the opposite of stack.


See the variable guide.
A variable maintains shared, persistent state manipulated by a program.
The Variable() constructor requires an initial value for the variable, which can be a Tensor of any type and shape. This initial value defines the type and shape of the variable. After construction, the type and shape of the variable are fixed. The value can be changed using one of the assign methods.
The shape argument to Variable's constructor allows you to construct a variable with a less defined shape than its initial_value:
Just like any Tensor, variables created with Variable() can be used as inputs to operations. Additionally, all the operators overloaded for the Tensor class are carried over to variables.
When building a machine learning model it is often convenient to distinguish between variables holding trainable model parameters and other variables such as a step variable used to count training steps. To make this easier, the variable constructor supports a trainable=<bool> parameter. tf.GradientTape watches trainable variables by default:
Variables are automatically tracked when assigned to attributes of types inheriting from tf.Module.
This tracking then allows saving variable values to training checkpoints, or to SavedModels which include serialized TensorFlow graphs.
Variables are often captured and manipulated by tf.functions. This works the same way the un-decorated function would have:
Variables created inside a tf.function must be owned outside the function and be created only once:
See the tf.function documentation for details.
Note that this is different from initialized_value() which runs the op that initializes the variable before returning its value. This method returns the tensor that is used by the op that initializes the variable.
class SaveSliceInfo
View source
Assigns a new value to the variable.
This is essentially a shortcut for assign(self, value).
View source
Adds a value to this variable.
This is essentially a shortcut for assign_add(self, delta).
View source
Subtracts a value from this variable.
This is essentially a shortcut for assign_sub(self, delta).
View source
Assigns tf.IndexedSlices to this variable batch-wise.
Analogous to batch_gather. This assumes that this variable and the sparse_delta IndexedSlices have a series of leading dimensions that are the same for all of them, and the updates are performed on the last dimension of indices. In other words, the dimensions should be the following:
num_prefix_dims = sparse_delta.indices.ndims - 1 batch_dim = num_prefix_dims + 1 sparse_delta.updates.shape = sparse_delta.indices.shape + var.shape[ batch_dim:]
where
sparse_delta.updates.shape[:num_prefix_dims] == sparse_delta.indices.shape[:num_prefix_dims] == var.shape[:num_prefix_dims]
And the operation performed can be expressed as:
var[i_1, ..., i_n, sparse_delta.indices[i_1, ..., i_n, j]] = sparse_delta.updates[ i_1, ..., i_n, j]
When sparse_delta.indices is a 1D tensor, this operation is equivalent to scatter_update.
To avoid this operation one can looping over the first ndims of the variable and using scatter_update on the subtensors that result of slicing the first dimension. This is a valid option for ndims = 1, but less efficient than this implementation.
View source
Increments this variable until it reaches limit. (deprecated)
When that Op is run it tries to increment the variable by 1. If incrementing the variable would bring it above limit then the Op raises the exception OutOfRangeError.
If no error is raised, the Op outputs the value of the variable before the increment.
This is essentially a shortcut for count_up_to(self, limit).
View source
In a session, computes and returns the value of this variable.
This is not a graph construction method, it does not add ops to the graph.
This convenience method requires a session where the graph containing this variable has been launched. If no session is passed, the default session is used. See tf.compat.v1.Session for more information on launching a graph and on sessions.
View source
DEPRECATED FUNCTION
View source
Returns a Variable object created from variable_def.
View source
Gather slices from params into a Tensor with shape specified by indices.
See tf.gather_nd for details.
View source
Alias of Variable.shape.
View source
Returns the value of the initialized variable. (deprecated)
You should use this instead of the variable itself to initialize another variable with a value that depends on the value of this variable.
View source
Load new value into this variable. (deprecated)
Writes new value to variable's memory. Doesn't add ops to the graph.
This convenience method requires a session where the graph containing this variable has been launched. If no session is passed, the default session is used. See tf.compat.v1.Session for more information on launching a graph and on sessions.
View source
Returns the value of this variable, read in the current context.
Can be different from value() if it's on another device, with control dependencies, etc.
View source
Returns a hashable reference object to this Variable.
The primary use case for this API is to put variables in a set/dictionary. We can't put variables in a set/dictionary as variable.__hash__() is no longer available starting Tensorflow 2.0.
The following will raise an exception starting 2.0
Instead, we can use variable.ref().
Also, the reference object provides .deref() function that returns the original Variable.
View source
Adds tf.IndexedSlices to this variable.
View source
Divide this variable by tf.IndexedSlices.
View source
Updates this variable with the max of tf.IndexedSlices and itself.
View source
Updates this variable with the min of tf.IndexedSlices and itself.
View source
Multiply this variable by tf.IndexedSlices.
View source
Applies sparse addition to individual values or slices in a Variable.
The Variable has rank P and indices is a Tensor of rank Q.
indices must be integer tensor, containing indices into self. It must be shape [d_0, ..., d_{Q-2}, K] where 0 < K <= P.
The innermost dimension of indices (with length K) corresponds to indices into elements (if K = P) or slices (if K < P) along the Kth dimension of self.
updates is Tensor of rank Q-1+P-K with shape:
For example, say we want to add 4 scattered elements to a rank-1 tensor to 8 elements. In Python, that update would look like this:
The resulting update to v would look like this:
See tf.scatter_nd for more details about how to make updates to slices.
View source
Applies sparse subtraction to individual values or slices in a Variable.
Assuming the variable has rank P and indices is a Tensor of rank Q.
indices must be integer tensor, containing indices into self. It must be shape [d_0, ..., d_{Q-2}, K] where 0 < K <= P.
The innermost dimension of indices (with length K) corresponds to indices into elements (if K = P) or slices (if K < P) along the Kth dimension of self.
updates is Tensor of rank Q-1+P-K with shape:
For example, say we want to add 4 scattered elements to a rank-1 tensor to 8 elements. In Python, that update would look like this:
The resulting update to v would look like this:
See tf.scatter_nd for more details about how to make updates to slices.
View source
Applies sparse assignment to individual values or slices in a Variable.
The Variable has rank P and indices is a Tensor of rank Q.
indices must be integer tensor, containing indices into self. It must be shape [d_0, ..., d_{Q-2}, K] where 0 < K <= P.
The innermost dimension of indices (with length K) corresponds to indices into elements (if K = P) or slices (if K < P) along the Kth dimension of self.
updates is Tensor of rank Q-1+P-K with shape:
For example, say we want to add 4 scattered elements to a rank-1 tensor to 8 elements. In Python, that update would look like this:
The resulting update to v would look like this:
See tf.scatter_nd for more details about how to make updates to slices.
View source
Subtracts tf.IndexedSlices from this variable.
View source
Assigns tf.IndexedSlices to this variable.
View source
Overrides the shape for this variable.
View source
Gather slices from params axis axis according to indices.
This function supports a subset of tf.gather, see tf.gather for details on usage.
View source
Converts a Variable to a VariableDef protocol buffer.
View source
Returns the last snapshot of this variable.
You usually do not need to call this method as all ops that need the value of the variable call it automatically through a convert_to_tensor() call.
Returns a Tensor which holds the value of the variable. You can not assign a new value to this tensor as it is not a reference to the variable.
To avoid copies, if the consumer of the returned value is on the same device as the variable, this actually returns the live value of the variable, not a copy. Updates to the variable are seen by the consumer. If the consumer is on a different device it will get a copy of the variable.
View source
Computes the absolute value of a tensor.
Given a tensor of integer or floating-point values, this operation returns a tensor of the same type, where each element contains the absolute value of the corresponding element in the input.
Given a tensor x of complex numbers, this operation returns a tensor of type float32 or float64 that is the absolute value of each element in x. For a complex number \(a + bj\), its absolute value is computed as \(\sqrt{a^2 + b^2}\).
View source
The operation invoked by the Tensor.add operator.
Purpose in the API:
View source
View source
Divides x / y elementwise (using Python 2 division operator semantics). (deprecated)
This function divides x and y, forcing Python 2 semantics. That is, if x and y are both integers then the result will be an integer. This is in contrast to Python 3, where division with / is always a float while division with // is always an integer.
View source
Compares two variables element-wise for equality.
View source
Divides x / y elementwise, rounding toward the most negative integer.
The same as tf.compat.v1.div(x,y) for integers, but uses tf.floor(tf.compat.v1.div(x,y)) for floating point arguments so that the result is always an integer (though possibly an integer represented as floating point). This op is generated by x // y floor division in Python 3 and in Python 2.7 with from __future__ import division.
x and y must have the same type, and the result will have the same type as well.
Returns the truth value of (x >= y) element-wise.
View source
Creates a slice helper object given a variable.
This allows creating a sub-tensor from part of the current contents of a variable. See tf.Tensor.getitem for detailed examples of slicing.
This function in addition also allows assignment to a sliced range. This is similar to __setitem__ functionality in Python. However, the syntax is different so that the user can capture the assignment operation for grouping or passing to sess.run(). For example,
Note that assignments currently do not support NumPy broadcasting semantics.
Returns the truth value of (x > y) element-wise.
View source
View source
Dummy method to prevent iteration.
Do not call.
NOTE(mrry): If we register getitem as an overloaded operator, Python will valiantly attempt to iterate over the variable's Tensor from 0 to infinity. Declaring this method prevents this unintended behavior.
Returns the truth value of (x <= y) element-wise.
Returns the truth value of (x < y) element-wise.
View source
Multiplies matrix a by matrix b, producing a * b.
The inputs must, following any transpositions, be tensors of rank >= 2 where the inner 2 dimensions specify valid matrix multiplication dimensions, and any further outer dimensions specify matching batch size.
Both matrices must be of the same type. The supported types are: float16, float32, float64, int32, complex64, complex128.
Either matrix can be transposed or adjointed (conjugated and transposed) on the fly by setting one of the corresponding flag to True. These are False by default.
If one or both of the matrices contain a lot of zeros, a more efficient multiplication algorithm can be used by setting the corresponding a_is_sparse or b_is_sparse flag to True. These are False by default. This optimization is only available for plain matrices (rank-2 tensors) with datatypes bfloat16 or float32.
A simple 2-D tensor matrix multiplication:
A batch matrix multiplication with batch shape [2]:
Since python >= 3.5 the @ operator is supported (see PEP 465). In TensorFlow, it simply calls the tf.matmul() function, so the following lines are equivalent:
output[..., i, j] = sum_k (a[..., i, k] * b[..., k, j]), for all indices i, j.
View source
Returns element-wise remainder of division. When x < 0 xor y < 0 is
true, this follows Python semantics in that the result here is consistent with a flooring divide. E.g. floor(x / y) * y + mod(x, y) = x.
View source
Dispatches cwise mul for "DenseDense" and "DenseSparse".
View source
Compares two variables element-wise for equality.
Computes numerical negative value element-wise.
I.e., \(y = -x\).
View source
View source
Computes the power of one value to another.
Given a tensor x and a tensor y, this operation computes \(x^y\) for corresponding elements in x and y. For example:
View source
The operation invoked by the Tensor.add operator.
Purpose in the API:
View source
View source
Divides x / y elementwise (using Python 2 division operator semantics). (deprecated)
This function divides x and y, forcing Python 2 semantics. That is, if x and y are both integers then the result will be an integer. This is in contrast to Python 3, where division with / is always a float while division with // is always an integer.
View source
Divides x / y elementwise, rounding toward the most negative integer.
The same as tf.compat.v1.div(x,y) for integers, but uses tf.floor(tf.compat.v1.div(x,y)) for floating point arguments so that the result is always an integer (though possibly an integer represented as floating point). This op is generated by x // y floor division in Python 3 and in Python 2.7 with from __future__ import division.
x and y must have the same type, and the result will have the same type as well.
View source
Multiplies matrix a by matrix b, producing a * b.
The inputs must, following any transpositions, be tensors of rank >= 2 where the inner 2 dimensions specify valid matrix multiplication dimensions, and any further outer dimensions specify matching batch size.
Both matrices must be of the same type. The supported types are: float16, float32, float64, int32, complex64, complex128.
Either matrix can be transposed or adjointed (conjugated and transposed) on the fly by setting one of the corresponding flag to True. These are False by default.
If one or both of the matrices contain a lot of zeros, a more efficient multiplication algorithm can be used by setting the corresponding a_is_sparse or b_is_sparse flag to True. These are False by default. This optimization is only available for plain matrices (rank-2 tensors) with datatypes bfloat16 or float32.
A simple 2-D tensor matrix multiplication:
A batch matrix multiplication with batch shape [2]:
Since python >= 3.5 the @ operator is supported (see PEP 465). In TensorFlow, it simply calls the tf.matmul() function, so the following lines are equivalent:
output[..., i, j] = sum_k (a[..., i, k] * b[..., k, j]), for all indices i, j.
View source
Returns element-wise remainder of division. When x < 0 xor y < 0 is
true, this follows Python semantics in that the result here is consistent with a flooring divide. E.g. floor(x / y) * y + mod(x, y) = x.
View source
Dispatches cwise mul for "DenseDense" and "DenseSparse".
View source
View source
Computes the power of one value to another.
Given a tensor x and a tensor y, this operation computes \(x^y\) for corresponding elements in x and y. For example:
View source
Returns x - y element-wise.
View source
Divides x / y elementwise (using Python 3 division operator semantics).
This function forces Python 3 division operator semantics where all integer arguments are cast to floating types first. This op is generated by normal x / y division in Python 3 and in Python 2.7 with from __future__ import division. If you want integer division that rounds down, use x // y or tf.math.floordiv.
x and y must have the same numeric type. If the inputs are floating point, the output will have the same type. If the inputs are integral, the inputs are cast to float32 for int8 and int16 and float64 for int32 and int64 (matching the behavior of Numpy).
View source
View source
Returns x - y element-wise.
View source
Divides x / y elementwise (using Python 3 division operator semantics).
This function forces Python 3 division operator semantics where all integer arguments are cast to floating types first. This op is generated by normal x / y division in Python 3 and in Python 2.7 with from __future__ import division. If you want integer division that rounds down, use x // y or tf.math.floordiv.
x and y must have the same numeric type. If the inputs are floating point, the output will have the same type. If the inputs are integral, the inputs are cast to float32 for int8 and int16 and float64 for int32 and int64 (matching the behavior of Numpy).
View source


Information on how to save this Variable as a slice.
Compat aliases for migration
See Migration guide for more details.
tf.compat.v1.Variable.SaveSliceInfo
Provides internal support for saving variables as slices of a larger variable. This API is not public and is subject to change.
View source
Returns a SaveSliceInfoDef() proto.


Indicates how a distributed variable will be aggregated.
tf.distribute.Strategy distributes a model by making multiple copies (called "replicas") acting data-parallel on different elements of the input batch. When performing some variable-update operation, say var.assign_add(x), in a model, we need to resolve how to combine the different values for x computed in the different replicas.


Indicates when a distributed variable will be synced.
Compat aliases for migration
See Migration guide for more details.
tf.compat.v1.VariableSynchronization


Scope which defines a variable creation function to be used by variable().
variable_creator is expected to be a function with the following signature:
The creator is supposed to eventually call the next_creator to create a variable if it does want to create a variable and not call Variable or ResourceVariable directly. This helps make creators composable. A creator may choose to create multiple variables, return already existing variables, or simply register that a variable was created and defer to the next creators in line. Creators can also modify the keyword arguments seen by the next creators.
Custom getters in the variable scope will eventually resolve down to these custom creators when they do create variables.
The valid keyword arguments in kwds are:
This set may grow over time, so it's important the signature of creators is as mentioned above.


Parallel map on the list of tensors unpacked from elems on dimension 0.
Compat aliases for migration
See Migration guide for more details.
tf.compat.v1.vectorized_map
This method works similar to tf.map_fn but is optimized to run much faster, possibly with a much larger memory footprint. The speedups are obtained by vectorization (see Auto-Vectorizing TensorFlow Graphs: Jacobians, Auto-Batching and Beyond). The idea behind vectorization is to semantically launch all the invocations of fn in parallel and fuse corresponding operations across all these invocations. This fusion is done statically at graph generation time and the generated code is often similar in performance to a manually fused version.
Because tf.vectorized_map fully parallelizes the batch, this method will generally be significantly faster than using tf.map_fn, especially in eager mode. However this is an experimental feature and currently has a lot of limitations:
Although they are less common as user-visible inputs and outputs, note that tensors of type tf.variant which represent tensor lists (for example from tf.raw_ops.TensorListFromTensor) are vectorized by stacking the list contents rather than the variant itself, and so the container tensor will have a scalar shape when returned rather than the usual stacked shape. This improves the performance of control flow gradient vectorization.


Return the elements where condition is True (multiplexing x and y).
Compat aliases for migration
See Migration guide for more details.
tf.compat.v1.where_v2
This operator has two modes: in one mode both x and y are provided, in another mode neither are provided. condition is always expected to be a tf.Tensor of type bool.
If x and y are not provided (both are None):
tf.where will return the indices of condition that are True, in the form of a 2-D tensor with shape (n, d). (Where n is the number of matching indices in condition, and d is the number of dimensions in condition).
Indices are output in row-major order.
If x and y are provided (both have non-None values):
tf.where will choose an output shape from the shapes of condition, x, and y that all three shapes are broadcastable to.
The condition tensor acts as a mask that chooses whether the corresponding element / row in the output should be taken from x (if the element in condition is True) or y (if it is false).
Note that if the gradient of either branch of the tf.where generates a NaN, then the gradient of the entire tf.where will be NaN. A workaround is to use an inner tf.where to ensure the function has no asymptote, and to avoid computing a value whose gradient is NaN by replacing dangerous inputs with safe inputs.
Instead of this,
Use this


Repeat body while the condition cond is true. (deprecated argument values)
cond is a callable returning a boolean scalar tensor. body is a callable returning a (possibly nested) tuple, namedtuple or list of tensors of the same arity (length and structure) and types as loop_vars. loop_vars is a (possibly nested) tuple, namedtuple or list of tensors that is passed to both cond and body. cond and body both take as many arguments as there are loop_vars.
In addition to regular Tensors or IndexedSlices, the body may accept and return TensorArray objects. The flows of the TensorArray objects will be appropriately forwarded between loops and during gradient calculations.
Note that while_loop calls cond and body exactly once (inside the call to while_loop, and not at all during Session.run()). while_loop stitches together the graph fragments created during the cond and body calls with some additional graph nodes to create the graph flow that repeats body until cond returns false.
For correctness, tf.while_loop() strictly enforces shape invariants for the loop variables. A shape invariant is a (possibly partial) shape that is unchanged across the iterations of the loop. An error will be raised if the shape of a loop variable after an iteration is determined to be more general than or incompatible with its shape invariant. For example, a shape of [11, None] is more general than a shape of [11, 17], and [11, 21] is not compatible with [11, 17]. By default (if the argument shape_invariants is not specified), it is assumed that the initial shape of each tensor in loop_vars is the same in every iteration. The shape_invariants argument allows the caller to specify a less specific shape invariant for each loop variable, which is needed if the shape varies between iterations. The tf.Tensor.set_shape function may also be used in the body function to indicate that the output loop variable has a particular shape. The shape invariant for SparseTensor and IndexedSlices are treated specially as follows:
a) If a loop variable is a SparseTensor, the shape invariant must be TensorShape([r]) where r is the rank of the dense tensor represented by the sparse tensor. It means the shapes of the three tensors of the SparseTensor are ([None], [None, r], [r]). NOTE: The shape invariant here is the shape of the SparseTensor.dense_shape property. It must be the shape of a vector.
b) If a loop variable is an IndexedSlices, the shape invariant must be a shape invariant of the values tensor of the IndexedSlices. It means the shapes of the three tensors of the IndexedSlices are (shape, [shape[0]], [shape.ndims]).
while_loop implements non-strict semantics, enabling multiple iterations to run in parallel. The maximum number of parallel iterations can be controlled by parallel_iterations, which gives users some control over memory consumption and execution order. For correct programs, while_loop should return the same result for any parallel_iterations > 0.
For training, TensorFlow stores the tensors that are produced in the forward inference and are needed in back propagation. These tensors are a main source of memory consumption and often cause OOM errors when training on GPUs. When the flag swap_memory is true, we swap out these tensors from GPU to CPU. This for example allows us to train RNN models with very long sequences and large batches.
Example with nesting and a namedtuple:
Example using shape_invariants:
Example which demonstrates non-strict semantics: In the following example, the final value of the counter i does not depend on x. So the while_loop can increment the counter parallel to updates of x. However, because the loop counter at one loop iteration depends on the value at the previous iteration, the loop counter itself cannot be incremented in parallel. Hence if we just want the final value of the counter (which we print on the line print(sess.run(i))), then x will never be incremented, but the counter will be updated on a single thread. Conversely, if we want the value of the output (which we print on the line print(sess.run(out).shape)), then the counter may be incremented on its own thread, while x can be incremented in parallel on a separate thread. In the extreme case, it is conceivable that the thread incrementing the counter runs until completion before x is incremented even a single time. The only thing that can never happen is that the thread updating x can never get ahead of the counter thread because the thread incrementing x depends on the value of the counter.


Public API for tf.xla namespace.
experimental module: Public API for tf.xla.experimental namespace.


Public API for tf.xla.experimental namespace.
compile(...): Builds an operator that compiles and runs computation with XLA. (deprecated)
jit_scope(...): Enable or disable JIT compilation of operators within the scope.


Builds an operator that compiles and runs computation with XLA. (deprecated)
Compat aliases for migration
See Migration guide for more details.
tf.compat.v1.xla.experimental.compile
computation may return a list of operations and tensors. Tensors must come before operations in the returned list. The return value of compile is a list of tensors corresponding to the tensors from the output of computation.
All Operations returned from computation will be executed when evaluating any of the returned output tensors.
1) None output: a NoOp would be returned which control-depends on computation. 2) Single value output: A tuple containing the value would be returned. 3) Operation-only outputs: a NoOp would be returned which control-depends on computation.
When a tf.random operation is built with XLA, the implementation doesn't pass the user provided seed to the XLA compiler. As such, the XLA compiler generates a random number and uses it as a seed when compiling the operation. This implementation causes a violation of the Tensorflow defined semantics in two aspects. First, changing the value of the user defined seed doesn't change the numbers generated by the operation. Second, when a seed is not specified, running the program multiple times will generate the same numbers.


Enable or disable JIT compilation of operators within the scope.
Compat aliases for migration
See Migration guide for more details.
tf.compat.v1.xla.experimental.jit_scope
The compilation is a hint and only supported on a best-effort basis.
Example of separate_compiled_gradients:
Ops that are not in the scope may be clustered and compiled with ops in the scope with compile_ops=True, while the ops in the scope with compile_ops=False will never be compiled.
If you want to only compile the ops in the scope with compile_ops=True, consider adding an outer jit_scope(compile_ops=False):


Creates a tensor with all elements set to zero.
Compat aliases for migration
See Migration guide for more details.
tf.compat.v1.zeros
See also tf.zeros_like, tf.ones, tf.fill, tf.eye.
This operation returns a tensor of type dtype with shape shape and all elements set to zero.


Initializer that generates tensors initialized to 0.
Initializers allow you to pre-specify an initialization strategy, encoded in the Initializer object, without knowing the shape and dtype of the variable being initialized.
View source
Instantiates an initializer from a configuration dictionary.
View source
Returns the configuration of the initializer as a JSON-serializable dict.
View source
Returns a tensor object initialized as specified by the initializer.


Creates a tensor with all elements set to zero.
See also tf.zeros.
Given a single tensor or array-like object (input), this operation returns a tensor of the same type and shape as input with all elements set to zero. Optionally, you can use dtype to specify a new type for the returned tensor.


The torch package contains data structures for multi-dimensional tensors and defines mathematical operations over these tensors. Additionally, it provides many utilities for efficient serializing of Tensors and arbitrary types, and other useful utilities.
It has a CUDA counterpart, that enables you to run your tensor computations on an NVIDIA GPU with compute capability >= 3.0
Returns True if obj is a PyTorch tensor.
Returns True if obj is a PyTorch storage object.
Returns True if the data type of input is a complex data type i.e., one of torch.complex64, and torch.complex128.
Returns True if the data type of input is a floating point data type i.e., one of torch.float64, torch.float32, torch.float16, and torch.bfloat16.
Returns True if the input is a single element tensor which is not equal to zero after type conversions.
Sets the default floating point dtype to d.
Get the current default floating point torch.dtype.
Sets the default torch.Tensor type to floating point tensor type t.
Returns the total number of elements in the input tensor.
Set options for printing.
Disables denormal floating numbers on CPU.
Note
Random sampling creation ops are listed under Random sampling and include: torch.rand() torch.rand_like() torch.randn() torch.randn_like() torch.randint() torch.randint_like() torch.randperm() You may also use torch.empty() with the In-place random sampling methods to create torch.Tensor s with values sampled from a broader range of distributions.
Constructs a tensor with data.
Constructs a sparse tensor in COO(rdinate) format with specified values at the given indices.
Convert the data into a torch.Tensor.
Create a view of an existing torch.Tensor input with specified size, stride and storage_offset.
Creates a Tensor from a numpy.ndarray.
Returns a tensor filled with the scalar value 0, with the shape defined by the variable argument size.
Returns a tensor filled with the scalar value 0, with the same size as input.
Returns a tensor filled with the scalar value 1, with the shape defined by the variable argument size.
Returns a tensor filled with the scalar value 1, with the same size as input.
Returns a 1-D tensor of size ⌈end−startstep⌉\left\lceil \frac{\text{end} - \text{start}}{\text{step}} \right\rceil  with values from the interval [start, end) taken with common difference step beginning from start.
Returns a 1-D tensor of size ⌊end−startstep⌋+1\left\lfloor \frac{\text{end} - \text{start}}{\text{step}} \right\rfloor + 1  with values from start to end with step step.
Creates a one-dimensional tensor of size steps whose values are evenly spaced from start to end, inclusive.
Creates a one-dimensional tensor of size steps whose values are evenly spaced from basestart{{\text{{base}}}}^{{\text{{start}}}}  to baseend{{\text{{base}}}}^{{\text{{end}}}} , inclusive, on a logarithmic scale with base base.
Returns a 2-D tensor with ones on the diagonal and zeros elsewhere.
Returns a tensor filled with uninitialized data.
Returns an uninitialized tensor with the same size as input.
Returns a tensor filled with uninitialized data.
Creates a tensor of size size filled with fill_value.
Returns a tensor with the same size as input filled with fill_value.
Converts a float tensor to a quantized tensor with given scale and zero point.
Converts a float tensor to a per-channel quantized tensor with given scales and zero points.
Returns an fp32 Tensor by dequantizing a quantized Tensor
Constructs a complex tensor with its real part equal to real and its imaginary part equal to imag.
Constructs a complex tensor whose elements are Cartesian coordinates corresponding to the polar coordinates with absolute value abs and angle angle.
Computes the Heaviside step function for each element in input.
Concatenates the given sequence of seq tensors in the given dimension.
Splits a tensor into a specific number of chunks.
Creates a new tensor by horizontally stacking the tensors in tensors.
Stack tensors in sequence depthwise (along third axis).
Gathers values along an axis specified by dim.
Stack tensors in sequence horizontally (column wise).
Returns a new tensor which indexes the input tensor along dimension dim using the entries in index which is a LongTensor.
Returns a new 1-D tensor which indexes the input tensor according to the boolean mask mask which is a BoolTensor.
Moves the dimension(s) of input at the position(s) in source to the position(s) in destination.
Alias for torch.movedim().
Returns a new tensor that is a narrowed version of input tensor.
Returns a tensor with the same data and number of elements as input, but with the specified shape.
Alias of torch.vstack().
Out-of-place version of torch.Tensor.scatter_()
Out-of-place version of torch.Tensor.scatter_add_()
Splits the tensor into chunks.
Returns a tensor with all the dimensions of input of size 1 removed.
Concatenates a sequence of tensors along a new dimension.
Alias for torch.transpose().
Alias for torch.transpose().
Expects input to be <= 2-D tensor and transposes dimensions 0 and 1.
Returns a new tensor with the elements of input at the given indices.
Splits a tensor into multiple sub-tensors, all of which are views of input, along dimension dim according to the indices or number of sections specified by indices_or_sections.
Constructs a tensor by repeating the elements of input.
Returns a tensor that is a transposed version of input.
Removes a tensor dimension.
Returns a new tensor with a dimension of size one inserted at the specified position.
Stack tensors in sequence vertically (row wise).
Return a tensor of elements selected from either x or y, depending on condition.
Creates and returns a generator object that manages the state of the algorithm which produces pseudo random numbers.
Sets the seed for generating random numbers to a non-deterministic random number.
Sets the seed for generating random numbers.
Returns the initial seed for generating random numbers as a Python long.
Returns the random number generator state as a torch.ByteTensor.
Sets the random number generator state.
Draws binary random numbers (0 or 1) from a Bernoulli distribution.
Returns a tensor where each row contains num_samples indices sampled from the multinomial probability distribution located in the corresponding row of tensor input.
Returns a tensor of random numbers drawn from separate normal distributions whose mean and standard deviation are given.
Returns a tensor of the same size as input with each element sampled from a Poisson distribution with rate parameter given by the corresponding element in input i.e.,
Returns a tensor filled with random numbers from a uniform distribution on the interval [0,1)[0, 1)
Returns a tensor with the same size as input that is filled with random numbers from a uniform distribution on the interval [0,1)[0, 1) .
Returns a tensor filled with random integers generated uniformly between low (inclusive) and high (exclusive).
Returns a tensor with the same shape as Tensor input filled with random integers generated uniformly between low (inclusive) and high (exclusive).
Returns a tensor filled with random numbers from a normal distribution with mean 0 and variance 1 (also called the standard normal distribution).
Returns a tensor with the same size as input that is filled with random numbers from a normal distribution with mean 0 and variance 1.
Returns a random permutation of integers from 0 to n - 1.
There are a few more in-place random sampling functions defined on Tensors as well. Click through to refer to their documentation:
quasirandom.SobolEngine
The torch.quasirandom.SobolEngine is an engine for generating (scrambled) Sobol sequences.
Saves an object to a disk file.
Loads an object saved with torch.save() from a file.
Returns the number of threads used for parallelizing CPU operations
Sets the number of threads used for intraop parallelism on CPU.
Returns the number of threads used for inter-op parallelism on CPU (e.g.
Sets the number of threads used for interop parallelism (e.g.
The context managers torch.no_grad(), torch.enable_grad(), and torch.set_grad_enabled() are helpful for locally disabling and enabling gradient computation. See Locally disabling gradient computation for more details on their usage. These context managers are thread local, so they won’t work if you send work to another thread using the threading module, etc.
Examples:
Context-manager that disabled gradient calculation.
Context-manager that enables gradient calculation.
Context-manager that sets gradient calculation to on or off.
Computes the absolute value of each element in input.
Alias for torch.abs()
Computes the inverse cosine of each element in input.
Alias for torch.acos().
Returns a new tensor with the inverse hyperbolic cosine of the elements of input.
Alias for torch.acosh().
Adds the scalar other to each element of the input input and returns a new resulting tensor.
Performs the element-wise division of tensor1 by tensor2, multiply the result by the scalar value and add it to input.
Performs the element-wise multiplication of tensor1 by tensor2, multiply the result by the scalar value and add it to input.
Computes the element-wise angle (in radians) of the given input tensor.
Returns a new tensor with the arcsine of the elements of input.
Alias for torch.asin().
Returns a new tensor with the inverse hyperbolic sine of the elements of input.
Alias for torch.asinh().
Returns a new tensor with the arctangent of the elements of input.
Alias for torch.atan().
Returns a new tensor with the inverse hyperbolic tangent of the elements of input.
Alias for torch.atanh().
Element-wise arctangent of inputi/otheri\text{input}_{i} / \text{other}_{i}  with consideration of the quadrant.
Computes the bitwise NOT of the given input tensor.
Computes the bitwise AND of input and other.
Computes the bitwise OR of input and other.
Computes the bitwise XOR of input and other.
Returns a new tensor with the ceil of the elements of input, the smallest integer greater than or equal to each element.
Clamp all elements in input into the range [ min, max ].
Alias for torch.clamp().
Computes the element-wise conjugate of the given input tensor.
Create a new floating-point tensor with the magnitude of input and the sign of other, elementwise.
Returns a new tensor with the cosine of the elements of input.
Returns a new tensor with the hyperbolic cosine of the elements of input.
Returns a new tensor with each of the elements of input converted from angles in degrees to radians.
Divides each element of the input input by the corresponding element of other.
Alias for torch.div().
Computes the logarithmic derivative of the gamma function on input.
Computes the error function of each element.
Computes the complementary error function of each element of input.
Computes the inverse error function of each element of input.
Returns a new tensor with the exponential of the elements of the input tensor input.
Computes the base two exponential function of input.
Returns a new tensor with the exponential of the elements minus 1 of input.
Returns a new tensor with the data in input fake quantized per channel using scale, zero_point, quant_min and quant_max, across the channel specified by axis.
Returns a new tensor with the data in input fake quantized using scale, zero_point, quant_min and quant_max.
Alias for torch.trunc()
Raises input to the power of exponent, elementwise, in double precision.
Returns a new tensor with the floor of the elements of input, the largest integer less than or equal to each element.
Computes the element-wise remainder of division.
Computes the fractional portion of each element in input.
Returns a new tensor containing imaginary values of the self tensor.
Multiplies input by 2**:attr:other.
Does a linear interpolation of two tensors start (given by input) and end based on a scalar or tensor weight and returns the resulting out tensor.
Computes the logarithm of the gamma function on input.
Returns a new tensor with the natural logarithm of the elements of input.
Returns a new tensor with the logarithm to the base 10 of the elements of input.
Returns a new tensor with the natural logarithm of (1 + input).
Returns a new tensor with the logarithm to the base 2 of the elements of input.
Logarithm of the sum of exponentiations of the inputs.
Logarithm of the sum of exponentiations of the inputs in base-2.
Computes the element-wise logical AND of the given input tensors.
Computes the element-wise logical NOT of the given input tensor.
Computes the element-wise logical OR of the given input tensors.
Computes the element-wise logical XOR of the given input tensors.
Returns a new tensor with the logit of the elements of input.
Given the legs of a right triangle, return its hypotenuse.
Computes the zeroth order modified Bessel function of the first kind for each element of input.
Computes the regularized lower incomplete gamma function:
Computes the regularized upper incomplete gamma function:
Multiplies each element of the input input with the scalar other and returns a new resulting tensor.
Alias for torch.mul().
Computes the multivariate log-gamma function) with dimension pp  element-wise, given by
Replaces NaN, positive infinity, and negative infinity values in input with the values specified by nan, posinf, and neginf, respectively.
Returns a new tensor with the negative of the elements of input.
Alias for torch.neg()
Return the next floating-point value after input towards other, elementwise.
Computes the nthn^{th}  derivative of the digamma function on input.
Takes the power of each element in input with exponent and returns a tensor with the result.
Returns a new tensor with each of the elements of input converted from angles in radians to degrees.
Returns a new tensor containing real values of the self tensor.
Returns a new tensor with the reciprocal of the elements of input
Computes the element-wise remainder of division.
Returns a new tensor with each of the elements of input rounded to the closest integer.
Returns a new tensor with the reciprocal of the square-root of each of the elements of input.
Returns a new tensor with the sigmoid of the elements of input.
Returns a new tensor with the signs of the elements of input.
For complex tensors, this function returns a new tensor whose elemants have the same angle as that of the elements of input and absolute value 1.
Tests if each element of input has its sign bit set (is less than zero) or not.
Returns a new tensor with the sine of the elements of input.
Computes the normalized sinc of input.
Returns a new tensor with the hyperbolic sine of the elements of input.
Returns a new tensor with the square-root of the elements of input.
Returns a new tensor with the square of the elements of input.
Subtracts other, scaled by alpha, from input.
Alias for torch.sub().
Returns a new tensor with the tangent of the elements of input.
Returns a new tensor with the hyperbolic tangent of the elements of input.
Alias for torch.div() with rounding_mode=None.
Returns a new tensor with the truncated integer values of the elements of input.
Computes input * log(other) with the following cases.
Returns the indices of the maximum value of all elements in the input tensor.
Returns the indices of the minimum value(s) of the flattened tensor or along a dimension
Returns the maximum value of each slice of the input tensor in the given dimension(s) dim.
Returns the minimum value of each slice of the input tensor in the given dimension(s) dim.
Tests if all elements in input evaluate to True.
the input tensor.
Returns the maximum value of all elements in the input tensor.
Returns the minimum value of all elements in the input tensor.
Returns the p-norm of (input - other)
Returns the log of summed exponentials of each row of the input tensor in the given dimension dim.
Returns the mean value of all elements in the input tensor.
Returns the median of the values in input.
Returns the median of the values in input, ignoring NaN values.
Returns a namedtuple (values, indices) where values is the mode value of each row of the input tensor in the given dimension dim, i.e.
Returns the matrix norm or vector norm of a given tensor.
Returns the sum of all elements, treating Not a Numbers (NaNs) as zero.
Returns the product of all elements in the input tensor.
Returns the q-th quantiles of all elements in the input tensor, doing a linear interpolation when the q-th quantile lies between two data points.
This is a variant of torch.quantile() that “ignores” NaN values, computing the quantiles q as if NaN values in input did not exist.
Returns the standard-deviation of all elements in the input tensor.
Returns the standard-deviation and mean of all elements in the input tensor.
Returns the sum of all elements in the input tensor.
Returns the unique elements of the input tensor.
Eliminates all but the first element from every consecutive group of equivalent elements.
Returns the variance of all elements in the input tensor.
Returns the variance and mean of all elements in the input tensor.
Counts the number of non-zero values in the tensor input along the given dim.
This function checks if all input and other satisfy the condition:
Returns the indices that sort a tensor along a given dimension in ascending order by value.
Computes element-wise equality
True if two tensors have the same size and elements, False otherwise.
Computes input≥other\text{input} \geq \text{other}  element-wise.
Alias for torch.ge().
Computes input>other\text{input} > \text{other}  element-wise.
Alias for torch.gt().
Returns a new tensor with boolean elements representing if each element of input is “close” to the corresponding element of other.
Returns a new tensor with boolean elements representing if each element is finite or not.
Tests if each element of input is infinite (positive or negative infinity) or not.
Tests if each element of input is positive infinity or not.
Tests if each element of input is negative infinity or not.
Returns a new tensor with boolean elements representing if each element of input is NaN or not.
Returns a new tensor with boolean elements representing if each element of input is real-valued or not.
Returns a namedtuple (values, indices) where values is the k th smallest element of each row of the input tensor in the given dimension dim.
Computes input≤other\text{input} \leq \text{other}  element-wise.
Alias for torch.le().
Computes input<other\text{input} < \text{other}  element-wise.
Alias for torch.lt().
Computes the element-wise maximum of input and other.
Computes the element-wise minimum of input and other.
Computes the element-wise maximum of input and other.
Computes the element-wise minimum of input and other.
Computes input≠other\text{input} \neq \text{other}  element-wise.
Alias for torch.ne().
Sorts the elements of the input tensor along a given dimension in ascending order by value.
Returns the k largest elements of the given input tensor along a given dimension.
Sorts the elements of the input tensor along its first dimension in ascending order by value.
Short-time Fourier transform (STFT).
Inverse short time Fourier Transform.
Bartlett window function.
Blackman window function.
Hamming window function.
Hann window function.
Computes the Kaiser window with window length window_length and shape parameter beta.
Returns a 1-dimensional view of each input tensor with zero dimensions.
Returns a 2-dimensional view of each input tensor with zero dimensions.
Returns a 3-dimensional view of each input tensor with zero dimensions.
Count the frequency of each value in an array of non-negative ints.
Create a block diagonal matrix from provided tensors.
Broadcasts the given tensors according to Broadcasting semantics.
Broadcasts input to the shape shape.
Similar to broadcast_tensors() but for shapes.
Returns the indices of the buckets to which each value in the input belongs, where the boundaries of the buckets are set by boundaries.
Do cartesian product of the given sequence of tensors.
Computes batched the p-norm distance between each pair of the two collections of row vectors.
Returns a copy of input.
Compute combinations of length rr  of the given tensor.
Returns the cross product of vectors in dimension dim of input and other.
Returns a namedtuple (values, indices) where values is the cumulative maximum of elements of input in the dimension dim.
Returns a namedtuple (values, indices) where values is the cumulative minimum of elements of input in the dimension dim.
Returns the cumulative product of elements of input in the dimension dim.
Returns the cumulative sum of elements of input in the dimension dim.
Creates a tensor whose diagonals of certain 2D planes (specified by dim1 and dim2) are filled by input.
Returns a partial view of input with the its diagonal elements with respect to dim1 and dim2 appended as a dimension at the end of the shape.
Computes the n-th forward difference along the given dimension.
Sums the product of the elements of the input operands along dimensions specified using a notation based on the Einstein summation convention.
Flattens input by reshaping it into a one-dimensional tensor.
Reverse the order of a n-D tensor along given axis in dims.
Flip tensor in the left/right direction, returning a new tensor.
Flip tensor in the up/down direction, returning a new tensor.
Computes the Kronecker product, denoted by ⊗\otimes , of input and other.
Rotate a n-D tensor by 90 degrees in the plane specified by dims axis.
Computes the element-wise greatest common divisor (GCD) of input and other.
Computes the histogram of a tensor.
Take NN  tensors, each of which can be either scalar or 1-dimensional vector, and create NN  N-dimensional grids, where the ii  th grid is defined by expanding the ii  th input over dimensions defined by other inputs.
Computes the element-wise least common multiple (LCM) of input and other.
Returns the logarithm of the cumulative summation of the exponentiation of elements of input in the dimension dim.
Return a contiguous flattened tensor.
Returns a tensor where each sub-tensor of input along dimension dim is normalized such that the p-norm of the sub-tensor is lower than the value maxnorm
Repeat elements of a tensor.
Roll the tensor along the given dimension(s).
Find the indices from the innermost dimension of sorted_sequence such that, if the corresponding values in values were inserted before the indices, the order of the corresponding innermost dimension within sorted_sequence would be preserved.
Returns a contraction of a and b over multiple dimensions.
Returns the sum of the elements of the diagonal of the input 2-D matrix.
Returns the lower triangular part of the matrix (2-D tensor) or batch of matrices input, the other elements of the result tensor out are set to 0.
Returns the indices of the lower triangular part of a row-by- col matrix in a 2-by-N Tensor, where the first row contains row coordinates of all indices and the second row contains column coordinates.
Returns the upper triangular part of a matrix (2-D tensor) or batch of matrices input, the other elements of the result tensor out are set to 0.
Returns the indices of the upper triangular part of a row by col matrix in a 2-by-N Tensor, where the first row contains row coordinates of all indices and the second row contains column coordinates.
Generates a Vandermonde matrix.
Returns a view of input as a real tensor.
Returns a view of input as a complex tensor.
Performs a batch matrix-matrix product of matrices stored in batch1 and batch2, with a reduced add step (all matrix multiplications get accumulated along the first dimension).
Performs a matrix multiplication of the matrices mat1 and mat2.
Performs a matrix-vector product of the matrix mat and the vector vec.
Performs the outer-product of vectors vec1 and vec2 and adds it to the matrix input.
Performs a batch matrix-matrix product of matrices in batch1 and batch2.
Performs a batch matrix-matrix product of matrices stored in input and mat2.
Returns the matrix product of the NN  2-D tensors.
Computes the Cholesky decomposition of a symmetric positive-definite matrix AA  or for batches of symmetric positive-definite matrices.
Computes the inverse of a symmetric positive-definite matrix AA  using its Cholesky factor uu : returns matrix inv.
Solves a linear system of equations with a positive semidefinite matrix to be inverted given its Cholesky factor matrix uu .
Computes the dot product of two 1D tensors.
Computes the eigenvalues and eigenvectors of a real square matrix.
This is a low-level function for calling LAPACK directly.
Alias of torch.outer().
Computes the dot product for 1D tensors.
Takes the inverse of the square matrix input.
Calculates determinant of a square matrix or batches of square matrices.
Calculates log determinant of a square matrix or batches of square matrices.
Calculates the sign and log absolute value of the determinant(s) of a square matrix or batches of square matrices.
Computes the solution to the least squares and least norm problems for a full rank matrix AA  of size (m×n)(m \times n)  and a matrix BB  of size (m×k)(m \times k) .
Computes the LU factorization of a matrix or batches of matrices A.
Returns the LU solve of the linear system Ax=bAx = b  using the partially pivoted LU factorization of A from torch.lu().
Unpacks the data and pivots from a LU factorization of a tensor.
Matrix product of two tensors.
Returns the matrix raised to the power n for square matrices.
Returns the numerical rank of a 2-D tensor.
Returns the matrix exponential.
Performs a matrix multiplication of the matrices input and mat2.
Performs a matrix-vector product of the matrix input and the vector vec.
Computes the orthogonal matrix Q of a QR factorization, from the (input, input2) tuple returned by torch.geqrf().
Multiplies mat (given by input3) by the orthogonal Q matrix of the QR factorization formed by torch.geqrf() that is represented by (a, tau) (given by (input, input2)).
Outer product of input and vec2.
Calculates the pseudo-inverse (also known as the Moore-Penrose inverse) of a 2D tensor.
Computes the QR decomposition of a matrix or a batch of matrices input, and returns a namedtuple (Q, R) of tensors such that input=QR\text{input} = Q R  with QQ  being an orthogonal matrix or batch of orthogonal matrices and RR  being an upper triangular matrix or batch of upper triangular matrices.
This function returns the solution to the system of linear equations represented by AX=BAX = B  and the LU factorization of A, in order as a namedtuple solution, LU.
Computes the singular value decomposition of either a matrix or batch of matrices input.
Return the singular value decomposition (U, S, V) of a matrix, batches of matrices, or a sparse matrix AA  such that A≈Udiag(S)VTA \approx U diag(S) V^T .
Performs linear Principal Component Analysis (PCA) on a low-rank matrix, batches of such matrices, or sparse matrix.
This function returns eigenvalues and eigenvectors of a real symmetric matrix input or a batch of real symmetric matrices, represented by a namedtuple (eigenvalues, eigenvectors).
Find the k largest (or smallest) eigenvalues and the corresponding eigenvectors of a symmetric positive defined generalized eigenvalue problem using matrix-free LOBPCG methods.
Estimate ∫ydx\int y\,dx  along dim, using the trapezoid rule.
Solves a system of equations with a triangular coefficient matrix AA  and multiple right-hand sides bb .
Computes the dot product of two 1D tensors.
Returns whether PyTorch was built with _GLIBCXX_USE_CXX11_ABI=1
Returns the torch.dtype that would result from performing an arithmetic operation on the provided input tensors.
Determines if a type conversion is allowed under PyTorch casting rules described in the type promotion documentation.
Returns the torch.dtype with the smallest size and scalar kind that is not smaller nor of lower kind than either type1 or type2.
Sets whether PyTorch operations must use “deterministic” algorithms.
Returns True if the global deterministic flag is turned on.
A wrapper around Python’s assert which is symbolically traceable.


Computes the absolute value of each element in input.
input (Tensor) – the input tensor.
out (Tensor, optional) – the output tensor.
Example:


Computes the inverse cosine of each element in input.
input (Tensor) – the input tensor.
out (Tensor, optional) – the output tensor.
Example:


Returns a new tensor with the inverse hyperbolic cosine of the elements of input.
Note
The domain of the inverse hyperbolic cosine is [1, inf) and values outside this range will be mapped to NaN, except for + INF for which the output is mapped to + INF.
input (Tensor) – the input tensor.
out (Tensor, optional) – the output tensor.
Example:


Adds the scalar other to each element of the input input and returns a new resulting tensor.
If input is of type FloatTensor or DoubleTensor, other must be a real number, otherwise it should be an integer.
out (Tensor, optional) – the output tensor.
Example:
Each element of the tensor other is multiplied by the scalar alpha and added to each element of the tensor input. The resulting tensor is returned.
The shapes of input and other must be broadcastable.
If other is of type FloatTensor or DoubleTensor, alpha must be a real number, otherwise it should be an integer.
Example:


Performs a batch matrix-matrix product of matrices stored in batch1 and batch2, with a reduced add step (all matrix multiplications get accumulated along the first dimension). input is added to the final result.
batch1 and batch2 must be 3-D tensors each containing the same number of matrices.
If batch1 is a (b×n×m)(b \times n \times m)  tensor, batch2 is a (b×m×p)(b \times m \times p)  tensor, input must be broadcastable with a (n×p)(n \times p)  tensor and out will be a (n×p)(n \times p)  tensor.
If beta is 0, then input will be ignored, and nan and inf in it will not be propagated.
For inputs of type FloatTensor or DoubleTensor, arguments beta and alpha must be real numbers, otherwise they should be integers.
This operator supports TensorFloat32.
Example:


Performs the element-wise division of tensor1 by tensor2, multiply the result by the scalar value and add it to input.
Warning
Integer division with addcdiv is no longer supported, and in a future release addcdiv will perform a true division of tensor1 and tensor2. The historic addcdiv behavior can be implemented as (input + value * torch.trunc(tensor1 / tensor2)).to(input.dtype) for integer inputs and as (input + value * tensor1 / tensor2) for float inputs. The future addcdiv behavior is just the latter implementation: (input + value * tensor1 / tensor2), for all dtypes.
The shapes of input, tensor1, and tensor2 must be broadcastable.
For inputs of type FloatTensor or DoubleTensor, value must be a real number, otherwise an integer.
Example:


Performs the element-wise multiplication of tensor1 by tensor2, multiply the result by the scalar value and add it to input.
The shapes of tensor, tensor1, and tensor2 must be broadcastable.
For inputs of type FloatTensor or DoubleTensor, value must be a real number, otherwise an integer.
Example:


Performs a matrix multiplication of the matrices mat1 and mat2. The matrix input is added to the final result.
If mat1 is a (n×m)(n \times m)  tensor, mat2 is a (m×p)(m \times p)  tensor, then input must be broadcastable with a (n×p)(n \times p)  tensor and out will be a (n×p)(n \times p)  tensor.
alpha and beta are scaling factors on matrix-vector product between mat1 and mat2 and the added matrix input respectively.
If beta is 0, then input will be ignored, and nan and inf in it will not be propagated.
For inputs of type FloatTensor or DoubleTensor, arguments beta and alpha must be real numbers, otherwise they should be integers.
This operator supports TensorFloat32.
Example:


Performs a matrix-vector product of the matrix mat and the vector vec. The vector input is added to the final result.
If mat is a (n×m)(n \times m)  tensor, vec is a 1-D tensor of size m, then input must be broadcastable with a 1-D tensor of size n and out will be 1-D tensor of size n.
alpha and beta are scaling factors on matrix-vector product between mat and vec and the added tensor input respectively.
If beta is 0, then input will be ignored, and nan and inf in it will not be propagated.
For inputs of type FloatTensor or DoubleTensor, arguments beta and alpha must be real numbers, otherwise they should be integers
Example:


Performs the outer-product of vectors vec1 and vec2 and adds it to the matrix input.
Optional values beta and alpha are scaling factors on the outer product between vec1 and vec2 and the added matrix input respectively.
If beta is 0, then input will be ignored, and nan and inf in it will not be propagated.
If vec1 is a vector of size n and vec2 is a vector of size m, then input must be broadcastable with a matrix of size (n×m)(n \times m)  and out will be a matrix of size (n×m)(n \times m) .
Example:


Tests if all elements in input evaluate to True.
Note
This function matches the behaviour of NumPy in returning output of dtype bool for all supported dtypes except uint8. For uint8 the dtype of output is uint8 itself.
Example:
For each row of input in the given dimension dim, returns True if all elements in the row evaluate to True and False otherwise.
If keepdim is True, the output tensor is of the same size as input except in the dimension dim where it is of size 1. Otherwise, dim is squeezed (see torch.squeeze()), resulting in the output tensor having 1 fewer dimension than input.
out (Tensor, optional) – the output tensor.
Example:


This function checks if all input and other satisfy the condition:
elementwise, for all elements of input and other. The behaviour of this function is analogous to numpy.allclose
Example:


Returns the maximum value of each slice of the input tensor in the given dimension(s) dim.
Note
If keepdim is ``True`, the output tensors are of the same size as input except in the dimension(s) dim where they are of size 1. Otherwise, dim`s are squeezed (see :func:`torch.squeeze), resulting in the output tensors having fewer dimension than input.
out (Tensor, optional) – the output tensor.
Example:


Returns the minimum value of each slice of the input tensor in the given dimension(s) dim.
Note
If keepdim is True, the output tensors are of the same size as input except in the dimension(s) dim where they are of size 1. Otherwise, dim`s are squeezed (see :func:`torch.squeeze), resulting in the output tensors having fewer dimensions than input.
out (Tensor, optional) – the output tensor.
Example:


Computes the element-wise angle (in radians) of the given input tensor.
input (Tensor) – the input tensor.
out (Tensor, optional) – the output tensor.
Note
Starting in PyTorch 1.8, angle returns pi for negative real numbers, zero for non-negative real numbers, and propagates NaNs. Previously the function would return zero for all real numbers and not propagate floating-point NaNs.
Example:


input (Tensor) – the input tensor.
Tests if any element in input evaluates to True.
Note
This function matches the behaviour of NumPy in returning output of dtype bool for all supported dtypes except uint8. For uint8 the dtype of output is uint8 itself.
Example:
For each row of input in the given dimension dim, returns True if any element in the row evaluate to True and False otherwise.
If keepdim is True, the output tensor is of the same size as input except in the dimension dim where it is of size 1. Otherwise, dim is squeezed (see torch.squeeze()), resulting in the output tensor having 1 fewer dimension than input.
out (Tensor, optional) – the output tensor.
Example:


Returns a 1-D tensor of size ⌈end−startstep⌉\left\lceil \frac{\text{end} - \text{start}}{\text{step}} \right\rceil  with values from the interval [start, end) taken with common difference step beginning from start.
Note that non-integer step is subject to floating point rounding errors when comparing against end; to avoid inconsistency, we advise adding a small epsilon to end in such cases.
Example: