tf.compat.v1.train.cosine_decay Applies cosine decay to the learning rate.
tf.compat.v1.train.cosine_decay(
    learning_rate, global_step, decay_steps, alpha=0.0, name=None
)
 When training a model, it is often recommended to lower the learning rate as the training progresses. This function applies a cosine decay function to a provided initial learning rate. It requires a global_step value to compute the decayed learning rate. You can just pass a TensorFlow variable that you increment at each training step. The function returns the decayed learning rate. It is computed as: global_step = min(global_step, decay_steps)
cosine_decay = 0.5 * (1 + cos(pi * global_step / decay_steps))
decayed = (1 - alpha) * cosine_decay + alpha
decayed_learning_rate = learning_rate * decayed
 Example usage: decay_steps = 1000
lr_decayed = cosine_decay(learning_rate, global_step, decay_steps)

 Args
  learning_rate   A scalar float32 or float64 Tensor or a Python number. The initial learning rate.
  global_step   A scalar int32 or int64 Tensor or a Python number. Global step to use for the decay computation.
  decay_steps   A scalar int32 or int64 Tensor or a Python number. Number of steps to decay over.
  alpha   A scalar float32 or float64 Tensor or a Python number. Minimum learning rate value as a fraction of learning_rate.
  name   String. Optional name of the operation. Defaults to 'CosineDecay'.

 Returns   A scalar Tensor of the same type as learning_rate. The decayed learning rate.

 Raises
  ValueError   if global_step is not supplied.    References: Stochastic Gradient Descent with Warm Restarts: Loshchilov et al., 2017 (pdf) Eager Compatibility When eager execution is enabled, this function returns a function which in turn returns the decayed learning rate Tensor. This can be useful for changing the learning rate value across different invocations of optimizer functions.


tf.compat.v1.train.cosine_decay_restarts Applies cosine decay with restarts to the learning rate.
tf.compat.v1.train.cosine_decay_restarts(
    learning_rate, global_step, first_decay_steps, t_mul=2.0, m_mul=1.0, alpha=0.0,
    name=None
)
 When training a model, it is often recommended to lower the learning rate as the training progresses. This function applies a cosine decay function with restarts to a provided initial learning rate. It requires a global_step value to compute the decayed learning rate. You can just pass a TensorFlow variable that you increment at each training step. The function returns the decayed learning rate while taking into account possible warm restarts. The learning rate multiplier first decays from 1 to alpha for first_decay_steps steps. Then, a warm restart is performed. Each new warm restart runs for t_mul times more steps and with m_mul times smaller initial learning rate. Example usage: first_decay_steps = 1000
lr_decayed = cosine_decay_restarts(learning_rate, global_step,
                                   first_decay_steps)

 Args
  learning_rate   A scalar float32 or float64 Tensor or a Python number. The initial learning rate.
  global_step   A scalar int32 or int64 Tensor or a Python number. Global step to use for the decay computation.
  first_decay_steps   A scalar int32 or int64 Tensor or a Python number. Number of steps to decay over.
  t_mul   A scalar float32 or float64 Tensor or a Python number. Used to derive the number of iterations in the i-th period
  m_mul   A scalar float32 or float64 Tensor or a Python number. Used to derive the initial learning rate of the i-th period:
  alpha   A scalar float32 or float64 Tensor or a Python number. Minimum learning rate value as a fraction of the learning_rate.
  name   String. Optional name of the operation. Defaults to 'SGDRDecay'.

 Returns   A scalar Tensor of the same type as learning_rate. The decayed learning rate.

 Raises
  ValueError   if global_step is not supplied.    References: Stochastic Gradient Descent with Warm Restarts: Loshchilov et al., 2017 (pdf) Eager Compatibility When eager execution is enabled, this function returns a function which in turn returns the decayed learning rate Tensor. This can be useful for changing the learning rate value across different invocations of optimizer functions.


tf.compat.v1.train.create_global_step Create global step tensor in graph.
tf.compat.v1.train.create_global_step(
    graph=None
)

 Args
  graph   The graph in which to create the global step tensor. If missing, use default graph.

 Returns   Global step tensor.

 Raises
  ValueError   if global step tensor is already defined.


tf.compat.v1.train.do_quantize_training_on_graphdef A general quantization scheme is being developed in tf.contrib.quantize. (deprecated)
tf.compat.v1.train.do_quantize_training_on_graphdef(
    input_graph, num_bits
)
 Warning: THIS FUNCTION IS DEPRECATED. It will be removed in a future version. Instructions for updating: GraphDef quantized training rewriter is deprecated in the long term. Consider using that instead, though since it is in the tf.contrib namespace, it is not subject to backward compatibility guarantees.

 Args
  input_graph   A GraphDef.
  num_bits   The number of bits for quantize training.

 Returns   The graph with quantize training done.


Module: tf.compat.v1.train.experimental Public API for tf.train.experimental namespace. Classes class DynamicLossScale: Loss scale that dynamically adjusts itself. class FixedLossScale: Loss scale with a fixed value. class LossScale: Base class for all TF1 loss scales. class MixedPrecisionLossScaleOptimizer: An optimizer that applies loss scaling. class PythonState: A mixin for putting Python state in an object-based checkpoint. Functions disable_mixed_precision_graph_rewrite(...): Disables the mixed precision graph rewrite. enable_mixed_precision_graph_rewrite(...): Enable mixed precision via a graph rewrite.


tf.compat.v1.train.exponential_decay Applies exponential decay to the learning rate.
tf.compat.v1.train.exponential_decay(
    learning_rate, global_step, decay_steps, decay_rate, staircase=False, name=None
)
 When training a model, it is often recommended to lower the learning rate as the training progresses. This function applies an exponential decay function to a provided initial learning rate. It requires a global_step value to compute the decayed learning rate. You can just pass a TensorFlow variable that you increment at each training step. The function returns the decayed learning rate. It is computed as: decayed_learning_rate = learning_rate *
                        decay_rate ^ (global_step / decay_steps)
 If the argument staircase is True, then global_step / decay_steps is an integer division and the decayed learning rate follows a staircase function. Example: decay every 100000 steps with a base of 0.96: ...
global_step = tf.Variable(0, trainable=False)
starter_learning_rate = 0.1
learning_rate = tf.compat.v1.train.exponential_decay(starter_learning_rate,
global_step,
                                           100000, 0.96, staircase=True)
# Passing global_step to minimize() will increment it at each step.
learning_step = (
    tf.compat.v1.train.GradientDescentOptimizer(learning_rate)
    .minimize(...my loss..., global_step=global_step)
)

 Args
  learning_rate   A scalar float32 or float64 Tensor or a Python number. The initial learning rate.
  global_step   A scalar int32 or int64 Tensor or a Python number. Global step to use for the decay computation. Must not be negative.
  decay_steps   A scalar int32 or int64 Tensor or a Python number. Must be positive. See the decay computation above.
  decay_rate   A scalar float32 or float64 Tensor or a Python number. The decay rate.
  staircase   Boolean. If True decay the learning rate at discrete intervals
  name   String. Optional name of the operation. Defaults to 'ExponentialDecay'.

 Returns   A scalar Tensor of the same type as learning_rate. The decayed learning rate.

 Raises
  ValueError   if global_step is not supplied.    Eager Compatibility When eager execution is enabled, this function returns a function which in turn returns the decayed learning rate Tensor. This can be useful for changing the learning rate value across different invocations of optimizer functions.


tf.compat.v1.train.export_meta_graph Returns MetaGraphDef proto.
tf.compat.v1.train.export_meta_graph(
    filename=None, meta_info_def=None, graph_def=None, saver_def=None,
    collection_list=None, as_text=False, graph=None, export_scope=None,
    clear_devices=False, clear_extraneous_savers=False, strip_default_attrs=False,
    save_debug_info=False, **kwargs
)
 Optionally writes it to filename. This function exports the graph, saver, and collection objects into MetaGraphDef protocol buffer with the intention of it being imported at a later time or location to restart training, run inference, or be a subgraph.

 Args
  filename   Optional filename including the path for writing the generated MetaGraphDef protocol buffer.
  meta_info_def   MetaInfoDef protocol buffer.
  graph_def   GraphDef protocol buffer.
  saver_def   SaverDef protocol buffer.
  collection_list   List of string keys to collect.
  as_text   If True, writes the MetaGraphDef as an ASCII proto.
  graph   The Graph to export. If None, use the default graph.
  export_scope   Optional string. Name scope under which to extract the subgraph. The scope name will be striped from the node definitions for easy import later into new name scopes. If None, the whole graph is exported. graph_def and export_scope cannot both be specified.
  clear_devices   Whether or not to clear the device field for an Operation or Tensor during export.
  clear_extraneous_savers   Remove any Saver-related information from the graph (both Save/Restore ops and SaverDefs) that are not associated with the provided SaverDef.
  strip_default_attrs   Boolean. If True, default-valued attributes will be removed from the NodeDefs. For a detailed guide, see Stripping Default-Valued Attributes.
  save_debug_info   If True, save the GraphDebugInfo to a separate file, which in the same directory of filename and with _debug added before the file extend.
  **kwargs   Optional keyed arguments.

 Returns   A MetaGraphDef proto.

 Raises
  ValueError   When the GraphDef is larger than 2GB.
  RuntimeError   If called with eager execution enabled.    Eager Compatibility Exporting/importing meta graphs is not supported unless both graph_def and graph are provided. No graph exists when eager execution is enabled.


tf.compat.v1.train.FtrlOptimizer Optimizer that implements the FTRL algorithm. Inherits From: Optimizer
tf.compat.v1.train.FtrlOptimizer(
    learning_rate, learning_rate_power=-0.5, initial_accumulator_value=0.1,
    l1_regularization_strength=0.0, l2_regularization_strength=0.0,
    use_locking=False, name='Ftrl', accum_name=None, linear_name=None,
    l2_shrinkage_regularization_strength=0.0, beta=None
)
 This version has support for both online L2 (McMahan et al., 2013) and shrinkage-type L2, which is the addition of an L2 penalty to the loss function. References: Ad-click prediction: McMahan et al., 2013 (pdf)

 Args
  learning_rate   A float value or a constant float Tensor.
  learning_rate_power   A float value, must be less or equal to zero. Controls how the learning rate decreases during training. Use zero for a fixed learning rate. See section 3.1 in (McMahan et al., 2013).
  initial_accumulator_value   The starting value for accumulators. Only zero or positive values are allowed.
  l1_regularization_strength   A float value, must be greater than or equal to zero.
  l2_regularization_strength   A float value, must be greater than or equal to zero.
  use_locking   If True use locks for update operations.
  name   Optional name prefix for the operations created when applying gradients. Defaults to "Ftrl".
  accum_name   The suffix for the variable that keeps the gradient squared accumulator. If not present, defaults to name.
  linear_name   The suffix for the variable that keeps the linear gradient accumulator. If not present, defaults to name + "1".

  l2_shrinkage_regularization_strength   A float value, must be greater than or equal to zero. This differs from L2 above in that the L2 above is a stabilization penalty, whereas this L2 shrinkage is a magnitude penalty. The FTRL formulation can be written as: w{t+1} = argminw(\hat{g}{1:t}w + L1||w||_1 + L2||w||_2^2), where \hat{g} = g + (2L2_shrinkagew), and g is the gradient of the loss function w.r.t. the weights w. Specifically, in the absence of L1 regularization, it is equivalent to the following update rule: w_{t+1} = w_t - lr_t / (beta + 2L2lr_t) * g_t - 2L2_shrinkagelr_t / (beta + 2L2lr_t) * w_t where lr_t is the learning rate at t. When input is sparse shrinkage will only happen on the active weights.
  beta   A float value; corresponds to the beta parameter in the paper.

 Raises
  ValueError   If one of the arguments is invalid.    Methods apply_gradients View source
apply_gradients(
    grads_and_vars, global_step=None, name=None
)
 Apply gradients to variables. This is the second part of minimize(). It returns an Operation that applies gradients.

 Args
  grads_and_vars   List of (gradient, variable) pairs as returned by compute_gradients().
  global_step   Optional Variable to increment by one after the variables have been updated.
  name   Optional name for the returned operation. Default to the name passed to the Optimizer constructor.

 Returns   An Operation that applies the specified gradients. If global_step was not None, that operation also increments global_step.

 Raises
  TypeError   If grads_and_vars is malformed.
  ValueError   If none of the variables have gradients.
  RuntimeError   If you should use _distributed_apply() instead.    compute_gradients View source
compute_gradients(
    loss, var_list=None, gate_gradients=GATE_OP, aggregation_method=None,
    colocate_gradients_with_ops=False, grad_loss=None
)
 Compute gradients of loss for the variables in var_list. This is the first part of minimize(). It returns a list of (gradient, variable) pairs where "gradient" is the gradient for "variable". Note that "gradient" can be a Tensor, an IndexedSlices, or None if there is no gradient for the given variable.

 Args
  loss   A Tensor containing the value to minimize or a callable taking no arguments which returns the value to minimize. When eager execution is enabled it must be a callable.
  var_list   Optional list or tuple of tf.Variable to update to minimize loss. Defaults to the list of variables collected in the graph under the key GraphKeys.TRAINABLE_VARIABLES.
  gate_gradients   How to gate the computation of gradients. Can be GATE_NONE, GATE_OP, or GATE_GRAPH.
  aggregation_method   Specifies the method used to combine gradient terms. Valid values are defined in the class AggregationMethod.
  colocate_gradients_with_ops   If True, try colocating gradients with the corresponding op.
  grad_loss   Optional. A Tensor holding the gradient computed for loss.

 Returns   A list of (gradient, variable) pairs. Variable is always present, but gradient can be None.

 Raises
  TypeError   If var_list contains anything else than Variable objects.
  ValueError   If some arguments are invalid.
  RuntimeError   If called with eager execution enabled and loss is not callable.    Eager Compatibility When eager execution is enabled, gate_gradients, aggregation_method, and colocate_gradients_with_ops are ignored. get_name View source
get_name()
 get_slot View source
get_slot(
    var, name
)
 Return a slot named name created for var by the Optimizer. Some Optimizer subclasses use additional variables. For example Momentum and Adagrad use variables to accumulate updates. This method gives access to these Variable objects if for some reason you need them. Use get_slot_names() to get the list of slot names created by the Optimizer.

 Args
  var   A variable passed to minimize() or apply_gradients().
  name   A string.

 Returns   The Variable for the slot if it was created, None otherwise.
 get_slot_names View source
get_slot_names()
 Return a list of the names of slots created by the Optimizer. See get_slot().

 Returns   A list of strings.
 minimize View source
minimize(
    loss, global_step=None, var_list=None, gate_gradients=GATE_OP,
    aggregation_method=None, colocate_gradients_with_ops=False, name=None,
    grad_loss=None
)
 Add operations to minimize loss by updating var_list. This method simply combines calls compute_gradients() and apply_gradients(). If you want to process the gradient before applying them call compute_gradients() and apply_gradients() explicitly instead of using this function.

 Args
  loss   A Tensor containing the value to minimize.
  global_step   Optional Variable to increment by one after the variables have been updated.
  var_list   Optional list or tuple of Variable objects to update to minimize loss. Defaults to the list of variables collected in the graph under the key GraphKeys.TRAINABLE_VARIABLES.
  gate_gradients   How to gate the computation of gradients. Can be GATE_NONE, GATE_OP, or GATE_GRAPH.
  aggregation_method   Specifies the method used to combine gradient terms. Valid values are defined in the class AggregationMethod.
  colocate_gradients_with_ops   If True, try colocating gradients with the corresponding op.
  name   Optional name for the returned operation.
  grad_loss   Optional. A Tensor holding the gradient computed for loss.

 Returns   An Operation that updates the variables in var_list. If global_step was not None, that operation also increments global_step.

 Raises
  ValueError   If some of the variables are not Variable objects.    Eager Compatibility When eager execution is enabled, loss should be a Python function that takes no arguments and computes the value to be minimized. Minimization (and gradient computation) is done with respect to the elements of var_list if not None, else with respect to any trainable variables created during the execution of the loss function. gate_gradients, aggregation_method, colocate_gradients_with_ops and grad_loss are ignored when eager execution is enabled. variables View source
variables()
 A list of variables which encode the current state of Optimizer. Includes slot variables and additional global variables created by the optimizer in the current default graph.

 Returns   A list of variables.

 Class Variables
  GATE_GRAPH   2
  GATE_NONE   0
  GATE_OP   1


tf.compat.v1.train.generate_checkpoint_state_proto Generates a checkpoint state proto.
tf.compat.v1.train.generate_checkpoint_state_proto(
    save_dir, model_checkpoint_path, all_model_checkpoint_paths=None,
    all_model_checkpoint_timestamps=None, last_preserved_timestamp=None
)

 Args
  save_dir   Directory where the model was saved.
  model_checkpoint_path   The checkpoint file.
  all_model_checkpoint_paths   List of strings. Paths to all not-yet-deleted checkpoints, sorted from oldest to newest. If this is a non-empty list, the last element must be equal to model_checkpoint_path. These paths are also saved in the CheckpointState proto.
  all_model_checkpoint_timestamps   A list of floats, indicating the number of seconds since the Epoch when each checkpoint was generated.
  last_preserved_timestamp   A float, indicating the number of seconds since the Epoch when the last preserved checkpoint was written, e.g. due to a keep_checkpoint_every_n_hours parameter (see tf.train.CheckpointManager for an implementation).

 Returns   CheckpointState proto with model_checkpoint_path and all_model_checkpoint_paths updated to either absolute paths or relative paths to the current save_dir.

 Raises
  ValueError   If all_model_checkpoint_timestamps was provided but its length does not match all_model_checkpoint_paths.


tf.compat.v1.train.get_checkpoint_mtimes Returns the mtimes (modification timestamps) of the checkpoints. (deprecated)
tf.compat.v1.train.get_checkpoint_mtimes(
    checkpoint_prefixes
)
 Warning: THIS FUNCTION IS DEPRECATED. It will be removed in a future version. Instructions for updating: Use standard file utilities to get mtimes. Globs for the checkpoints pointed to by checkpoint_prefixes. If the files exist, collect their mtime. Both V2 and V1 checkpoints are considered, in that priority. This is the recommended way to get the mtimes, since it takes into account the naming difference between V1 and V2 formats.
Note: If not all checkpoints exist, the length of the returned mtimes list will be smaller than the length of checkpoint_prefixes list, so mapping checkpoints to corresponding mtimes will not be possible.

 Args
  checkpoint_prefixes   a list of checkpoint paths, typically the results of Saver.save() or those of tf.train.latest_checkpoint(), regardless of sharded/non-sharded or V1/V2.

 Returns   A list of mtimes (in microseconds) of the found checkpoints.


tf.compat.v1.train.get_global_step Get the global step tensor.
tf.compat.v1.train.get_global_step(
    graph=None
)
 The global step tensor must be an integer variable. We first try to find it in the collection GLOBAL_STEP, or by name global_step:0.

 Args
  graph   The graph to find the global step in. If missing, use default graph.

 Returns   The global step variable, or None if none was found.

 Raises
  TypeError   If the global step tensor has a non-integer type, or if it is not a Variable.


tf.compat.v1.train.get_or_create_global_step Returns and create (if necessary) the global step tensor.
tf.compat.v1.train.get_or_create_global_step(
    graph=None
)

 Args
  graph   The graph in which to create the global step tensor. If missing, use default graph.

 Returns   The global step tensor.


tf.compat.v1.train.global_step Small helper to get the global step.
tf.compat.v1.train.global_step(
    sess, global_step_tensor
)
 # Create a variable to hold the global_step.
global_step_tensor = tf.Variable(10, trainable=False, name='global_step')
# Create a session.
sess = tf.compat.v1.Session()
# Initialize the variable
sess.run(global_step_tensor.initializer)
# Get the variable value.
print('global_step: %s' % tf.compat.v1.train.global_step(sess,
global_step_tensor))

global_step: 10

 Args
  sess   A TensorFlow Session object.
  global_step_tensor   Tensor or the name of the operation that contains the global step.

 Returns   The global step value.


tf.compat.v1.train.GradientDescentOptimizer Optimizer that implements the gradient descent algorithm. Inherits From: Optimizer
tf.compat.v1.train.GradientDescentOptimizer(
    learning_rate, use_locking=False, name='GradientDescent'
)

 Args
  learning_rate   A Tensor or a floating point value. The learning rate to use.
  use_locking   If True use locks for update operations.
  name   Optional name prefix for the operations created when applying gradients. Defaults to "GradientDescent".    Methods apply_gradients View source
apply_gradients(
    grads_and_vars, global_step=None, name=None
)
 Apply gradients to variables. This is the second part of minimize(). It returns an Operation that applies gradients.

 Args
  grads_and_vars   List of (gradient, variable) pairs as returned by compute_gradients().
  global_step   Optional Variable to increment by one after the variables have been updated.
  name   Optional name for the returned operation. Default to the name passed to the Optimizer constructor.

 Returns   An Operation that applies the specified gradients. If global_step was not None, that operation also increments global_step.

 Raises
  TypeError   If grads_and_vars is malformed.
  ValueError   If none of the variables have gradients.
  RuntimeError   If you should use _distributed_apply() instead.    compute_gradients View source
compute_gradients(
    loss, var_list=None, gate_gradients=GATE_OP, aggregation_method=None,
    colocate_gradients_with_ops=False, grad_loss=None
)
 Compute gradients of loss for the variables in var_list. This is the first part of minimize(). It returns a list of (gradient, variable) pairs where "gradient" is the gradient for "variable". Note that "gradient" can be a Tensor, an IndexedSlices, or None if there is no gradient for the given variable.

 Args
  loss   A Tensor containing the value to minimize or a callable taking no arguments which returns the value to minimize. When eager execution is enabled it must be a callable.
  var_list   Optional list or tuple of tf.Variable to update to minimize loss. Defaults to the list of variables collected in the graph under the key GraphKeys.TRAINABLE_VARIABLES.
  gate_gradients   How to gate the computation of gradients. Can be GATE_NONE, GATE_OP, or GATE_GRAPH.
  aggregation_method   Specifies the method used to combine gradient terms. Valid values are defined in the class AggregationMethod.
  colocate_gradients_with_ops   If True, try colocating gradients with the corresponding op.
  grad_loss   Optional. A Tensor holding the gradient computed for loss.

 Returns   A list of (gradient, variable) pairs. Variable is always present, but gradient can be None.

 Raises
  TypeError   If var_list contains anything else than Variable objects.
  ValueError   If some arguments are invalid.
  RuntimeError   If called with eager execution enabled and loss is not callable.    Eager Compatibility When eager execution is enabled, gate_gradients, aggregation_method, and colocate_gradients_with_ops are ignored. get_name View source
get_name()
 get_slot View source
get_slot(
    var, name
)
 Return a slot named name created for var by the Optimizer. Some Optimizer subclasses use additional variables. For example Momentum and Adagrad use variables to accumulate updates. This method gives access to these Variable objects if for some reason you need them. Use get_slot_names() to get the list of slot names created by the Optimizer.

 Args
  var   A variable passed to minimize() or apply_gradients().
  name   A string.

 Returns   The Variable for the slot if it was created, None otherwise.
 get_slot_names View source
get_slot_names()
 Return a list of the names of slots created by the Optimizer. See get_slot().

 Returns   A list of strings.
 minimize View source
minimize(
    loss, global_step=None, var_list=None, gate_gradients=GATE_OP,
    aggregation_method=None, colocate_gradients_with_ops=False, name=None,
    grad_loss=None
)
 Add operations to minimize loss by updating var_list. This method simply combines calls compute_gradients() and apply_gradients(). If you want to process the gradient before applying them call compute_gradients() and apply_gradients() explicitly instead of using this function.

 Args
  loss   A Tensor containing the value to minimize.
  global_step   Optional Variable to increment by one after the variables have been updated.
  var_list   Optional list or tuple of Variable objects to update to minimize loss. Defaults to the list of variables collected in the graph under the key GraphKeys.TRAINABLE_VARIABLES.
  gate_gradients   How to gate the computation of gradients. Can be GATE_NONE, GATE_OP, or GATE_GRAPH.
  aggregation_method   Specifies the method used to combine gradient terms. Valid values are defined in the class AggregationMethod.
  colocate_gradients_with_ops   If True, try colocating gradients with the corresponding op.
  name   Optional name for the returned operation.
  grad_loss   Optional. A Tensor holding the gradient computed for loss.

 Returns   An Operation that updates the variables in var_list. If global_step was not None, that operation also increments global_step.

 Raises
  ValueError   If some of the variables are not Variable objects.    Eager Compatibility When eager execution is enabled, loss should be a Python function that takes no arguments and computes the value to be minimized. Minimization (and gradient computation) is done with respect to the elements of var_list if not None, else with respect to any trainable variables created during the execution of the loss function. gate_gradients, aggregation_method, colocate_gradients_with_ops and grad_loss are ignored when eager execution is enabled. variables View source
variables()
 A list of variables which encode the current state of Optimizer. Includes slot variables and additional global variables created by the optimizer in the current default graph.

 Returns   A list of variables.

 Class Variables
  GATE_GRAPH   2
  GATE_NONE   0
  GATE_OP   1


tf.compat.v1.train.import_meta_graph Recreates a Graph saved in a MetaGraphDef proto.
tf.compat.v1.train.import_meta_graph(
    meta_graph_or_file, clear_devices=False, import_scope=None, **kwargs
)
 This function takes a MetaGraphDef protocol buffer as input. If the argument is a file containing a MetaGraphDef protocol buffer , it constructs a protocol buffer from the file content. The function then adds all the nodes from the graph_def field to the current graph, recreates all the collections, and returns a saver constructed from the saver_def field. In combination with export_meta_graph(), this function can be used to  Serialize a graph along with other Python objects such as QueueRunner, Variable into a MetaGraphDef. Restart training from a saved graph and checkpoints. Run inference from a saved graph and checkpoints.  ...
# Create a saver.
saver = tf.compat.v1.train.Saver(...variables...)
# Remember the training_op we want to run by adding it to a collection.
tf.compat.v1.add_to_collection('train_op', train_op)
sess = tf.compat.v1.Session()
for step in xrange(1000000):
    sess.run(train_op)
    if step % 1000 == 0:
        # Saves checkpoint, which by default also exports a meta_graph
        # named 'my-model-global_step.meta'.
        saver.save(sess, 'my-model', global_step=step)
 Later we can continue training from this saved meta_graph without building the model from scratch. with tf.Session() as sess:
  new_saver =
  tf.train.import_meta_graph('my-save-dir/my-model-10000.meta')
  new_saver.restore(sess, 'my-save-dir/my-model-10000')
  # tf.get_collection() returns a list. In this example we only want
  # the first one.
  train_op = tf.get_collection('train_op')[0]
  for step in xrange(1000000):
    sess.run(train_op)

Note: Restarting training from saved meta_graph only works if the device assignments have not changed.
 Example: Variables, placeholders, and independent operations can also be stored, as shown in the following example. # Saving contents and operations.
v1 = tf.placeholder(tf.float32, name="v1")
v2 = tf.placeholder(tf.float32, name="v2")
v3 = tf.math.multiply(v1, v2)
vx = tf.Variable(10.0, name="vx")
v4 = tf.add(v3, vx, name="v4")
saver = tf.train.Saver([vx])
sess = tf.Session()
sess.run(tf.global_variables_initializer())
sess.run(vx.assign(tf.add(vx, vx)))
result = sess.run(v4, feed_dict={v1:12.0, v2:3.3})
print(result)
saver.save(sess, "./model_ex1")
 Later this model can be restored and contents loaded. # Restoring variables and running operations.
saver = tf.train.import_meta_graph("./model_ex1.meta")
sess = tf.Session()
saver.restore(sess, "./model_ex1")
result = sess.run("v4:0", feed_dict={"v1:0": 12.0, "v2:0": 3.3})
print(result)

 Args
  meta_graph_or_file   MetaGraphDef protocol buffer or filename (including the path) containing a MetaGraphDef.
  clear_devices   Whether or not to clear the device field for an Operation or Tensor during import.
  import_scope   Optional string. Name scope to add. Only used when initializing from protocol buffer.
  **kwargs   Optional keyed arguments.

 Returns   A saver constructed from saver_def in MetaGraphDef or None. A None value is returned if no variables exist in the MetaGraphDef (i.e., there are no variables to restore).

 Raises
  RuntimeError   If called with eager execution enabled.    Eager Compatibility Exporting/importing meta graphs is not supported. No graph exists when eager execution is enabled.


tf.compat.v1.train.init_from_checkpoint Replaces tf.Variable initializers so they load from a checkpoint file.
tf.compat.v1.train.init_from_checkpoint(
    ckpt_dir_or_file, assignment_map
)
 Values are not loaded immediately, but when the initializer is run (typically by running a tf.compat.v1.global_variables_initializer op).
Note: This overrides default initialization ops of specified variables and redefines dtype.
 Assignment map supports following syntax:
'checkpoint_scope_name/': 'scope_name/' - will load all variables in current scope_name from checkpoint_scope_name with matching tensor names.
'checkpoint_scope_name/some_other_variable': 'scope_name/variable_name' - will initialize scope_name/variable_name variable from checkpoint_scope_name/some_other_variable.
'scope_variable_name': variable - will initialize given tf.Variable object with tensor 'scope_variable_name' from the checkpoint.
'scope_variable_name': list(variable) - will initialize list of partitioned variables with tensor 'scope_variable_name' from the checkpoint.
'/': 'scope_name/' - will load all variables in current scope_name from checkpoint's root (e.g. no scope).  Supports loading into partitioned variables, which are represented as '<variable>/part_<part #>'. Example:
# Say, '/tmp/model.ckpt' has the following tensors:
#  -- name='old_scope_1/var1', shape=[20, 2]
#  -- name='old_scope_1/var2', shape=[50, 4]
#  -- name='old_scope_2/var3', shape=[100, 100]

# Create new model's variables
with tf.compat.v1.variable_scope('new_scope_1'):
  var1 = tf.compat.v1.get_variable('var1', shape=[20, 2],
                         initializer=tf.compat.v1.zeros_initializer())
with tf.compat.v1.variable_scope('new_scope_2'):
  var2 = tf.compat.v1.get_variable('var2', shape=[50, 4],
                         initializer=tf.compat.v1.zeros_initializer())
  # Partition into 5 variables along the first axis.
  var3 = tf.compat.v1.get_variable(name='var3', shape=[100, 100],
                         initializer=tf.compat.v1.zeros_initializer(),
                         partitioner=lambda shape, dtype: [5, 1])

# Initialize all variables in `new_scope_1` from `old_scope_1`.
init_from_checkpoint('/tmp/model.ckpt', {'old_scope_1/': 'new_scope_1'})

# Use names to specify which variables to initialize from checkpoint.
init_from_checkpoint('/tmp/model.ckpt',
                     {'old_scope_1/var1': 'new_scope_1/var1',
                      'old_scope_1/var2': 'new_scope_2/var2'})

# Or use tf.Variable objects to identify what to initialize.
init_from_checkpoint('/tmp/model.ckpt',
                     {'old_scope_1/var1': var1,
                      'old_scope_1/var2': var2})

# Initialize partitioned variables using variable's name
init_from_checkpoint('/tmp/model.ckpt',
                     {'old_scope_2/var3': 'new_scope_2/var3'})

# Or specify the list of tf.Variable objects.
init_from_checkpoint('/tmp/model.ckpt',
                     {'old_scope_2/var3': var3._get_variable_list()})

 Args
  ckpt_dir_or_file   Directory with checkpoints file or path to checkpoint.
  assignment_map   Dict, where keys are names of the variables in the checkpoint and values are current variables or names of current variables (in default graph).

 Raises
  ValueError   If missing variables in current graph, or if missing checkpoints or tensors in checkpoints.


tf.compat.v1.train.input_producer Output the rows of input_tensor to a queue for an input pipeline. (deprecated)
tf.compat.v1.train.input_producer(
    input_tensor, element_shape=None, num_epochs=None, shuffle=True, seed=None,
    capacity=32, shared_name=None, summary_name=None, name=None, cancel_op=None
)
 Warning: THIS FUNCTION IS DEPRECATED. It will be removed in a future version. Instructions for updating: Queue-based input pipelines have been replaced by tf.data. Use tf.data.Dataset.from_tensor_slices(input_tensor).shuffle(tf.shape(input_tensor, out_type=tf.int64)[0]).repeat(num_epochs). If shuffle=False, omit the .shuffle(...).
Note: if num_epochs is not None, this function creates local counter epochs. Use local_variables_initializer() to initialize local variables.

 Args
  input_tensor   A tensor with the rows to produce. Must be at least one-dimensional. Must either have a fully-defined shape, or element_shape must be defined.
  element_shape   (Optional.) A TensorShape representing the shape of a row of input_tensor, if it cannot be inferred.
  num_epochs   (Optional.) An integer. If specified input_producer produces each row of input_tensor num_epochs times before generating an OutOfRange error. If not specified, input_producer can cycle through the rows of input_tensor an unlimited number of times.
  shuffle   (Optional.) A boolean. If true, the rows are randomly shuffled within each epoch.
  seed   (Optional.) An integer. The seed to use if shuffle is true.
  capacity   (Optional.) The capacity of the queue to be used for buffering the input.
  shared_name   (Optional.) If set, this queue will be shared under the given name across multiple sessions.
  summary_name   (Optional.) If set, a scalar summary for the current queue size will be generated, using this name as part of the tag.
  name   (Optional.) A name for queue.
  cancel_op   (Optional.) Cancel op for the queue

 Returns   A queue with the output rows. A QueueRunner for the queue is added to the current QUEUE_RUNNER collection of the current graph.

 Raises
  ValueError   If the shape of the input cannot be inferred from the arguments.
  RuntimeError   If called with eager execution enabled.    Eager Compatibility Input pipelines based on Queues are not supported when eager execution is enabled. Please use the tf.data API to ingest data under eager execution.


tf.compat.v1.train.inverse_time_decay Applies inverse time decay to the initial learning rate.
tf.compat.v1.train.inverse_time_decay(
    learning_rate, global_step, decay_steps, decay_rate, staircase=False, name=None
)
 When training a model, it is often recommended to lower the learning rate as the training progresses. This function applies an inverse decay function to a provided initial learning rate. It requires an global_step value to compute the decayed learning rate. You can just pass a TensorFlow variable that you increment at each training step. The function returns the decayed learning rate. It is computed as: decayed_learning_rate = learning_rate / (1 + decay_rate * global_step /
decay_step)
 or, if staircase is True, as: decayed_learning_rate = learning_rate / (1 + decay_rate * floor(global_step /
decay_step))
 Example: decay 1/t with a rate of 0.5: ...
global_step = tf.Variable(0, trainable=False)
learning_rate = 0.1
decay_steps = 1.0
decay_rate = 0.5
learning_rate = tf.compat.v1.train.inverse_time_decay(learning_rate,
global_step,
decay_steps, decay_rate)

# Passing global_step to minimize() will increment it at each step.
learning_step = (
    tf.compat.v1.train.GradientDescentOptimizer(learning_rate)
    .minimize(...my loss..., global_step=global_step)
)

 Args
  learning_rate   A scalar float32 or float64 Tensor or a Python number. The initial learning rate.
  global_step   A Python number. Global step to use for the decay computation. Must not be negative.
  decay_steps   How often to apply decay.
  decay_rate   A Python number. The decay rate.
  staircase   Whether to apply decay in a discrete staircase, as opposed to continuous, fashion.
  name   String. Optional name of the operation. Defaults to 'InverseTimeDecay'.

 Returns   A scalar Tensor of the same type as learning_rate. The decayed learning rate.

 Raises
  ValueError   if global_step is not supplied.    Eager Compatibility When eager execution is enabled, this function returns a function which in turn returns the decayed learning rate Tensor. This can be useful for changing the learning rate value across different invocations of optimizer functions.


tf.compat.v1.train.limit_epochs Returns tensor num_epochs times and then raises an OutOfRange error. (deprecated)
tf.compat.v1.train.limit_epochs(
    tensor, num_epochs=None, name=None
)
 Warning: THIS FUNCTION IS DEPRECATED. It will be removed in a future version. Instructions for updating: Queue-based input pipelines have been replaced by tf.data. Use tf.data.Dataset.from_tensors(tensor).repeat(num_epochs).
Note: creates local counter epochs. Use local_variables_initializer() to initialize local variables.

 Args
  tensor   Any Tensor.
  num_epochs   A positive integer (optional). If specified, limits the number of steps the output tensor may be evaluated.
  name   A name for the operations (optional).

 Returns   tensor or OutOfRange.

 Raises
  ValueError   if num_epochs is invalid.


tf.compat.v1.train.linear_cosine_decay Applies linear cosine decay to the learning rate.
tf.compat.v1.train.linear_cosine_decay(
    learning_rate, global_step, decay_steps, num_periods=0.5, alpha=0.0, beta=0.001,
    name=None
)
 Note that linear cosine decay is more aggressive than cosine decay and larger initial learning rates can typically be used. When training a model, it is often recommended to lower the learning rate as the training progresses. This function applies a linear cosine decay function to a provided initial learning rate. It requires a global_step value to compute the decayed learning rate. You can just pass a TensorFlow variable that you increment at each training step. The function returns the decayed learning rate. It is computed as: global_step = min(global_step, decay_steps)
linear_decay = (decay_steps - global_step) / decay_steps)
cosine_decay = 0.5 * (
    1 + cos(pi * 2 * num_periods * global_step / decay_steps))
decayed = (alpha + linear_decay) * cosine_decay + beta
decayed_learning_rate = learning_rate * decayed
 Example usage: decay_steps = 1000
lr_decayed = linear_cosine_decay(learning_rate, global_step, decay_steps)

 Args
  learning_rate   A scalar float32 or float64 Tensor or a Python number. The initial learning rate.
  global_step   A scalar int32 or int64 Tensor or a Python number. Global step to use for the decay computation.
  decay_steps   A scalar int32 or int64 Tensor or a Python number. Number of steps to decay over.
  num_periods   Number of periods in the cosine part of the decay. See computation above.
  alpha   See computation above.
  beta   See computation above.
  name   String. Optional name of the operation. Defaults to 'LinearCosineDecay'.

 Returns   A scalar Tensor of the same type as learning_rate. The decayed learning rate.

 Raises
  ValueError   if global_step is not supplied.    References: Neural Optimizer Search with Reinforcement Learning: Bello et al., 2017 (pdf) Stochastic Gradient Descent with Warm Restarts: Loshchilov et al., 2017 (pdf) Eager Compatibility When eager execution is enabled, this function returns a function which in turn returns the decayed learning rate Tensor. This can be useful for changing the learning rate value across different invocations of optimizer functions.


tf.compat.v1.train.LooperThread A thread that runs code repeatedly, optionally on a timer.
tf.compat.v1.train.LooperThread(
    coord, timer_interval_secs, target=None, args=None, kwargs=None
)
 This thread class is intended to be used with a Coordinator. It repeatedly runs code specified either as target and args or by the run_loop() method. Before each run the thread checks if the coordinator has requested stop. In that case the looper thread terminates immediately. If the code being run raises an exception, that exception is reported to the coordinator and the thread terminates. The coordinator will then request all the other threads it coordinates to stop. You typically pass looper threads to the supervisor Join() method.

 Args
  coord   A Coordinator.
  timer_interval_secs   Time boundaries at which to call Run(), or None if it should be called back to back.
  target   Optional callable object that will be executed in the thread.
  args   Optional arguments to pass to target when calling it.
  kwargs   Optional keyword arguments to pass to target when calling it.

 Raises
  ValueError   If one of the arguments is invalid.

 Attributes
  daemon   A boolean value indicating whether this thread is a daemon thread. This must be set before start() is called, otherwise RuntimeError is raised. Its initial value is inherited from the creating thread; the main thread is not a daemon thread and therefore all threads created in the main thread default to daemon = False. The entire Python program exits when only daemon threads are left.

  ident   Thread identifier of this thread or None if it has not been started. This is a nonzero integer. See the get_ident() function. Thread identifiers may be recycled when a thread exits and another thread is created. The identifier is available even after the thread has exited.

  name   A string used for identification purposes only. It has no semantics. Multiple threads may be given the same name. The initial name is set by the constructor.
   Methods getName
getName()
 isAlive
isAlive()
 Return whether the thread is alive. This method is deprecated, use is_alive() instead. isDaemon
isDaemon()
 is_alive
is_alive()
 Return whether the thread is alive. This method returns True just before the run() method starts until just after the run() method terminates. The module function enumerate() returns a list of all alive threads. join
join(
    timeout=None
)
 Wait until the thread terminates. This blocks the calling thread until the thread whose join() method is called terminates -- either normally or through an unhandled exception or until the optional timeout occurs. When the timeout argument is present and not None, it should be a floating point number specifying a timeout for the operation in seconds (or fractions thereof). As join() always returns None, you must call is_alive() after join() to decide whether a timeout happened -- if the thread is still alive, the join() call timed out. When the timeout argument is not present or None, the operation will block until the thread terminates. A thread can be join()ed many times. join() raises a RuntimeError if an attempt is made to join the current thread as that would cause a deadlock. It is also an error to join() a thread before it has been started and attempts to do so raises the same exception. loop View source
@staticmethod
loop(
    coord, timer_interval_secs, target, args=None, kwargs=None
)
 Start a LooperThread that calls a function periodically. If timer_interval_secs is None the thread calls target(args) repeatedly. Otherwise target(args) is called every timer_interval_secs seconds. The thread terminates when a stop of the coordinator is requested.

 Args
  coord   A Coordinator.
  timer_interval_secs   Number. Time boundaries at which to call target.
  target   A callable object.
  args   Optional arguments to pass to target when calling it.
  kwargs   Optional keyword arguments to pass to target when calling it.

 Returns   The started thread.
 run View source
run()
 Method representing the thread's activity. You may override this method in a subclass. The standard run() method invokes the callable object passed to the object's constructor as the target argument, if any, with sequential and keyword arguments taken from the args and kwargs arguments, respectively. run_loop View source
run_loop()
 Called at 'timer_interval_secs' boundaries. setDaemon
setDaemon(
    daemonic
)
 setName
setName(
    name
)
 start
start()
 Start the thread's activity. It must be called at most once per thread object. It arranges for the object's run() method to be invoked in a separate thread of control. This method will raise a RuntimeError if called more than once on the same thread object. start_loop View source
start_loop()
 Called when the thread starts. stop_loop View source
stop_loop()
 Called when the thread stops.


tf.compat.v1.train.maybe_batch Conditionally creates batches of tensors based on keep_input. (deprecated)
tf.compat.v1.train.maybe_batch(
    tensors, keep_input, batch_size, num_threads=1, capacity=32, enqueue_many=False,
    shapes=None, dynamic_pad=False, allow_smaller_final_batch=False,
    shared_name=None, name=None
)
 Warning: THIS FUNCTION IS DEPRECATED. It will be removed in a future version. Instructions for updating: Queue-based input pipelines have been replaced by tf.data. Use tf.data.Dataset.filter(...).batch(batch_size) (or padded_batch(...) if dynamic_pad=True). See docstring in batch for more details.

 Args
  tensors   The list or dictionary of tensors to enqueue.
  keep_input   A bool Tensor. This tensor controls whether the input is added to the queue or not. If it is a scalar and evaluates True, then tensors are all added to the queue. If it is a vector and enqueue_many is True, then each example is added to the queue only if the corresponding value in keep_input is True. This tensor essentially acts as a filtering mechanism.
  batch_size   The new batch size pulled from the queue.
  num_threads   The number of threads enqueuing tensors. The batching will be nondeterministic if num_threads > 1.
  capacity   An integer. The maximum number of elements in the queue.
  enqueue_many   Whether each tensor in tensors is a single example.
  shapes   (Optional) The shapes for each example. Defaults to the inferred shapes for tensors.
  dynamic_pad   Boolean. Allow variable dimensions in input shapes. The given dimensions are padded upon dequeue so that tensors within a batch have the same shapes.
  allow_smaller_final_batch   (Optional) Boolean. If True, allow the final batch to be smaller if there are insufficient items left in the queue.
  shared_name   (Optional). If set, this queue will be shared under the given name across multiple sessions.
  name   (Optional) A name for the operations.

 Returns   A list or dictionary of tensors with the same types as tensors.

 Raises
  ValueError   If the shapes are not specified, and cannot be inferred from the elements of tensors.


tf.compat.v1.train.maybe_batch_join Runs a list of tensors to conditionally fill a queue to create batches. (deprecated)
tf.compat.v1.train.maybe_batch_join(
    tensors_list, keep_input, batch_size, capacity=32, enqueue_many=False,
    shapes=None, dynamic_pad=False, allow_smaller_final_batch=False,
    shared_name=None, name=None
)
 Warning: THIS FUNCTION IS DEPRECATED. It will be removed in a future version. Instructions for updating: Queue-based input pipelines have been replaced by tf.data. Use tf.data.Dataset.interleave(...).filter(...).batch(batch_size) (or padded_batch(...) if dynamic_pad=True). See docstring in batch_join for more details.

 Args
  tensors_list   A list of tuples or dictionaries of tensors to enqueue.
  keep_input   A bool Tensor. This tensor controls whether the input is added to the queue or not. If it is a scalar and evaluates True, then tensors are all added to the queue. If it is a vector and enqueue_many is True, then each example is added to the queue only if the corresponding value in keep_input is True. This tensor essentially acts as a filtering mechanism.
  batch_size   An integer. The new batch size pulled from the queue.
  capacity   An integer. The maximum number of elements in the queue.
  enqueue_many   Whether each tensor in tensor_list_list is a single example.
  shapes   (Optional) The shapes for each example. Defaults to the inferred shapes for tensor_list_list[i].
  dynamic_pad   Boolean. Allow variable dimensions in input shapes. The given dimensions are padded upon dequeue so that tensors within a batch have the same shapes.
  allow_smaller_final_batch   (Optional) Boolean. If True, allow the final batch to be smaller if there are insufficient items left in the queue.
  shared_name   (Optional) If set, this queue will be shared under the given name across multiple sessions.
  name   (Optional) A name for the operations.

 Returns   A list or dictionary of tensors with the same number and types as tensors_list[i].

 Raises
  ValueError   If the shapes are not specified, and cannot be inferred from the elements of tensor_list_list.


tf.compat.v1.train.maybe_shuffle_batch Creates batches by randomly shuffling conditionally-enqueued tensors. (deprecated)
tf.compat.v1.train.maybe_shuffle_batch(
    tensors, batch_size, capacity, min_after_dequeue, keep_input, num_threads=1,
    seed=None, enqueue_many=False, shapes=None, allow_smaller_final_batch=False,
    shared_name=None, name=None
)
 Warning: THIS FUNCTION IS DEPRECATED. It will be removed in a future version. Instructions for updating: Queue-based input pipelines have been replaced by tf.data. Use tf.data.Dataset.filter(...).shuffle(min_after_dequeue).batch(batch_size). See docstring in shuffle_batch for more details.

 Args
  tensors   The list or dictionary of tensors to enqueue.
  batch_size   The new batch size pulled from the queue.
  capacity   An integer. The maximum number of elements in the queue.
  min_after_dequeue   Minimum number elements in the queue after a dequeue, used to ensure a level of mixing of elements.
  keep_input   A bool Tensor. This tensor controls whether the input is added to the queue or not. If it is a scalar and evaluates True, then tensors are all added to the queue. If it is a vector and enqueue_many is True, then each example is added to the queue only if the corresponding value in keep_input is True. This tensor essentially acts as a filtering mechanism.
  num_threads   The number of threads enqueuing tensor_list.
  seed   Seed for the random shuffling within the queue.
  enqueue_many   Whether each tensor in tensor_list is a single example.
  shapes   (Optional) The shapes for each example. Defaults to the inferred shapes for tensor_list.
  allow_smaller_final_batch   (Optional) Boolean. If True, allow the final batch to be smaller if there are insufficient items left in the queue.
  shared_name   (Optional) If set, this queue will be shared under the given name across multiple sessions.
  name   (Optional) A name for the operations.

 Returns   A list or dictionary of tensors with the types as tensors.

 Raises
  ValueError   If the shapes are not specified, and cannot be inferred from the elements of tensors.    Eager Compatibility Input pipelines based on Queues are not supported when eager execution is enabled. Please use the tf.data API to ingest data under eager execution.


tf.compat.v1.train.maybe_shuffle_batch_join Create batches by randomly shuffling conditionally-enqueued tensors. (deprecated)
tf.compat.v1.train.maybe_shuffle_batch_join(
    tensors_list, batch_size, capacity, min_after_dequeue, keep_input, seed=None,
    enqueue_many=False, shapes=None, allow_smaller_final_batch=False,
    shared_name=None, name=None
)
 Warning: THIS FUNCTION IS DEPRECATED. It will be removed in a future version. Instructions for updating: Queue-based input pipelines have been replaced by tf.data. Use tf.data.Dataset.interleave(...).filter(...).shuffle(min_after_dequeue).batch(batch_size). See docstring in shuffle_batch_join for more details.

 Args
  tensors_list   A list of tuples or dictionaries of tensors to enqueue.
  batch_size   An integer. The new batch size pulled from the queue.
  capacity   An integer. The maximum number of elements in the queue.
  min_after_dequeue   Minimum number elements in the queue after a dequeue, used to ensure a level of mixing of elements.
  keep_input   A bool Tensor. This tensor controls whether the input is added to the queue or not. If it is a scalar and evaluates True, then tensors are all added to the queue. If it is a vector and enqueue_many is True, then each example is added to the queue only if the corresponding value in keep_input is True. This tensor essentially acts as a filtering mechanism.
  seed   Seed for the random shuffling within the queue.
  enqueue_many   Whether each tensor in tensor_list_list is a single example.
  shapes   (Optional) The shapes for each example. Defaults to the inferred shapes for tensors_list[i].
  allow_smaller_final_batch   (Optional) Boolean. If True, allow the final batch to be smaller if there are insufficient items left in the queue.
  shared_name   (optional). If set, this queue will be shared under the given name across multiple sessions.
  name   (Optional) A name for the operations.

 Returns   A list or dictionary of tensors with the same number and types as tensors_list[i].

 Raises
  ValueError   If the shapes are not specified, and cannot be inferred from the elements of tensors_list.    Eager Compatibility Input pipelines based on Queues are not supported when eager execution is enabled. Please use the tf.data API to ingest data under eager execution.


tf.compat.v1.train.MomentumOptimizer Optimizer that implements the Momentum algorithm. Inherits From: Optimizer
tf.compat.v1.train.MomentumOptimizer(
    learning_rate, momentum, use_locking=False, name='Momentum',
    use_nesterov=False
)
 Computes (if use_nesterov = False): accumulation = momentum * accumulation + gradient
variable -= learning_rate * accumulation
 Note that in the dense version of this algorithm, accumulation is updated and applied regardless of a gradient's value, whereas the sparse version (when the gradient is an IndexedSlices, typically because of tf.gather or an embedding) only updates variable slices and corresponding accumulation terms when that part of the variable was used in the forward pass.

 Args
  learning_rate   A Tensor or a floating point value. The learning rate.
  momentum   A Tensor or a floating point value. The momentum.
  use_locking   If True use locks for update operations.
  name   Optional name prefix for the operations created when applying gradients. Defaults to "Momentum".
  use_nesterov   If True use Nesterov Momentum. See (Sutskever et al., 2013). This implementation always computes gradients at the value of the variable(s) passed to the optimizer. Using Nesterov Momentum makes the variable(s) track the values called theta_t + mu*v_t in the paper. This implementation is an approximation of the original formula, valid for high values of momentum. It will compute the "adjusted gradient" in NAG by assuming that the new gradient will be estimated by the current average gradient plus the product of momentum and the change in the average gradient.    Methods apply_gradients View source
apply_gradients(
    grads_and_vars, global_step=None, name=None
)
 Apply gradients to variables. This is the second part of minimize(). It returns an Operation that applies gradients.

 Args
  grads_and_vars   List of (gradient, variable) pairs as returned by compute_gradients().
  global_step   Optional Variable to increment by one after the variables have been updated.
  name   Optional name for the returned operation. Default to the name passed to the Optimizer constructor.

 Returns   An Operation that applies the specified gradients. If global_step was not None, that operation also increments global_step.

 Raises
  TypeError   If grads_and_vars is malformed.
  ValueError   If none of the variables have gradients.
  RuntimeError   If you should use _distributed_apply() instead.    compute_gradients View source
compute_gradients(
    loss, var_list=None, gate_gradients=GATE_OP, aggregation_method=None,
    colocate_gradients_with_ops=False, grad_loss=None
)
 Compute gradients of loss for the variables in var_list. This is the first part of minimize(). It returns a list of (gradient, variable) pairs where "gradient" is the gradient for "variable". Note that "gradient" can be a Tensor, an IndexedSlices, or None if there is no gradient for the given variable.

 Args
  loss   A Tensor containing the value to minimize or a callable taking no arguments which returns the value to minimize. When eager execution is enabled it must be a callable.
  var_list   Optional list or tuple of tf.Variable to update to minimize loss. Defaults to the list of variables collected in the graph under the key GraphKeys.TRAINABLE_VARIABLES.
  gate_gradients   How to gate the computation of gradients. Can be GATE_NONE, GATE_OP, or GATE_GRAPH.
  aggregation_method   Specifies the method used to combine gradient terms. Valid values are defined in the class AggregationMethod.
  colocate_gradients_with_ops   If True, try colocating gradients with the corresponding op.
  grad_loss   Optional. A Tensor holding the gradient computed for loss.

 Returns   A list of (gradient, variable) pairs. Variable is always present, but gradient can be None.

 Raises
  TypeError   If var_list contains anything else than Variable objects.
  ValueError   If some arguments are invalid.
  RuntimeError   If called with eager execution enabled and loss is not callable.    Eager Compatibility When eager execution is enabled, gate_gradients, aggregation_method, and colocate_gradients_with_ops are ignored. get_name View source
get_name()
 get_slot View source
get_slot(
    var, name
)
 Return a slot named name created for var by the Optimizer. Some Optimizer subclasses use additional variables. For example Momentum and Adagrad use variables to accumulate updates. This method gives access to these Variable objects if for some reason you need them. Use get_slot_names() to get the list of slot names created by the Optimizer.

 Args
  var   A variable passed to minimize() or apply_gradients().
  name   A string.

 Returns   The Variable for the slot if it was created, None otherwise.
 get_slot_names View source
get_slot_names()
 Return a list of the names of slots created by the Optimizer. See get_slot().

 Returns   A list of strings.
 minimize View source
minimize(
    loss, global_step=None, var_list=None, gate_gradients=GATE_OP,
    aggregation_method=None, colocate_gradients_with_ops=False, name=None,
    grad_loss=None
)
 Add operations to minimize loss by updating var_list. This method simply combines calls compute_gradients() and apply_gradients(). If you want to process the gradient before applying them call compute_gradients() and apply_gradients() explicitly instead of using this function.

 Args
  loss   A Tensor containing the value to minimize.
  global_step   Optional Variable to increment by one after the variables have been updated.
  var_list   Optional list or tuple of Variable objects to update to minimize loss. Defaults to the list of variables collected in the graph under the key GraphKeys.TRAINABLE_VARIABLES.
  gate_gradients   How to gate the computation of gradients. Can be GATE_NONE, GATE_OP, or GATE_GRAPH.
  aggregation_method   Specifies the method used to combine gradient terms. Valid values are defined in the class AggregationMethod.
  colocate_gradients_with_ops   If True, try colocating gradients with the corresponding op.
  name   Optional name for the returned operation.
  grad_loss   Optional. A Tensor holding the gradient computed for loss.

 Returns   An Operation that updates the variables in var_list. If global_step was not None, that operation also increments global_step.

 Raises
  ValueError   If some of the variables are not Variable objects.    Eager Compatibility When eager execution is enabled, loss should be a Python function that takes no arguments and computes the value to be minimized. Minimization (and gradient computation) is done with respect to the elements of var_list if not None, else with respect to any trainable variables created during the execution of the loss function. gate_gradients, aggregation_method, colocate_gradients_with_ops and grad_loss are ignored when eager execution is enabled. variables View source
variables()
 A list of variables which encode the current state of Optimizer. Includes slot variables and additional global variables created by the optimizer in the current default graph.

 Returns   A list of variables.

 Class Variables
  GATE_GRAPH   2
  GATE_NONE   0
  GATE_OP   1


tf.compat.v1.train.MonitoredSession Session-like object that handles initialization, recovery and hooks.
tf.compat.v1.train.MonitoredSession(
    session_creator=None, hooks=None, stop_grace_period_secs=120
)
 Example usage: saver_hook = CheckpointSaverHook(...)
summary_hook = SummarySaverHook(...)
with MonitoredSession(session_creator=ChiefSessionCreator(...),
                      hooks=[saver_hook, summary_hook]) as sess:
  while not sess.should_stop():
    sess.run(train_op)
 Initialization: At creation time the monitored session does following things in given order:  calls hook.begin() for each given hook finalizes the graph via scaffold.finalize()
 create session initializes the model via initialization ops provided by Scaffold
 restores variables if a checkpoint exists launches queue runners calls hook.after_create_session()
  Run: When run() is called, the monitored session does following things:  calls hook.before_run()
 calls TensorFlow session.run() with merged fetches and feed_dict calls hook.after_run()
 returns result of session.run() asked by user if AbortedError or UnavailableError occurs, it recovers or reinitializes the session before executing the run() call again  Exit: At the close(), the monitored session does following things in order:  calls hook.end()
 closes the queue runners and the session suppresses OutOfRange error which indicates that all inputs have been processed if the monitored_session is used as a context  How to set tf.compat.v1.Session arguments:  In most cases you can set session arguments as follows:  MonitoredSession(
  session_creator=ChiefSessionCreator(master=..., config=...))
  In distributed setting for a non-chief worker, you can use following:  MonitoredSession(
  session_creator=WorkerSessionCreator(master=..., config=...))
 See MonitoredTrainingSession for an example usage based on chief or worker.
Note: This is not a tf.compat.v1.Session. For example, it cannot do following:
  it cannot be set as default session. it cannot be sent to saver.save. it cannot be sent to tf.train.start_queue_runners.

 Args
  session_creator   A factory object to create session. Typically a ChiefSessionCreator which is the default one.
  hooks   An iterable of `SessionRunHook' objects.

 Returns   A MonitoredSession object.

 Attributes
  graph   The graph that was launched in this session.    Child Classes class StepContext Methods close View source
close()
 run View source
run(
    fetches, feed_dict=None, options=None, run_metadata=None
)
 Run ops in the monitored session. This method is completely compatible with the tf.Session.run() method.

 Args
  fetches   Same as tf.Session.run().
  feed_dict   Same as tf.Session.run().
  options   Same as tf.Session.run().
  run_metadata   Same as tf.Session.run().

 Returns   Same as tf.Session.run().
 run_step_fn View source
run_step_fn(
    step_fn
)
 Run ops using a step function.

 Args
  step_fn   A function or a method with a single argument of type StepContext. The function may use methods of the argument to perform computations with access to a raw session. The returned value of the step_fn will be returned from run_step_fn, unless a stop is requested. In that case, the next should_stop call will return True. Example usage: with tf.Graph().as_default():
c = tf.compat.v1.placeholder(dtypes.float32)
v = tf.add(c, 4.0)
w = tf.add(c, 0.5)
def step_fn(step_context):
a = step_context.session.run(fetches=v, feed_dict={c: 0.5})
if a <= 4.5:
step_context.request_stop()
return step_context.run_with_hooks(fetches=w,
feed_dict={c: 0.1})

with tf.MonitoredSession() as session:
while not session.should_stop():
a = session.run_step_fn(step_fn)
 Hooks interact with the run_with_hooks() call inside the step_fn as they do with a MonitoredSession.run call.

 Returns   Returns the returned value of step_fn.

 Raises
  StopIteration   if step_fn has called request_stop(). It may be caught by with tf.MonitoredSession() to close the session.
  ValueError   if step_fn doesn't have a single argument called step_context. It may also optionally have self for cases when it belongs to an object.    should_stop View source
should_stop()
 __enter__ View source
__enter__()
 __exit__ View source
__exit__(
    exception_type, exception_value, traceback
)


tf.compat.v1.train.MonitoredSession.StepContext Control flow instrument for the step_fn from run_step_fn().  View aliases  Compat aliases for migration
See Migration guide for more details. tf.compat.v1.train.SingularMonitoredSession.StepContext
tf.compat.v1.train.MonitoredSession.StepContext(
    session, run_with_hooks_fn
)
 Users of step_fn may perform run() calls without running hooks by accessing the session. A run() call with hooks may be performed using run_with_hooks(). Computation flow can be interrupted using request_stop().

 Args
  session   An instance of tf.compat.v1.Session.
  run_with_hooks_fn   A function for running fetches and hooks.

 Attributes
  session
   Methods request_stop View source
request_stop()
 Exit the training loop by causing should_stop() to return True. Causes step_fn to exit by raising an exception.

 Raises   StopIteration
 run_with_hooks View source
run_with_hooks(
    *args, **kwargs
)
 Same as MonitoredSession.run. Accepts the same arguments.


tf.compat.v1.train.MonitoredTrainingSession Creates a MonitoredSession for training.
tf.compat.v1.train.MonitoredTrainingSession(
    master='', is_chief=True, checkpoint_dir=None, scaffold=None,
    hooks=None, chief_only_hooks=None, save_checkpoint_secs=USE_DEFAULT,
    save_summaries_steps=USE_DEFAULT, save_summaries_secs=USE_DEFAULT, config=None,
    stop_grace_period_secs=120, log_step_count_steps=100, max_wait_secs=7200,
    save_checkpoint_steps=USE_DEFAULT, summary_dir=None, save_graph_def=True
)
 For a chief, this utility sets proper session initializer/restorer. It also creates hooks related to checkpoint and summary saving. For workers, this utility sets proper session creator which waits for the chief to initialize/restore. Please check tf.compat.v1.train.MonitoredSession for more information.

 Args
  master   String the TensorFlow master to use.
  is_chief   If True, it will take care of initialization and recovery the underlying TensorFlow session. If False, it will wait on a chief to initialize or recover the TensorFlow session.
  checkpoint_dir   A string. Optional path to a directory where to restore variables.
  scaffold   A Scaffold used for gathering or building supportive ops. If not specified, a default one is created. It's used to finalize the graph.
  hooks   Optional list of SessionRunHook objects.
  chief_only_hooks   list of SessionRunHook objects. Activate these hooks if is_chief==True, ignore otherwise.
  save_checkpoint_secs   The frequency, in seconds, that a checkpoint is saved using a default checkpoint saver. If both save_checkpoint_steps and save_checkpoint_secs are set to None, then the default checkpoint saver isn't used. If both are provided, then only save_checkpoint_secs is used. Default 600.
  save_summaries_steps   The frequency, in number of global steps, that the summaries are written to disk using a default summary saver. If both save_summaries_steps and save_summaries_secs are set to None, then the default summary saver isn't used. Default 100.
  save_summaries_secs   The frequency, in secs, that the summaries are written to disk using a default summary saver. If both save_summaries_steps and save_summaries_secs are set to None, then the default summary saver isn't used. Default not enabled.
  config   an instance of tf.compat.v1.ConfigProto proto used to configure the session. It's the config argument of constructor of tf.compat.v1.Session.
  stop_grace_period_secs   Number of seconds given to threads to stop after close() has been called.
  log_step_count_steps   The frequency, in number of global steps, that the global step/sec is logged.
  max_wait_secs   Maximum time workers should wait for the session to become available. This should be kept relatively short to help detect incorrect code, but sometimes may need to be increased if the chief takes a while to start up.
  save_checkpoint_steps   The frequency, in number of global steps, that a checkpoint is saved using a default checkpoint saver. If both save_checkpoint_steps and save_checkpoint_secs are set to None, then the default checkpoint saver isn't used. If both are provided, then only save_checkpoint_secs is used. Default not enabled.
  summary_dir   A string. Optional path to a directory where to save summaries. If None, checkpoint_dir is used instead.
  save_graph_def   Whether to save the GraphDef and MetaGraphDef to checkpoint_dir. The GraphDef is saved after the session is created as graph.pbtxt. MetaGraphDefs are saved out for every checkpoint as model.ckpt-*.meta.

 Returns   A MonitoredSession object.


tf.compat.v1.train.natural_exp_decay Applies natural exponential decay to the initial learning rate.
tf.compat.v1.train.natural_exp_decay(
    learning_rate, global_step, decay_steps, decay_rate, staircase=False, name=None
)
 When training a model, it is often recommended to lower the learning rate as the training progresses. This function applies an exponential decay function to a provided initial learning rate. It requires an global_step value to compute the decayed learning rate. You can just pass a TensorFlow variable that you increment at each training step. The function returns the decayed learning rate. It is computed as: decayed_learning_rate = learning_rate * exp(-decay_rate * global_step /
decay_step)
 or, if staircase is True, as: decayed_learning_rate = learning_rate * exp(-decay_rate * floor(global_step /
decay_step))
 Example: decay exponentially with a base of 0.96: ...
global_step = tf.Variable(0, trainable=False)
learning_rate = 0.1
decay_steps = 5
k = 0.5
learning_rate = tf.compat.v1.train.natural_exp_decay(learning_rate,
global_step,
                                           decay_steps, k)

# Passing global_step to minimize() will increment it at each step.
learning_step = (
    tf.compat.v1.train.GradientDescentOptimizer(learning_rate)
    .minimize(...my loss..., global_step=global_step)
)

 Args
  learning_rate   A scalar float32 or float64 Tensor or a Python number. The initial learning rate.
  global_step   A Python number. Global step to use for the decay computation. Must not be negative.
  decay_steps   How often to apply decay.
  decay_rate   A Python number. The decay rate.
  staircase   Whether to apply decay in a discrete staircase, as opposed to continuous, fashion.
  name   String. Optional name of the operation. Defaults to 'ExponentialTimeDecay'.

 Returns   A scalar Tensor of the same type as learning_rate. The decayed learning rate.

 Raises
  ValueError   if global_step is not supplied.    Eager Compatibility When eager execution is enabled, this function returns a function which in turn returns the decayed learning rate Tensor. This can be useful for changing the learning rate value across different invocations of optimizer functions.


tf.compat.v1.train.NewCheckpointReader A function that returns a CheckPointReader.
tf.compat.v1.train.NewCheckpointReader(
    filepattern
)

 Args
  filepattern   The filename.

 Returns   A CheckpointReader object.


tf.compat.v1.train.noisy_linear_cosine_decay Applies noisy linear cosine decay to the learning rate.
tf.compat.v1.train.noisy_linear_cosine_decay(
    learning_rate, global_step, decay_steps, initial_variance=1.0,
    variance_decay=0.55, num_periods=0.5, alpha=0.0, beta=0.001, name=None
)
 Note that linear cosine decay is more aggressive than cosine decay and larger initial learning rates can typically be used. When training a model, it is often recommended to lower the learning rate as the training progresses. This function applies a noisy linear cosine decay function to a provided initial learning rate. It requires a global_step value to compute the decayed learning rate. You can just pass a TensorFlow variable that you increment at each training step. The function returns the decayed learning rate. It is computed as: global_step = min(global_step, decay_steps)
linear_decay = (decay_steps - global_step) / decay_steps)
cosine_decay = 0.5 * (
    1 + cos(pi * 2 * num_periods * global_step / decay_steps))
decayed = (alpha + linear_decay + eps_t) * cosine_decay + beta
decayed_learning_rate = learning_rate * decayed
 where eps_t is 0-centered gaussian noise with variance initial_variance / (1 + global_step) ** variance_decay Example usage: decay_steps = 1000
lr_decayed = noisy_linear_cosine_decay(
  learning_rate, global_step, decay_steps)

 Args
  learning_rate   A scalar float32 or float64 Tensor or a Python number. The initial learning rate.
  global_step   A scalar int32 or int64 Tensor or a Python number. Global step to use for the decay computation.
  decay_steps   A scalar int32 or int64 Tensor or a Python number. Number of steps to decay over.
  initial_variance   initial variance for the noise. See computation above.
  variance_decay   decay for the noise's variance. See computation above.
  num_periods   Number of periods in the cosine part of the decay. See computation above.
  alpha   See computation above.
  beta   See computation above.
  name   String. Optional name of the operation. Defaults to 'NoisyLinearCosineDecay'.

 Returns   A scalar Tensor of the same type as learning_rate. The decayed learning rate.

 Raises
  ValueError   if global_step is not supplied.    References: Neural Optimizer Search with Reinforcement Learning: Bello et al., 2017 (pdf) Stochastic Gradient Descent with Warm Restarts: Loshchilov et al., 2017 (pdf) Eager Compatibility When eager execution is enabled, this function returns a function which in turn returns the decayed learning rate Tensor. This can be useful for changing the learning rate value across different invocations of optimizer functions.


tf.compat.v1.train.Optimizer Base class for optimizers.
tf.compat.v1.train.Optimizer(
    use_locking, name
)
 This class defines the API to add Ops to train a model. You never use this class directly, but instead instantiate one of its subclasses such as GradientDescentOptimizer, AdagradOptimizer, or MomentumOptimizer. Usage # Create an optimizer with the desired parameters.
opt = GradientDescentOptimizer(learning_rate=0.1)
# Add Ops to the graph to minimize a cost by updating a list of variables.
# "cost" is a Tensor, and the list of variables contains tf.Variable
# objects.
opt_op = opt.minimize(cost, var_list=<list of variables>)
 In the training program you will just have to run the returned Op. # Execute opt_op to do one step of training:
opt_op.run()
 Processing gradients before applying them. Calling minimize() takes care of both computing the gradients and applying them to the variables. If you want to process the gradients before applying them you can instead use the optimizer in three steps:  Compute the gradients with compute_gradients(). Process the gradients as you wish. Apply the processed gradients with apply_gradients().  Example: # Create an optimizer.
opt = GradientDescentOptimizer(learning_rate=0.1)

# Compute the gradients for a list of variables.
grads_and_vars = opt.compute_gradients(loss, <list of variables>)

# grads_and_vars is a list of tuples (gradient, variable).  Do whatever you
# need to the 'gradient' part, for example cap them, etc.
capped_grads_and_vars = [(MyCapper(gv[0]), gv[1]) for gv in grads_and_vars]

# Ask the optimizer to apply the capped gradients.
opt.apply_gradients(capped_grads_and_vars)
 Gating Gradients Both minimize() and compute_gradients() accept a gate_gradients argument that controls the degree of parallelism during the application of the gradients. The possible values are: GATE_NONE, GATE_OP, and GATE_GRAPH. GATE_NONE: Compute and apply gradients in parallel. This provides the maximum parallelism in execution, at the cost of some non-reproducibility in the results. For example the two gradients of matmul depend on the input values: With GATE_NONE one of the gradients could be applied to one of the inputs before the other gradient is computed resulting in non-reproducible results. GATE_OP: For each Op, make sure all gradients are computed before they are used. This prevents race conditions for Ops that generate gradients for multiple inputs where the gradients depend on the inputs. GATE_GRAPH: Make sure all gradients for all variables are computed before any one of them is used. This provides the least parallelism but can be useful if you want to process all gradients before applying any of them. Slots Some optimizer subclasses, such as MomentumOptimizer and AdagradOptimizer allocate and manage additional variables associated with the variables to train. These are called Slots. Slots have names and you can ask the optimizer for the names of the slots that it uses. Once you have a slot name you can ask the optimizer for the variable it created to hold the slot value. This can be useful if you want to log debug a training algorithm, report stats about the slots, etc.

 Args
  use_locking   Bool. If True apply use locks to prevent concurrent updates to variables.
  name   A non-empty string. The name to use for accumulators created for the optimizer.

 Raises
  ValueError   If name is malformed.    Methods apply_gradients View source
apply_gradients(
    grads_and_vars, global_step=None, name=None
)
 Apply gradients to variables. This is the second part of minimize(). It returns an Operation that applies gradients.

 Args
  grads_and_vars   List of (gradient, variable) pairs as returned by compute_gradients().
  global_step   Optional Variable to increment by one after the variables have been updated.
  name   Optional name for the returned operation. Default to the name passed to the Optimizer constructor.

 Returns   An Operation that applies the specified gradients. If global_step was not None, that operation also increments global_step.

 Raises
  TypeError   If grads_and_vars is malformed.
  ValueError   If none of the variables have gradients.
  RuntimeError   If you should use _distributed_apply() instead.    compute_gradients View source
compute_gradients(
    loss, var_list=None, gate_gradients=GATE_OP, aggregation_method=None,
    colocate_gradients_with_ops=False, grad_loss=None
)
 Compute gradients of loss for the variables in var_list. This is the first part of minimize(). It returns a list of (gradient, variable) pairs where "gradient" is the gradient for "variable". Note that "gradient" can be a Tensor, an IndexedSlices, or None if there is no gradient for the given variable.

 Args
  loss   A Tensor containing the value to minimize or a callable taking no arguments which returns the value to minimize. When eager execution is enabled it must be a callable.
  var_list   Optional list or tuple of tf.Variable to update to minimize loss. Defaults to the list of variables collected in the graph under the key GraphKeys.TRAINABLE_VARIABLES.
  gate_gradients   How to gate the computation of gradients. Can be GATE_NONE, GATE_OP, or GATE_GRAPH.
  aggregation_method   Specifies the method used to combine gradient terms. Valid values are defined in the class AggregationMethod.
  colocate_gradients_with_ops   If True, try colocating gradients with the corresponding op.
  grad_loss   Optional. A Tensor holding the gradient computed for loss.

 Returns   A list of (gradient, variable) pairs. Variable is always present, but gradient can be None.

 Raises
  TypeError   If var_list contains anything else than Variable objects.
  ValueError   If some arguments are invalid.
  RuntimeError   If called with eager execution enabled and loss is not callable.    Eager Compatibility When eager execution is enabled, gate_gradients, aggregation_method, and colocate_gradients_with_ops are ignored. get_name View source
get_name()
 get_slot View source
get_slot(
    var, name
)
 Return a slot named name created for var by the Optimizer. Some Optimizer subclasses use additional variables. For example Momentum and Adagrad use variables to accumulate updates. This method gives access to these Variable objects if for some reason you need them. Use get_slot_names() to get the list of slot names created by the Optimizer.

 Args
  var   A variable passed to minimize() or apply_gradients().
  name   A string.

 Returns   The Variable for the slot if it was created, None otherwise.
 get_slot_names View source
get_slot_names()
 Return a list of the names of slots created by the Optimizer. See get_slot().

 Returns   A list of strings.
 minimize View source
minimize(
    loss, global_step=None, var_list=None, gate_gradients=GATE_OP,
    aggregation_method=None, colocate_gradients_with_ops=False, name=None,
    grad_loss=None
)
 Add operations to minimize loss by updating var_list. This method simply combines calls compute_gradients() and apply_gradients(). If you want to process the gradient before applying them call compute_gradients() and apply_gradients() explicitly instead of using this function.

 Args
  loss   A Tensor containing the value to minimize.
  global_step   Optional Variable to increment by one after the variables have been updated.
  var_list   Optional list or tuple of Variable objects to update to minimize loss. Defaults to the list of variables collected in the graph under the key GraphKeys.TRAINABLE_VARIABLES.
  gate_gradients   How to gate the computation of gradients. Can be GATE_NONE, GATE_OP, or GATE_GRAPH.
  aggregation_method   Specifies the method used to combine gradient terms. Valid values are defined in the class AggregationMethod.
  colocate_gradients_with_ops   If True, try colocating gradients with the corresponding op.
  name   Optional name for the returned operation.
  grad_loss   Optional. A Tensor holding the gradient computed for loss.

 Returns   An Operation that updates the variables in var_list. If global_step was not None, that operation also increments global_step.

 Raises
  ValueError   If some of the variables are not Variable objects.    Eager Compatibility When eager execution is enabled, loss should be a Python function that takes no arguments and computes the value to be minimized. Minimization (and gradient computation) is done with respect to the elements of var_list if not None, else with respect to any trainable variables created during the execution of the loss function. gate_gradients, aggregation_method, colocate_gradients_with_ops and grad_loss are ignored when eager execution is enabled. variables View source
variables()
 A list of variables which encode the current state of Optimizer. Includes slot variables and additional global variables created by the optimizer in the current default graph.

 Returns   A list of variables.

 Class Variables
  GATE_GRAPH   2
  GATE_NONE   0
  GATE_OP   1


tf.compat.v1.train.piecewise_constant Piecewise constant from boundaries and interval values.  View aliases  Compat aliases for migration
See Migration guide for more details. tf.compat.v1.train.piecewise_constant_decay
tf.compat.v1.train.piecewise_constant(
    x, boundaries, values, name=None
)
 Example: use a learning rate that's 1.0 for the first 100001 steps, 0.5 for the next 10000 steps, and 0.1 for any additional steps. global_step = tf.Variable(0, trainable=False)
boundaries = [100000, 110000]
values = [1.0, 0.5, 0.1]
learning_rate = tf.compat.v1.train.piecewise_constant(global_step, boundaries,
values)

# Later, whenever we perform an optimization step, we increment global_step.

 Args
  x   A 0-D scalar Tensor. Must be one of the following types: float32, float64, uint8, int8, int16, int32, int64.
  boundaries   A list of Tensors or ints or floats with strictly increasing entries, and with all elements having the same type as x.
  values   A list of Tensors or floats or ints that specifies the values for the intervals defined by boundaries. It should have one more element than boundaries, and all elements should have the same type.
  name   A string. Optional name of the operation. Defaults to 'PiecewiseConstant'.

 Returns   A 0-D Tensor. Its value is values[0] when x <= boundaries[0], values[1] when x > boundaries[0] and x <= boundaries[1], ..., and values[-1] when x > boundaries[-1].

 Raises
  ValueError   if types of x and boundaries do not match, or types of all values do not match or the number of elements in the lists does not match.    Eager Compatibility When eager execution is enabled, this function returns a function which in turn returns the decayed learning rate Tensor. This can be useful for changing the learning rate value across different invocations of optimizer functions.


tf.compat.v1.train.polynomial_decay Applies a polynomial decay to the learning rate.
tf.compat.v1.train.polynomial_decay(
    learning_rate, global_step, decay_steps, end_learning_rate=0.0001, power=1.0,
    cycle=False, name=None
)
 It is commonly observed that a monotonically decreasing learning rate, whose degree of change is carefully chosen, results in a better performing model. This function applies a polynomial decay function to a provided initial learning_rate to reach an end_learning_rate in the given decay_steps. It requires a global_step value to compute the decayed learning rate. You can just pass a TensorFlow variable that you increment at each training step. The function returns the decayed learning rate. It is computed as: global_step = min(global_step, decay_steps)
decayed_learning_rate = (learning_rate - end_learning_rate) *
                        (1 - global_step / decay_steps) ^ (power) +
                        end_learning_rate

 If cycle is True then a multiple of decay_steps is used, the first one that is bigger than global_steps. decay_steps = decay_steps * ceil(global_step / decay_steps)
decayed_learning_rate = (learning_rate - end_learning_rate) *
                        (1 - global_step / decay_steps) ^ (power) +
                        end_learning_rate

 Example: decay from 0.1 to 0.01 in 10000 steps using sqrt (i.e. power=0.5): ...
global_step = tf.Variable(0, trainable=False)
starter_learning_rate = 0.1
end_learning_rate = 0.01
decay_steps = 10000
learning_rate = tf.compat.v1.train.polynomial_decay(starter_learning_rate,
global_step,
                                          decay_steps, end_learning_rate,
                                          power=0.5)
# Passing global_step to minimize() will increment it at each step.
learning_step = (
    tf.compat.v1.train.GradientDescentOptimizer(learning_rate)
    .minimize(...my loss..., global_step=global_step)
)

 Args
  learning_rate   A scalar float32 or float64 Tensor or a Python number. The initial learning rate.
  global_step   A scalar int32 or int64 Tensor or a Python number. Global step to use for the decay computation. Must not be negative.
  decay_steps   A scalar int32 or int64 Tensor or a Python number. Must be positive. See the decay computation above.
  end_learning_rate   A scalar float32 or float64 Tensor or a Python number. The minimal end learning rate.
  power   A scalar float32 or float64 Tensor or a Python number. The power of the polynomial. Defaults to linear, 1.0.
  cycle   A boolean, whether or not it should cycle beyond decay_steps.
  name   String. Optional name of the operation. Defaults to 'PolynomialDecay'.

 Returns   A scalar Tensor of the same type as learning_rate. The decayed learning rate.

 Raises
  ValueError   if global_step is not supplied.    Eager Compatibility When eager execution is enabled, this function returns a function which in turn returns the decayed learning rate Tensor. This can be useful for changing the learning rate value across different invocations of optimizer functions.


tf.compat.v1.train.ProximalAdagradOptimizer Optimizer that implements the Proximal Adagrad algorithm. Inherits From: Optimizer
tf.compat.v1.train.ProximalAdagradOptimizer(
    learning_rate, initial_accumulator_value=0.1, l1_regularization_strength=0.0,
    l2_regularization_strength=0.0, use_locking=False,
    name='ProximalAdagrad'
)
 References: Adaptive Subgradient Methods for Online Learning and Stochastic Optimization: Duchi et al., 2011 (pdf) Efficient Learning using Forward-Backward Splitting: Duchi et al., 2009 (pdf)

 Args
  learning_rate   A Tensor or a floating point value. The learning rate.
  initial_accumulator_value   A floating point value. Starting value for the accumulators, must be positive.
  l1_regularization_strength   A float value, must be greater than or equal to zero.
  l2_regularization_strength   A float value, must be greater than or equal to zero.
  use_locking   If True use locks for update operations.
  name   Optional name prefix for the operations created when applying gradients. Defaults to "Adagrad".

 Raises
  ValueError   If the initial_accumulator_value is invalid.    Methods apply_gradients View source
apply_gradients(
    grads_and_vars, global_step=None, name=None
)
 Apply gradients to variables. This is the second part of minimize(). It returns an Operation that applies gradients.

 Args
  grads_and_vars   List of (gradient, variable) pairs as returned by compute_gradients().
  global_step   Optional Variable to increment by one after the variables have been updated.
  name   Optional name for the returned operation. Default to the name passed to the Optimizer constructor.

 Returns   An Operation that applies the specified gradients. If global_step was not None, that operation also increments global_step.

 Raises
  TypeError   If grads_and_vars is malformed.
  ValueError   If none of the variables have gradients.
  RuntimeError   If you should use _distributed_apply() instead.    compute_gradients View source
compute_gradients(
    loss, var_list=None, gate_gradients=GATE_OP, aggregation_method=None,
    colocate_gradients_with_ops=False, grad_loss=None
)
 Compute gradients of loss for the variables in var_list. This is the first part of minimize(). It returns a list of (gradient, variable) pairs where "gradient" is the gradient for "variable". Note that "gradient" can be a Tensor, an IndexedSlices, or None if there is no gradient for the given variable.

 Args
  loss   A Tensor containing the value to minimize or a callable taking no arguments which returns the value to minimize. When eager execution is enabled it must be a callable.
  var_list   Optional list or tuple of tf.Variable to update to minimize loss. Defaults to the list of variables collected in the graph under the key GraphKeys.TRAINABLE_VARIABLES.
  gate_gradients   How to gate the computation of gradients. Can be GATE_NONE, GATE_OP, or GATE_GRAPH.
  aggregation_method   Specifies the method used to combine gradient terms. Valid values are defined in the class AggregationMethod.
  colocate_gradients_with_ops   If True, try colocating gradients with the corresponding op.
  grad_loss   Optional. A Tensor holding the gradient computed for loss.

 Returns   A list of (gradient, variable) pairs. Variable is always present, but gradient can be None.

 Raises
  TypeError   If var_list contains anything else than Variable objects.
  ValueError   If some arguments are invalid.
  RuntimeError   If called with eager execution enabled and loss is not callable.    Eager Compatibility When eager execution is enabled, gate_gradients, aggregation_method, and colocate_gradients_with_ops are ignored. get_name View source
get_name()
 get_slot View source
get_slot(
    var, name
)
 Return a slot named name created for var by the Optimizer. Some Optimizer subclasses use additional variables. For example Momentum and Adagrad use variables to accumulate updates. This method gives access to these Variable objects if for some reason you need them. Use get_slot_names() to get the list of slot names created by the Optimizer.

 Args
  var   A variable passed to minimize() or apply_gradients().
  name   A string.

 Returns   The Variable for the slot if it was created, None otherwise.
 get_slot_names View source
get_slot_names()
 Return a list of the names of slots created by the Optimizer. See get_slot().

 Returns   A list of strings.
 minimize View source
minimize(
    loss, global_step=None, var_list=None, gate_gradients=GATE_OP,
    aggregation_method=None, colocate_gradients_with_ops=False, name=None,
    grad_loss=None
)
 Add operations to minimize loss by updating var_list. This method simply combines calls compute_gradients() and apply_gradients(). If you want to process the gradient before applying them call compute_gradients() and apply_gradients() explicitly instead of using this function.

 Args
  loss   A Tensor containing the value to minimize.
  global_step   Optional Variable to increment by one after the variables have been updated.
  var_list   Optional list or tuple of Variable objects to update to minimize loss. Defaults to the list of variables collected in the graph under the key GraphKeys.TRAINABLE_VARIABLES.
  gate_gradients   How to gate the computation of gradients. Can be GATE_NONE, GATE_OP, or GATE_GRAPH.
  aggregation_method   Specifies the method used to combine gradient terms. Valid values are defined in the class AggregationMethod.
  colocate_gradients_with_ops   If True, try colocating gradients with the corresponding op.
  name   Optional name for the returned operation.
  grad_loss   Optional. A Tensor holding the gradient computed for loss.

 Returns   An Operation that updates the variables in var_list. If global_step was not None, that operation also increments global_step.

 Raises
  ValueError   If some of the variables are not Variable objects.    Eager Compatibility When eager execution is enabled, loss should be a Python function that takes no arguments and computes the value to be minimized. Minimization (and gradient computation) is done with respect to the elements of var_list if not None, else with respect to any trainable variables created during the execution of the loss function. gate_gradients, aggregation_method, colocate_gradients_with_ops and grad_loss are ignored when eager execution is enabled. variables View source
variables()
 A list of variables which encode the current state of Optimizer. Includes slot variables and additional global variables created by the optimizer in the current default graph.

 Returns   A list of variables.

 Class Variables
  GATE_GRAPH   2
  GATE_NONE   0
  GATE_OP   1