neulab/docprompting-conala huggingface dataset:
tf.compat.v1.train.noisy_linear_cosine_decay Applies noisy linear cosine decay to the learning rate. 
tf.compat.v1.train.noisy_linear_cosine_decay(
    learning_rate, global_step, decay_steps, initial_variance=1.0,
    variance_decay=0.55, num_periods=0.5, alpha=0.0, beta=0.001, name=None
)
 Note that linear cosine decay is more aggressive than cosine decay and larger initial learning rates can typically be used. When training a model, it is often recommended to lower the learning rate as the training progresses. This function applies a noisy linear cosine decay function to a provided initial learning rate. It requires a global_step value to compute the decayed learning rate. You can just pass a TensorFlow variable that you increment at each training step. The function returns the decayed learning rate. It is computed as: global_step = min(global_step, decay_steps)
linear_decay = (decay_steps - global_step) / decay_steps)
cosine_decay = 0.5 * (
    1 + cos(pi * 2 * num_periods * global_step / decay_steps))
decayed = (alpha + linear_decay + eps_t) * cosine_decay + beta
decayed_learning_rate = learning_rate * decayed
 where eps_t is 0-centered gaussian noise with variance initial_variance / (1 + global_step) ** variance_decay Example usage: decay_steps = 1000
lr_decayed = noisy_linear_cosine_decay(
  learning_rate, global_step, decay_steps)

 


 Args
  learning_rate   A scalar float32 or float64 Tensor or a Python number. The initial learning rate.  
  global_step   A scalar int32 or int64 Tensor or a Python number. Global step to use for the decay computation.  
  decay_steps   A scalar int32 or int64 Tensor or a Python number. Number of steps to decay over.  
  initial_variance   initial variance for the noise. See computation above.  
  variance_decay   decay for the noise's variance. See computation above.  
  num_periods   Number of periods in the cosine part of the decay. See computation above.  
  alpha   See computation above.  
  beta   See computation above.  
  name   String. Optional name of the operation. Defaults to 'NoisyLinearCosineDecay'.   
 


 Returns   A scalar Tensor of the same type as learning_rate. The decayed learning rate.  

 


 Raises
  ValueError   if global_step is not supplied.    References: Neural Optimizer Search with Reinforcement Learning: Bello et al., 2017 (pdf) Stochastic Gradient Descent with Warm Restarts: Loshchilov et al., 2017 (pdf) Eager Compatibility When eager execution is enabled, this function returns a function which in turn returns the decayed learning rate Tensor. This can be useful for changing the learning rate value across different invocations of optimizer functions.
tf.compat.v1.train.piecewise_constant Piecewise constant from boundaries and interval values.  View aliases  Compat aliases for migration 
See Migration guide for more details. tf.compat.v1.train.piecewise_constant_decay  
tf.compat.v1.train.piecewise_constant(
    x, boundaries, values, name=None
)
 Example: use a learning rate that's 1.0 for the first 100001 steps, 0.5 for the next 10000 steps, and 0.1 for any additional steps. global_step = tf.Variable(0, trainable=False)
boundaries = [100000, 110000]
values = [1.0, 0.5, 0.1]
learning_rate = tf.compat.v1.train.piecewise_constant(global_step, boundaries,
values)

# Later, whenever we perform an optimization step, we increment global_step.

 


 Args
  x   A 0-D scalar Tensor. Must be one of the following types: float32, float64, uint8, int8, int16, int32, int64.  
  boundaries   A list of Tensors or ints or floats with strictly increasing entries, and with all elements having the same type as x.  
  values   A list of Tensors or floats or ints that specifies the values for the intervals defined by boundaries. It should have one more element than boundaries, and all elements should have the same type.  
  name   A string. Optional name of the operation. Defaults to 'PiecewiseConstant'.   
 


 Returns   A 0-D Tensor. Its value is values[0] when x <= boundaries[0], values[1] when x > boundaries[0] and x <= boundaries[1], ..., and values[-1] when x > boundaries[-1].  

 


 Raises
  ValueError   if types of x and boundaries do not match, or types of all values do not match or the number of elements in the lists does not match.    Eager Compatibility When eager execution is enabled, this function returns a function which in turn returns the decayed learning rate Tensor. This can be useful for changing the learning rate value across different invocations of optimizer functions.
tf.compat.v1.train.polynomial_decay Applies a polynomial decay to the learning rate. 
tf.compat.v1.train.polynomial_decay(
    learning_rate, global_step, decay_steps, end_learning_rate=0.0001, power=1.0,
    cycle=False, name=None
)
 It is commonly observed that a monotonically decreasing learning rate, whose degree of change is carefully chosen, results in a better performing model. This function applies a polynomial decay function to a provided initial learning_rate to reach an end_learning_rate in the given decay_steps. It requires a global_step value to compute the decayed learning rate. You can just pass a TensorFlow variable that you increment at each training step. The function returns the decayed learning rate. It is computed as: global_step = min(global_step, decay_steps)
decayed_learning_rate = (learning_rate - end_learning_rate) *
                        (1 - global_step / decay_steps) ^ (power) +
                        end_learning_rate

 If cycle is True then a multiple of decay_steps is used, the first one that is bigger than global_steps. decay_steps = decay_steps * ceil(global_step / decay_steps)
decayed_learning_rate = (learning_rate - end_learning_rate) *
                        (1 - global_step / decay_steps) ^ (power) +
                        end_learning_rate

 Example: decay from 0.1 to 0.01 in 10000 steps using sqrt (i.e. power=0.5): ...
global_step = tf.Variable(0, trainable=False)
starter_learning_rate = 0.1
end_learning_rate = 0.01
decay_steps = 10000
learning_rate = tf.compat.v1.train.polynomial_decay(starter_learning_rate,
global_step,
                                          decay_steps, end_learning_rate,
                                          power=0.5)
# Passing global_step to minimize() will increment it at each step.
learning_step = (
    tf.compat.v1.train.GradientDescentOptimizer(learning_rate)
    .minimize(...my loss..., global_step=global_step)
)

 


 Args
  learning_rate   A scalar float32 or float64 Tensor or a Python number. The initial learning rate.  
  global_step   A scalar int32 or int64 Tensor or a Python number. Global step to use for the decay computation. Must not be negative.  
  decay_steps   A scalar int32 or int64 Tensor or a Python number. Must be positive. See the decay computation above.  
  end_learning_rate   A scalar float32 or float64 Tensor or a Python number. The minimal end learning rate.  
  power   A scalar float32 or float64 Tensor or a Python number. The power of the polynomial. Defaults to linear, 1.0.  
  cycle   A boolean, whether or not it should cycle beyond decay_steps.  
  name   String. Optional name of the operation. Defaults to 'PolynomialDecay'.   
 


 Returns   A scalar Tensor of the same type as learning_rate. The decayed learning rate.  

 


 Raises
  ValueError   if global_step is not supplied.    Eager Compatibility When eager execution is enabled, this function returns a function which in turn returns the decayed learning rate Tensor. This can be useful for changing the learning rate value across different invocations of optimizer functions.
Module: tf.compat.v1.train.queue_runner Public API for tf.train.queue_runner namespace. Classes class QueueRunner: Holds a list of enqueue operations for a queue, each to be run in a thread. Functions add_queue_runner(...): Adds a QueueRunner to a collection in the graph. (deprecated) start_queue_runners(...): Starts all queue runners collected in the graph. (deprecated)
tf.compat.v1.train.range_input_producer Produces the integers from 0 to limit-1 in a queue. (deprecated) 
tf.compat.v1.train.range_input_producer(
    limit, num_epochs=None, shuffle=True, seed=None, capacity=32, shared_name=None,
    name=None
)
 Warning: THIS FUNCTION IS DEPRECATED. It will be removed in a future version. Instructions for updating: Queue-based input pipelines have been replaced by tf.data. Use tf.data.Dataset.range(limit).shuffle(limit).repeat(num_epochs). If shuffle=False, omit the .shuffle(...).
Note: if num_epochs is not None, this function creates local counter epochs. Use local_variables_initializer() to initialize local variables.

 


 Args
  limit   An int32 scalar tensor.  
  num_epochs   An integer (optional). If specified, range_input_producer produces each integer num_epochs times before generating an OutOfRange error. If not specified, range_input_producer can cycle through the integers an unlimited number of times.  
  shuffle   Boolean. If true, the integers are randomly shuffled within each epoch.  
  seed   An integer (optional). Seed used if shuffle == True.  
  capacity   An integer. Sets the queue capacity.  
  shared_name   (optional). If set, this queue will be shared under the given name across multiple sessions.  
  name   A name for the operations (optional).   
 


 Returns   A Queue with the output integers. A QueueRunner for the Queue is added to the current Graph's QUEUE_RUNNER collection.  
 Eager Compatibility Input pipelines based on Queues are not supported when eager execution is enabled. Please use the tf.data API to ingest data under eager execution.
tf.compat.v1.train.remove_checkpoint Removes a checkpoint given by checkpoint_prefix. (deprecated) 
tf.compat.v1.train.remove_checkpoint(
    checkpoint_prefix, checkpoint_format_version=tf.train.SaverDef.V2,
    meta_graph_suffix='meta'
)
 Warning: THIS FUNCTION IS DEPRECATED. It will be removed in a future version. Instructions for updating: Use standard file APIs to delete files with this prefix.
 


 Args
  checkpoint_prefix   The prefix of a V1 or V2 checkpoint. Typically the result of Saver.save() or that of tf.train.latest_checkpoint(), regardless of sharded/non-sharded or V1/V2.  
  checkpoint_format_version   SaverDef.CheckpointFormatVersion, defaults to SaverDef.V2.  
  meta_graph_suffix   Suffix for MetaGraphDef file. Defaults to 'meta'.
tf.compat.v1.train.replica_device_setter Return a device function to use when building a Graph for replicas. 
tf.compat.v1.train.replica_device_setter(
    ps_tasks=0, ps_device='/job:ps',
    worker_device='/job:worker', merge_devices=True, cluster=None,
    ps_ops=None, ps_strategy=None
)
 Device Functions are used in with tf.device(device_function): statement to automatically assign devices to Operation objects as they are constructed, Device constraints are added from the inner-most context first, working outwards. The merging behavior adds constraints to fields that are yet unset by a more inner context. Currently the fields are (job, task, cpu/gpu). If cluster is None, and ps_tasks is 0, the returned function is a no-op. Otherwise, the value of ps_tasks is derived from cluster. By default, only Variable ops are placed on ps tasks, and the placement strategy is round-robin over all ps tasks. A custom ps_strategy may be used to do more intelligent placement, such as tf.contrib.training.GreedyLoadBalancingStrategy. For example, # To build a cluster with two ps jobs on hosts ps0 and ps1, and 3 worker
# jobs on hosts worker0, worker1 and worker2.
cluster_spec = {
    "ps": ["ps0:2222", "ps1:2222"],
    "worker": ["worker0:2222", "worker1:2222", "worker2:2222"]}
with
tf.device(tf.compat.v1.train.replica_device_setter(cluster=cluster_spec)):
  # Build your graph
  v1 = tf.Variable(...)  # assigned to /job:ps/task:0
  v2 = tf.Variable(...)  # assigned to /job:ps/task:1
  v3 = tf.Variable(...)  # assigned to /job:ps/task:0
# Run compute

 


 Args
  ps_tasks   Number of tasks in the ps job. Ignored if cluster is provided.  
  ps_device   String. Device of the ps job. If empty no ps job is used. Defaults to ps.  
  worker_device   String. Device of the worker job. If empty no worker job is used.  
  merge_devices   Boolean. If True, merges or only sets a device if the device constraint is completely unset. merges device specification rather than overriding them.  
  cluster   ClusterDef proto or ClusterSpec.  
  ps_ops   List of strings representing Operation types that need to be placed on ps devices. If None, defaults to STANDARD_PS_OPS.  
  ps_strategy   A callable invoked for every ps Operation (i.e. matched by ps_ops), that takes the Operation and returns the ps task index to use. If None, defaults to a round-robin strategy across all ps devices.   
 


 Returns   A function to pass to tf.device().  

 


 Raises   TypeError if cluster is not a dictionary or ClusterDef protocol buffer, or if ps_strategy is provided but not a callable.
tf.compat.v1.train.SaverDef A ProtocolMessage
 


 Attributes
  filename_tensor_name   string filename_tensor_name  
  keep_checkpoint_every_n_hours   float keep_checkpoint_every_n_hours  
  max_to_keep   int32 max_to_keep  
  restore_op_name   string restore_op_name  
  save_tensor_name   string save_tensor_name  
  sharded   bool sharded  
  version   CheckpointFormatVersion version   
 


 Class Variables
  CheckpointFormatVersion  
 
  LEGACY   0  
  V1   1  
  V2   2
tf.compat.v1.train.sdca_fprint Computes fingerprints of the input strings. 
tf.compat.v1.train.sdca_fprint(
    input, name=None
)

 


 Args
  input   A Tensor of type string. vector of strings to compute fingerprints on.  
  name   A name for the operation (optional).   
 


 Returns   A Tensor of type int64.
tf.compat.v1.train.sdca_shrink_l1 Applies L1 regularization shrink step on the parameters. 
tf.compat.v1.train.sdca_shrink_l1(
    weights, l1, l2, name=None
)

 


 Args
  weights   A list of Tensor objects with type mutable float32. a list of vectors where each value is the weight associated with a feature group.  
  l1   A float. Symmetric l1 regularization strength.  
  l2   A float. Symmetric l2 regularization strength. Should be a positive float.  
  name   A name for the operation (optional).   
 


 Returns   The created Operation.
tf.compat.v1.train.SessionCreator A factory for tf.Session. Methods create_session View source 
@abc.abstractmethod
create_session()
tf.compat.v1.train.slice_input_producer Produces a slice of each Tensor in tensor_list. (deprecated) 
tf.compat.v1.train.slice_input_producer(
    tensor_list, num_epochs=None, shuffle=True, seed=None, capacity=32,
    shared_name=None, name=None
)
 Warning: THIS FUNCTION IS DEPRECATED. It will be removed in a future version. Instructions for updating: Queue-based input pipelines have been replaced by tf.data. Use tf.data.Dataset.from_tensor_slices(tuple(tensor_list)).shuffle(tf.shape(input_tensor, out_type=tf.int64)[0]).repeat(num_epochs). If shuffle=False, omit the .shuffle(...). Implemented using a Queue -- a QueueRunner for the Queue is added to the current Graph's QUEUE_RUNNER collection.
 


 Args
  tensor_list   A list of Tensor objects. Every Tensor in tensor_list must have the same size in the first dimension.  
  num_epochs   An integer (optional). If specified, slice_input_producer produces each slice num_epochs times before generating an OutOfRange error. If not specified, slice_input_producer can cycle through the slices an unlimited number of times.  
  shuffle   Boolean. If true, the integers are randomly shuffled within each epoch.  
  seed   An integer (optional). Seed used if shuffle == True.  
  capacity   An integer. Sets the queue capacity.  
  shared_name   (optional). If set, this queue will be shared under the given name across multiple sessions.  
  name   A name for the operations (optional).   
 


 Returns   A list of tensors, one for each element of tensor_list. If the tensor in tensor_list has shape [N, a, b, .., z], then the corresponding output tensor will have shape [a, b, ..., z].  

 


 Raises
  ValueError   if slice_input_producer produces nothing from tensor_list.    Eager Compatibility Input pipelines based on Queues are not supported when eager execution is enabled. Please use the tf.data API to ingest data under eager execution.
tf.compat.v1.train.start_queue_runners Starts all queue runners collected in the graph. (deprecated)  View aliases  Compat aliases for migration 
See Migration guide for more details. tf.compat.v1.train.queue_runner.start_queue_runners  
tf.compat.v1.train.start_queue_runners(
    sess=None, coord=None, daemon=True, start=True,
    collection=tf.GraphKeys.QUEUE_RUNNERS
)
 Warning: THIS FUNCTION IS DEPRECATED. It will be removed in a future version. Instructions for updating: To construct input pipelines, use the tf.data module. This is a companion method to add_queue_runner(). It just starts threads for all queue runners collected in the graph. It returns the list of all threads.
 


 Args
  sess   Session used to run the queue ops. Defaults to the default session.  
  coord   Optional Coordinator for coordinating the started threads.  
  daemon   Whether the threads should be marked as daemons, meaning they don't block program exit.  
  start   Set to False to only create the threads, not start them.  
  collection   A GraphKey specifying the graph collection to get the queue runners from. Defaults to GraphKeys.QUEUE_RUNNERS.   
 


 Raises
  ValueError   if sess is None and there isn't any default session.  
  TypeError   if sess is not a tf.compat.v1.Session object.   
 


 Returns   A list of threads.  

 


 Raises
  RuntimeError   If called with eager execution enabled.  
  ValueError   If called without a default tf.compat.v1.Session registered.    Eager Compatibility Not compatible with eager execution. To ingest data under eager execution, use the tf.data API instead.
tf.compat.v1.train.string_input_producer Output strings (e.g. filenames) to a queue for an input pipeline. (deprecated) 
tf.compat.v1.train.string_input_producer(
    string_tensor, num_epochs=None, shuffle=True, seed=None, capacity=32,
    shared_name=None, name=None, cancel_op=None
)
 Warning: THIS FUNCTION IS DEPRECATED. It will be removed in a future version. Instructions for updating: Queue-based input pipelines have been replaced by tf.data. Use tf.data.Dataset.from_tensor_slices(string_tensor).shuffle(tf.shape(input_tensor, out_type=tf.int64)[0]).repeat(num_epochs). If shuffle=False, omit the .shuffle(...).
Note: if num_epochs is not None, this function creates local counter epochs. Use local_variables_initializer() to initialize local variables.

 


 Args
  string_tensor   A 1-D string tensor with the strings to produce.  
  num_epochs   An integer (optional). If specified, string_input_producer produces each string from string_tensor num_epochs times before generating an OutOfRange error. If not specified, string_input_producer can cycle through the strings in string_tensor an unlimited number of times.  
  shuffle   Boolean. If true, the strings are randomly shuffled within each epoch.  
  seed   An integer (optional). Seed used if shuffle == True.  
  capacity   An integer. Sets the queue capacity.  
  shared_name   (optional). If set, this queue will be shared under the given name across multiple sessions. All sessions open to the device which has this queue will be able to access it via the shared_name. Using this in a distributed setting means each name will only be seen by one of the sessions which has access to this operation.  
  name   A name for the operations (optional).  
  cancel_op   Cancel op for the queue (optional).   
 


 Returns   A queue with the output strings. A QueueRunner for the Queue is added to the current Graph's QUEUE_RUNNER collection.  

 


 Raises
  ValueError   If the string_tensor is a null Python list. At runtime, will fail with an assertion if string_tensor becomes a null tensor.    Eager Compatibility Input pipelines based on Queues are not supported when eager execution is enabled. Please use the tf.data API to ingest data under eager execution.
tf.compat.v1.train.summary_iterator Returns a iterator for reading Event protocol buffers from an event file. 
tf.compat.v1.train.summary_iterator(
    path
)
 You can use this function to read events written to an event file. It returns a Python iterator that yields Event protocol buffers. Example: Print the contents of an events file. for e in tf.compat.v1.train.summary_iterator(path to events file):
    print(e)
 Example: Print selected summary values. # This example supposes that the events file contains summaries with a
# summary value tag 'loss'.  These could have been added by calling
# `add_summary()`, passing the output of a scalar summary op created with
# with: `tf.compat.v1.summary.scalar('loss', loss_tensor)`.
for e in tf.compat.v1.train.summary_iterator(path to events file):
    for v in e.summary.value:
        if v.tag == 'loss':
            print(v.simple_value)
 Example: Continuously check for new summary values. summaries = tf.compat.v1.train.summary_iterator(path to events file)
while True:
  for e in summaries:
      for v in e.summary.value:
          if v.tag == 'loss':
              print(v.simple_value)
  # Wait for a bit before checking the file for any new events
  time.sleep(wait time)
 See the protocol buffer definitions of Event and Summary for more information about their attributes.
 


 Args
  path   The path to an event file created by a SummaryWriter.   
 


 Returns   A iterator that yields Event protocol buffers
tf.compat.v1.train.update_checkpoint_state Updates the content of the 'checkpoint' file. (deprecated) 
tf.compat.v1.train.update_checkpoint_state(
    save_dir, model_checkpoint_path, all_model_checkpoint_paths=None,
    latest_filename=None, all_model_checkpoint_timestamps=None,
    last_preserved_timestamp=None
)
 Warning: THIS FUNCTION IS DEPRECATED. It will be removed in a future version. Instructions for updating: Use tf.train.CheckpointManager to manage checkpoints rather than manually editing the Checkpoint proto. This updates the checkpoint file containing a CheckpointState proto.
 


 Args
  save_dir   Directory where the model was saved.  
  model_checkpoint_path   The checkpoint file.  
  all_model_checkpoint_paths   List of strings. Paths to all not-yet-deleted checkpoints, sorted from oldest to newest. If this is a non-empty list, the last element must be equal to model_checkpoint_path. These paths are also saved in the CheckpointState proto.  
  latest_filename   Optional name of the checkpoint file. Default to 'checkpoint'.  
  all_model_checkpoint_timestamps   Optional list of timestamps (floats, seconds since the Epoch) indicating when the checkpoints in all_model_checkpoint_paths were created.  
  last_preserved_timestamp   A float, indicating the number of seconds since the Epoch when the last preserved checkpoint was written, e.g. due to a keep_checkpoint_every_n_hours parameter (see tf.train.CheckpointManager for an implementation).   
 


 Raises
  RuntimeError   If any of the model checkpoint paths conflict with the file containing CheckpointSate.