neulab/docprompting-conala huggingface dataset:
tf.compat.v1.tpu.outside_compilation Builds part of a computation outside any current TPU replicate scope. 
tf.compat.v1.tpu.outside_compilation(
    computation, *args, **kwargs
)
 tf.tpu.outside_compilation() is used to run ops in computation on CPU instead of running on TPU. For example, users can run ops that are not supported on TPU's (e.g. tf.summary.write()) by explicitly placing those ops on CPU's. Below usage of outside compilation will place ops in computation_with_string_ops on CPU. Example usage: def computation_with_string_ops(x):
  # strings types are not supported on TPU's and below ops must
  # run on CPU instead.
  output = tf.strings.format('1{}', x)
  return tf.strings.to_number(output)

def tpu_computation():
  # Expected output is 11.
  output = tf.tpu.outside_compilation(computation_with_string_ops, 1)
 Outside compilation should be called inside TPUReplicateContext. That is, tf.tpu.outside_compilation() should be called inside a function that is passed to tpu.split_compile_and_replicate() -- this is implied when outside compilation is invoked inside a function passed to TPUStrategy run(). If invoked outside of TPUReplicateContext, then this simply returns the result of computation, and therefore, would be a no-op. Note that outside compilation is different from tf.distribute.experimental.TPUStrategy.merge_call() as logic in outside compilation is replicated and executed separately for each replica. On the other hand, merge_call() requires a merge_fn to aggregate the inputs from different replicas and is executed only once. For variables placed in TPU device, which includes variables created inside TPUStrategy scope, outside compilation logic must not include variable read/write. For variables placed on host, which is the case when variables created via TPUEstimator, variable read/write is only allowed if the variable is not accessed by any other ops in the TPU computation. Variable read/write from outside compilation cluster is not visible from TPU computation and vice versa. Therefore, if outside compilation logic contains such host variables read/write ops and if the variables are accessed by TPU computation as well, then this may lead to deadlock. Internally, tf.tpu.outside_compilation() adds outside compilation attributes to all ops in computation. During later graph pass, these ops with outside compilation attribute is extracted out and replicated into a host-side graph. Inputs to this extract host-side graph is sent from TPU computation graph to host graph via a pair of XlaSendToHost and XlaRecvFromHost ops. Note that using tf.tpu.outside_compilation() may result in tensor transfer between TPU and CPU, leading to non-trivial performance impact.
 


 Args
  computation   A Python function that builds the computation to place on the host.  
  *args   the positional arguments for the computation.  
  **kwargs   the keyword arguments for the computation.   
 


 Returns   The Tensors returned by computation.
tf.compat.v1.tpu.PaddingSpec Represents the type of padding policies for tpu.replicate.
 


 Class Variables
  AUTO   <PaddingSpec.AUTO: 0>  
  POWER_OF_TWO   <PaddingSpec.POWER_OF_TWO: 1>
tf.compat.v1.tpu.rewrite Rewrites computation for execution on a TPU system. 
tf.compat.v1.tpu.rewrite(
    computation, inputs=None, infeed_queue=None, device_assignment=None, name=None,
    xla_options=None
)

 


 Args
  computation   A Python function that builds a computation to apply to the input. If the function takes n inputs, 'inputs' should be a list of n tensors. computation may return a list of operations and tensors. Tensors must come before operations in the returned list. The return value of rewrite is a list of tensors corresponding to the tensors from the output of computation. All Operations constructed during computation will be executed when evaluating any of the returned output tensors, not just the ones returned. 
 
  inputs   A list of input tensors or None (equivalent to an empty list). Each input can be a nested structure containing values that are convertible to tensors. Note that passing an N-dimension list of compatible values will result in a N-dimension list of scalar tensors rather than a single Rank-N tensors. If you need different behavior, convert part of inputs to tensors with tf.convert_to_tensor.  
  infeed_queue   If not None, the InfeedQueue from which to append a tuple of arguments as inputs to computation.  
  device_assignment   if not None, a DeviceAssignment describing the mapping between logical cores in the computation with physical cores in the TPU topology. May be omitted for a single-core computation, in which case the core attached to task 0, TPU device 0 is used.  
  name   (Deprecated) Does nothing.  
  xla_options   An instance of tpu.XLAOptions which indicates the options passed to XLA compiler. Use None for default options.   
 


 Returns   Same data structure as if computation(*inputs) is called directly with some exceptions for correctness. Exceptions include: 1) None output: a NoOp would be returned which control-depends on computation. 2) Single value output: A tuple containing the value would be returned. 3) Operation-only outputs: a NoOp would be returned which control-depends on computation.
tf.compat.v1.tpu.shard Shards computation for parallel execution. 
tf.compat.v1.tpu.shard(
    computation, inputs=None, num_shards=1, input_shard_axes=None,
    outputs_from_all_shards=True, output_shard_axes=None, infeed_queue=None,
    device_assignment=None, name=None, xla_options=None
)
 inputs must be a list of Tensors or None (equivalent to an empty list), each of which has a corresponding split axis (from input_shard_axes). Each input is split into num_shards pieces along the corresponding axis, and computation is applied to each shard in parallel. Tensors are broadcast to all shards if they are lexically captured by computation. e.g., x = tf.constant(7) def computation(): return x + 3 ... = shard(computation, ...) as inputs. If outputs_from_all_shards is true, the outputs from all shards of computation are concatenated back together along their output_shard_axes. Otherwise, each output is taken from an arbitrary shard. Inputs and outputs of the computation must be at least rank-1 Tensors.
 


 Args
  computation   A Python function that builds a computation to apply to each shard of the input.  
  inputs   A list of input tensors or None (equivalent to an empty list). Each input tensor has a corresponding shard axes, given by input_shard_axes, which must have size divisible by num_shards.  
  num_shards   The number of shards.  
  input_shard_axes   A list of dimensions along which to shard inputs, or None. None means "shard all inputs along dimension 0". If not None, there must be one dimension per input.  
  outputs_from_all_shards   Boolean or list of boolean. For each output, if True, outputs from all shards are concatenated along the corresponding output_shard_axes entry. Otherwise, each output is taken from an arbitrary shard. If the argument is a boolean, the argument's value is used for each output.  
  output_shard_axes   A list of dimensions along which to concatenate the outputs of computation, or None. None means "concatenate all outputs along dimension 0". If not None, there must be one dimension per output. Ignored if outputs_from_all_shards is False.  
  infeed_queue   If not None, the InfeedQueue to use to augment the inputs of computation.  
  device_assignment   If not None, a DeviceAssignment describing the mapping between logical cores in the computation with physical cores in the TPU topology. Uses a default device assignment if None. The DeviceAssignment may be omitted if each shard of the computation uses only one core, and there is either only one shard, or the number of shards is equal to the number of cores in the TPU system.  
  name   (Deprecated) Does nothing.  
  xla_options   An instance of tpu.XLAOptions which indicates the options passed to XLA compiler. Use None for default options.   
 


 Returns   A list of output tensors.  

 


 Raises
  ValueError   If num_shards <= 0  
  ValueError   If len(input_shard_axes) != len(inputs)  
  ValueError   If len(output_shard_axes) != len(outputs from computation)
tf.compat.v1.tpu.shutdown_system Shuts down a running a distributed TPU system. 
tf.compat.v1.tpu.shutdown_system(
    job=None
)

 


 Args
  job   The job (the XXX in TensorFlow device specification /job:XXX) that contains the TPU devices that will be shutdown. If job=None it is assumed there is only one job in the TensorFlow flock, and an error will be returned if this assumption does not hold.
tf.compat.v1.tpu.XLAOptions XLA compilation options. 
tf.compat.v1.tpu.XLAOptions(
    use_spmd_for_xla_partitioning=True
)

 


 Attributes
  use_spmd_for_xla_partitioning   Boolean. Whether to use XLA's SPMD partitioner instead of MPMD partitioner when compiler partitioning is requested.
tf.compat.v1.train.add_queue_runner Adds a QueueRunner to a collection in the graph. (deprecated)  View aliases  Compat aliases for migration 
See Migration guide for more details. tf.compat.v1.train.queue_runner.add_queue_runner  
tf.compat.v1.train.add_queue_runner(
    qr, collection=tf.GraphKeys.QUEUE_RUNNERS
)
 Warning: THIS FUNCTION IS DEPRECATED. It will be removed in a future version. Instructions for updating: To construct input pipelines, use the tf.data module. When building a complex model that uses many queues it is often difficult to gather all the queue runners that need to be run. This convenience function allows you to add a queue runner to a well known collection in the graph. The companion method start_queue_runners() can be used to start threads for all the collected queue runners.
 


 Args
  qr   A QueueRunner.  
  collection   A GraphKey specifying the graph collection to add the queue runner to. Defaults to GraphKeys.QUEUE_RUNNERS.
tf.compat.v1.train.assert_global_step Asserts global_step_tensor is a scalar int Variable or Tensor. 
tf.compat.v1.train.assert_global_step(
    global_step_tensor
)

 


 Args
  global_step_tensor   Tensor to test.
tf.compat.v1.train.basic_train_loop Basic loop to train a model. 
tf.compat.v1.train.basic_train_loop(
    supervisor, train_step_fn, args=None, kwargs=None, master=''
)
 Calls train_step_fn in a loop to train a model. The function is called as: train_step_fn(session, *args, **kwargs)
 It is passed a tf.compat.v1.Session in addition to args and kwargs. The function typically runs one training step in the session.
 


 Args
  supervisor   tf.compat.v1.train.Supervisor to run the training services.  
  train_step_fn   Callable to execute one training step. Called repeatedly as train_step_fn(session, *args **kwargs).  
  args   Optional positional arguments passed to train_step_fn.  
  kwargs   Optional keyword arguments passed to train_step_fn.  
  master   Master to use to create the training session. Defaults to "" which causes the session to be created in the local process.
tf.compat.v1.train.checkpoint_exists Checks whether a V1 or V2 checkpoint exists with the specified prefix. (deprecated) 
tf.compat.v1.train.checkpoint_exists(
    checkpoint_prefix
)
 Warning: THIS FUNCTION IS DEPRECATED. It will be removed in a future version. Instructions for updating: Use standard file APIs to check for files with this prefix. This is the recommended way to check if a checkpoint exists, since it takes into account the naming difference between V1 and V2 formats.
 


 Args
  checkpoint_prefix   the prefix of a V1 or V2 checkpoint, with V2 taking priority. Typically the result of Saver.save() or that of tf.train.latest_checkpoint(), regardless of sharded/non-sharded or V1/V2.   
 


 Returns   A bool, true if a checkpoint referred to by checkpoint_prefix exists.
tf.compat.v1.train.ChiefSessionCreator Creates a tf.compat.v1.Session for a chief. Inherits From: SessionCreator 
tf.compat.v1.train.ChiefSessionCreator(
    scaffold=None, master='', config=None, checkpoint_dir=None,
    checkpoint_filename_with_path=None
)

 


 Args
  scaffold   A Scaffold used for gathering or building supportive ops. If not specified a default one is created. It's used to finalize the graph.  
  master   String representation of the TensorFlow master to use.  
  config   ConfigProto proto used to configure the session.  
  checkpoint_dir   A string. Optional path to a directory where to restore variables.  
  checkpoint_filename_with_path   Full file name path to the checkpoint file.    Methods create_session View source 
create_session()
tf.compat.v1.train.cosine_decay Applies cosine decay to the learning rate. 
tf.compat.v1.train.cosine_decay(
    learning_rate, global_step, decay_steps, alpha=0.0, name=None
)
 When training a model, it is often recommended to lower the learning rate as the training progresses. This function applies a cosine decay function to a provided initial learning rate. It requires a global_step value to compute the decayed learning rate. You can just pass a TensorFlow variable that you increment at each training step. The function returns the decayed learning rate. It is computed as: global_step = min(global_step, decay_steps)
cosine_decay = 0.5 * (1 + cos(pi * global_step / decay_steps))
decayed = (1 - alpha) * cosine_decay + alpha
decayed_learning_rate = learning_rate * decayed
 Example usage: decay_steps = 1000
lr_decayed = cosine_decay(learning_rate, global_step, decay_steps)

 


 Args
  learning_rate   A scalar float32 or float64 Tensor or a Python number. The initial learning rate.  
  global_step   A scalar int32 or int64 Tensor or a Python number. Global step to use for the decay computation.  
  decay_steps   A scalar int32 or int64 Tensor or a Python number. Number of steps to decay over.  
  alpha   A scalar float32 or float64 Tensor or a Python number. Minimum learning rate value as a fraction of learning_rate.  
  name   String. Optional name of the operation. Defaults to 'CosineDecay'.   
 


 Returns   A scalar Tensor of the same type as learning_rate. The decayed learning rate.  

 


 Raises
  ValueError   if global_step is not supplied.    References: Stochastic Gradient Descent with Warm Restarts: Loshchilov et al., 2017 (pdf) Eager Compatibility When eager execution is enabled, this function returns a function which in turn returns the decayed learning rate Tensor. This can be useful for changing the learning rate value across different invocations of optimizer functions.
tf.compat.v1.train.cosine_decay_restarts Applies cosine decay with restarts to the learning rate. 
tf.compat.v1.train.cosine_decay_restarts(
    learning_rate, global_step, first_decay_steps, t_mul=2.0, m_mul=1.0, alpha=0.0,
    name=None
)
 When training a model, it is often recommended to lower the learning rate as the training progresses. This function applies a cosine decay function with restarts to a provided initial learning rate. It requires a global_step value to compute the decayed learning rate. You can just pass a TensorFlow variable that you increment at each training step. The function returns the decayed learning rate while taking into account possible warm restarts. The learning rate multiplier first decays from 1 to alpha for first_decay_steps steps. Then, a warm restart is performed. Each new warm restart runs for t_mul times more steps and with m_mul times smaller initial learning rate. Example usage: first_decay_steps = 1000
lr_decayed = cosine_decay_restarts(learning_rate, global_step,
                                   first_decay_steps)

 


 Args
  learning_rate   A scalar float32 or float64 Tensor or a Python number. The initial learning rate.  
  global_step   A scalar int32 or int64 Tensor or a Python number. Global step to use for the decay computation.  
  first_decay_steps   A scalar int32 or int64 Tensor or a Python number. Number of steps to decay over.  
  t_mul   A scalar float32 or float64 Tensor or a Python number. Used to derive the number of iterations in the i-th period  
  m_mul   A scalar float32 or float64 Tensor or a Python number. Used to derive the initial learning rate of the i-th period:  
  alpha   A scalar float32 or float64 Tensor or a Python number. Minimum learning rate value as a fraction of the learning_rate.  
  name   String. Optional name of the operation. Defaults to 'SGDRDecay'.   
 


 Returns   A scalar Tensor of the same type as learning_rate. The decayed learning rate.  

 


 Raises
  ValueError   if global_step is not supplied.    References: Stochastic Gradient Descent with Warm Restarts: Loshchilov et al., 2017 (pdf) Eager Compatibility When eager execution is enabled, this function returns a function which in turn returns the decayed learning rate Tensor. This can be useful for changing the learning rate value across different invocations of optimizer functions.
tf.compat.v1.train.create_global_step Create global step tensor in graph. 
tf.compat.v1.train.create_global_step(
    graph=None
)

 


 Args
  graph   The graph in which to create the global step tensor. If missing, use default graph.   
 


 Returns   Global step tensor.  

 


 Raises
  ValueError   if global step tensor is already defined.
tf.compat.v1.train.do_quantize_training_on_graphdef A general quantization scheme is being developed in tf.contrib.quantize. (deprecated) 
tf.compat.v1.train.do_quantize_training_on_graphdef(
    input_graph, num_bits
)
 Warning: THIS FUNCTION IS DEPRECATED. It will be removed in a future version. Instructions for updating: GraphDef quantized training rewriter is deprecated in the long term. Consider using that instead, though since it is in the tf.contrib namespace, it is not subject to backward compatibility guarantees.
 


 Args
  input_graph   A GraphDef.  
  num_bits   The number of bits for quantize training.   
 


 Returns   The graph with quantize training done.
Module: tf.compat.v1.train.experimental Public API for tf.train.experimental namespace. Classes class DynamicLossScale: Loss scale that dynamically adjusts itself. class FixedLossScale: Loss scale with a fixed value. class LossScale: Base class for all TF1 loss scales. class MixedPrecisionLossScaleOptimizer: An optimizer that applies loss scaling. class PythonState: A mixin for putting Python state in an object-based checkpoint. Functions disable_mixed_precision_graph_rewrite(...): Disables the mixed precision graph rewrite. enable_mixed_precision_graph_rewrite(...): Enable mixed precision via a graph rewrite.
tf.compat.v1.train.exponential_decay Applies exponential decay to the learning rate. 
tf.compat.v1.train.exponential_decay(
    learning_rate, global_step, decay_steps, decay_rate, staircase=False, name=None
)
 When training a model, it is often recommended to lower the learning rate as the training progresses. This function applies an exponential decay function to a provided initial learning rate. It requires a global_step value to compute the decayed learning rate. You can just pass a TensorFlow variable that you increment at each training step. The function returns the decayed learning rate. It is computed as: decayed_learning_rate = learning_rate *
                        decay_rate ^ (global_step / decay_steps)
 If the argument staircase is True, then global_step / decay_steps is an integer division and the decayed learning rate follows a staircase function. Example: decay every 100000 steps with a base of 0.96: ...
global_step = tf.Variable(0, trainable=False)
starter_learning_rate = 0.1
learning_rate = tf.compat.v1.train.exponential_decay(starter_learning_rate,
global_step,
                                           100000, 0.96, staircase=True)
# Passing global_step to minimize() will increment it at each step.
learning_step = (
    tf.compat.v1.train.GradientDescentOptimizer(learning_rate)
    .minimize(...my loss..., global_step=global_step)
)

 


 Args
  learning_rate   A scalar float32 or float64 Tensor or a Python number. The initial learning rate.  
  global_step   A scalar int32 or int64 Tensor or a Python number. Global step to use for the decay computation. Must not be negative.  
  decay_steps   A scalar int32 or int64 Tensor or a Python number. Must be positive. See the decay computation above.  
  decay_rate   A scalar float32 or float64 Tensor or a Python number. The decay rate.  
  staircase   Boolean. If True decay the learning rate at discrete intervals  
  name   String. Optional name of the operation. Defaults to 'ExponentialDecay'.   
 


 Returns   A scalar Tensor of the same type as learning_rate. The decayed learning rate.  

 


 Raises
  ValueError   if global_step is not supplied.    Eager Compatibility When eager execution is enabled, this function returns a function which in turn returns the decayed learning rate Tensor. This can be useful for changing the learning rate value across different invocations of optimizer functions.
tf.compat.v1.train.export_meta_graph Returns MetaGraphDef proto. 
tf.compat.v1.train.export_meta_graph(
    filename=None, meta_info_def=None, graph_def=None, saver_def=None,
    collection_list=None, as_text=False, graph=None, export_scope=None,
    clear_devices=False, clear_extraneous_savers=False, strip_default_attrs=False,
    save_debug_info=False, **kwargs
)
 Optionally writes it to filename. This function exports the graph, saver, and collection objects into MetaGraphDef protocol buffer with the intention of it being imported at a later time or location to restart training, run inference, or be a subgraph.
 


 Args
  filename   Optional filename including the path for writing the generated MetaGraphDef protocol buffer.  
  meta_info_def   MetaInfoDef protocol buffer.  
  graph_def   GraphDef protocol buffer.  
  saver_def   SaverDef protocol buffer.  
  collection_list   List of string keys to collect.  
  as_text   If True, writes the MetaGraphDef as an ASCII proto.  
  graph   The Graph to export. If None, use the default graph.  
  export_scope   Optional string. Name scope under which to extract the subgraph. The scope name will be striped from the node definitions for easy import later into new name scopes. If None, the whole graph is exported. graph_def and export_scope cannot both be specified.  
  clear_devices   Whether or not to clear the device field for an Operation or Tensor during export.  
  clear_extraneous_savers   Remove any Saver-related information from the graph (both Save/Restore ops and SaverDefs) that are not associated with the provided SaverDef.  
  strip_default_attrs   Boolean. If True, default-valued attributes will be removed from the NodeDefs. For a detailed guide, see Stripping Default-Valued Attributes.  
  save_debug_info   If True, save the GraphDebugInfo to a separate file, which in the same directory of filename and with _debug added before the file extend.  
  **kwargs   Optional keyed arguments.   
 


 Returns   A MetaGraphDef proto.  

 


 Raises
  ValueError   When the GraphDef is larger than 2GB.  
  RuntimeError   If called with eager execution enabled.    Eager Compatibility Exporting/importing meta graphs is not supported unless both graph_def and graph are provided. No graph exists when eager execution is enabled.