neulab/docprompting-conala huggingface dataset:
tf.keras.preprocessing.sequence.pad_sequences     View source on GitHub    Pads sequences to the same length.  View aliases  Compat aliases for migration 
See Migration guide for more details. tf.compat.v1.keras.preprocessing.sequence.pad_sequences  
tf.keras.preprocessing.sequence.pad_sequences(
    sequences, maxlen=None, dtype='int32', padding='pre',
    truncating='pre', value=0.0
)
 This function transforms a list (of length num_samples) of sequences (lists of integers) into a 2D Numpy array of shape (num_samples, num_timesteps). num_timesteps is either the maxlen argument if provided, or the length of the longest sequence in the list. Sequences that are shorter than num_timesteps are padded with value until they are num_timesteps long. Sequences longer than num_timesteps are truncated so that they fit the desired length. The position where padding or truncation happens is determined by the arguments padding and truncating, respectively. Pre-padding or removing values from the beginning of the sequence is the default. 
sequence = [[1], [2, 3], [4, 5, 6]]
tf.keras.preprocessing.sequence.pad_sequences(sequence)
array([[0, 0, 1],
       [0, 2, 3],
       [4, 5, 6]], dtype=int32)
 
tf.keras.preprocessing.sequence.pad_sequences(sequence, value=-1)
array([[-1, -1,  1],
       [-1,  2,  3],
       [ 4,  5,  6]], dtype=int32)
 
tf.keras.preprocessing.sequence.pad_sequences(sequence, padding='post')
array([[1, 0, 0],
       [2, 3, 0],
       [4, 5, 6]], dtype=int32)
 
tf.keras.preprocessing.sequence.pad_sequences(sequence, maxlen=2)
array([[0, 1],
       [2, 3],
       [5, 6]], dtype=int32)

 


 Arguments
  sequences   List of sequences (each sequence is a list of integers).  
  maxlen   Optional Int, maximum length of all sequences. If not provided, sequences will be padded to the length of the longest individual sequence.  
  dtype   (Optional, defaults to int32). Type of the output sequences. To pad sequences with variable length strings, you can use object.  
  padding   String, 'pre' or 'post' (optional, defaults to 'pre'): pad either before or after each sequence.  
  truncating   String, 'pre' or 'post' (optional, defaults to 'pre'): remove values from sequences larger than maxlen, either at the beginning or at the end of the sequences.  
  value   Float or String, padding value. (Optional, defaults to 0.)   
 


 Returns   Numpy array with shape (len(sequences), maxlen)  

 


 Raises
  ValueError   In case of invalid values for truncating or padding, or in case of invalid shape for a sequences entry.
tf.keras.preprocessing.sequence.skipgrams     View source on GitHub    Generates skipgram word pairs.  View aliases  Compat aliases for migration 
See Migration guide for more details. tf.compat.v1.keras.preprocessing.sequence.skipgrams  
tf.keras.preprocessing.sequence.skipgrams(
    sequence, vocabulary_size, window_size=4, negative_samples=1.0, shuffle=True,
    categorical=False, sampling_table=None, seed=None
)
 This function transforms a sequence of word indexes (list of integers) into tuples of words of the form:  (word, word in the same window), with label 1 (positive samples). (word, random word from the vocabulary), with label 0 (negative samples).  Read more about Skipgram in this gnomic paper by Mikolov et al.: Efficient Estimation of Word Representations in Vector Space
 


 Arguments
  sequence   A word sequence (sentence), encoded as a list of word indices (integers). If using a sampling_table, word indices are expected to match the rank of the words in a reference dataset (e.g. 10 would encode the 10-th most frequently occurring token). Note that index 0 is expected to be a non-word and will be skipped.  
  vocabulary_size   Int, maximum possible word index + 1  
  window_size   Int, size of sampling windows (technically half-window). The window of a word w_i will be [i - window_size, i + window_size+1].  
  negative_samples   Float >= 0. 0 for no negative (i.e. random) samples. 1 for same number as positive samples.  
  shuffle   Whether to shuffle the word couples before returning them.  
  categorical   bool. if False, labels will be integers (eg. [0, 1, 1 .. ]), if True, labels will be categorical, e.g. [[1,0],[0,1],[0,1] .. ].  
  sampling_table   1D array of size vocabulary_size where the entry i encodes the probability to sample a word of rank i.  
  seed   Random seed.   
 


 Returns   couples, labels: where couples are int pairs and labels are either 0 or 1.  
 Note: By convention, index 0 in the vocabulary is a non-word and will be skipped.
Module: tf.keras.preprocessing.text Utilities for text input preprocessing. Classes class Tokenizer: Text tokenization utility class. Functions hashing_trick(...): Converts a text to a sequence of indexes in a fixed-size hashing space. one_hot(...): One-hot encodes a text into a list of word indexes of size n. text_to_word_sequence(...): Converts a text to a sequence of words (or tokens). tokenizer_from_json(...): Parses a JSON tokenizer configuration file and returns a
tf.keras.preprocessing.text.hashing_trick       View source on GitHub    Converts a text to a sequence of indexes in a fixed-size hashing space.  View aliases  Compat aliases for migration 
See Migration guide for more details. tf.compat.v1.keras.preprocessing.text.hashing_trick  
tf.keras.preprocessing.text.hashing_trick(
    text, n, hash_function=None,
    filters='!"#$%&()*+,-./:;<=>?@[\\]^_`{|}~\t\n',
    lower=True, split=' '
)

 


 Arguments
  text   Input text (string).  
  n   Dimension of the hashing space.  
  hash_function   defaults to python hash function, can be 'md5' or any function that takes in input a string and returns a int. Note that 'hash' is not a stable hashing function, so it is not consistent across different runs, while 'md5' is a stable hashing function.  
  filters   list (or concatenation) of characters to filter out, such as punctuation. Default: !"#$%&()*+,-./:;<=>?@[\]^_`{|}~\t\n, includes basic punctuation, tabs, and newlines.  
  lower   boolean. Whether to set the text to lowercase.  
  split   str. Separator for word splitting.   
 


 Returns   A list of integer word indices (unicity non-guaranteed).  
 0 is a reserved index that won't be assigned to any word. Two or more words may be assigned to the same index, due to possible collisions by the hashing function. The probability of a collision is in relation to the dimension of the hashing space and the number of distinct objects.
tf.keras.preprocessing.text.one_hot       View source on GitHub    One-hot encodes a text into a list of word indexes of size n.  View aliases  Compat aliases for migration 
See Migration guide for more details. tf.compat.v1.keras.preprocessing.text.one_hot  
tf.keras.preprocessing.text.one_hot(
    input_text, n,
    filters='!"#$%&()*+,-./:;<=>?@[\\]^_`{|}~\t\n',
    lower=True, split=' '
)
 This function receives as input a string of text and returns a list of encoded integers each corresponding to a word (or token) in the given input string.
 


 Arguments
  input_text   Input text (string).  
  n   int. Size of vocabulary.  
  filters   list (or concatenation) of characters to filter out, such as punctuation. Default: '!"#$%&()*+,-./:;<=>?@[\]^_`{|}~\t\n, includes basic punctuation, tabs, and newlines.  
  lower   boolean. Whether to set the text to lowercase.  
  split   str. Separator for word splitting.   
 


 Returns   List of integers in [1, n]. Each integer encodes a word (unicity non-guaranteed).
tf.keras.preprocessing.text.text_to_word_sequence       View source on GitHub    Converts a text to a sequence of words (or tokens).  View aliases  Compat aliases for migration 
See Migration guide for more details. tf.compat.v1.keras.preprocessing.text.text_to_word_sequence  
tf.keras.preprocessing.text.text_to_word_sequence(
    input_text,
    filters='!"#$%&()*+,-./:;<=>?@[\\]^_`{|}~\t\n',
    lower=True, split=' '
)
 This function transforms a string of text into a list of words while ignoring filters which include punctuations by default. 
sample_text = 'This is a sample sentence.'
tf.keras.preprocessing.text.text_to_word_sequence(sample_text)
['this', 'is', 'a', 'sample', 'sentence']

 


 Arguments
  input_text   Input text (string).  
  filters   list (or concatenation) of characters to filter out, such as punctuation. Default: '!"#$%&()*+,-./:;<=>?@[\]^_{|}~\t\n', includes basic punctuation, tabs, and newlines. </td> </tr><tr> <td>lower</td> <td> boolean. Whether to convert the input to lowercase. </td> </tr><tr> <td>split`   str. Separator for word splitting.   
 


 Returns   A list of words (or tokens).
tf.keras.preprocessing.text.tokenizer_from_json Parses a JSON tokenizer configuration file and returns a  View aliases  Compat aliases for migration 
See Migration guide for more details. tf.compat.v1.keras.preprocessing.text.tokenizer_from_json  
tf.keras.preprocessing.text.tokenizer_from_json(
    json_string
)
 tokenizer instance.
 


 Arguments
  json_string   JSON string encoding a tokenizer configuration.   
 


 Returns   A Keras Tokenizer instance
Module: tf.keras.regularizers Built-in regularizers. Classes class L1: A regularizer that applies a L1 regularization penalty. class L1L2: A regularizer that applies both L1 and L2 regularization penalties. class L2: A regularizer that applies a L2 regularization penalty. class Regularizer: Regularizer base class. class l1: A regularizer that applies a L1 regularization penalty. class l2: A regularizer that applies a L2 regularization penalty. Functions deserialize(...) get(...): Retrieve a regularizer instance from a config or identifier. l1_l2(...): Create a regularizer that applies both L1 and L2 penalties. serialize(...)
tf.keras.regularizers.deserialize     View source on GitHub     View aliases  Compat aliases for migration 
See Migration guide for more details. tf.compat.v1.keras.regularizers.deserialize  
tf.keras.regularizers.deserialize(
    config, custom_objects=None
)
tf.keras.regularizers.get     View source on GitHub    Retrieve a regularizer instance from a config or identifier.  View aliases  Compat aliases for migration 
See Migration guide for more details. tf.compat.v1.keras.regularizers.get  
tf.keras.regularizers.get(
    identifier
)
tf.keras.regularizers.L1 A regularizer that applies a L1 regularization penalty. Inherits From: Regularizer  View aliases  Main aliases 
tf.keras.regularizers.l1 Compat aliases for migration See Migration guide for more details. tf.compat.v1.keras.regularizers.L1, tf.compat.v1.keras.regularizers.l1  
tf.keras.regularizers.L1(
    l1=0.01, **kwargs
)
 The L1 regularization penalty is computed as: loss = l1 * reduce_sum(abs(x)) L1 may be passed to a layer as a string identifier: 
dense = tf.keras.layers.Dense(3, kernel_regularizer='l1')
 In this case, the default value used is l1=0.01.
 


 Attributes
  l1   Float; L1 regularization factor.    Methods from_config View source 
@classmethod
from_config(
    config
)
 Creates a regularizer from its config. This method is the reverse of get_config, capable of instantiating the same regularizer from the config dictionary. This method is used by Keras model_to_estimator, saving and loading models to HDF5 formats, Keras model cloning, some visualization utilities, and exporting models to and from JSON.
 


 Arguments
  config   A Python dictionary, typically the output of get_config.   
 


 Returns   A regularizer instance.  
 get_config View source 
get_config()
 Returns the config of the regularizer. An regularizer config is a Python dictionary (serializable) containing all configuration parameters of the regularizer. The same regularizer can be reinstantiated later (without any saved state) from this configuration. This method is optional if you are just training and executing models, exporting to and from SavedModels, or using weight checkpoints. This method is required for Keras model_to_estimator, saving and loading models to HDF5 formats, Keras model cloning, some visualization utilities, and exporting models to and from JSON.
 


 Returns   Python dictionary.  
 __call__ View source 
__call__(
    x
)
 Compute a regularization penalty from an input tensor.
tf.keras.regularizers.L1L2     View source on GitHub    A regularizer that applies both L1 and L2 regularization penalties. Inherits From: Regularizer  View aliases  Compat aliases for migration 
See Migration guide for more details. tf.compat.v1.keras.regularizers.L1L2  
tf.keras.regularizers.L1L2(
    l1=0.0, l2=0.0
)
 The L1 regularization penalty is computed as: loss = l1 * reduce_sum(abs(x)) The L2 regularization penalty is computed as loss = l2 * reduce_sum(square(x)) L1L2 may be passed to a layer as a string identifier: 
dense = tf.keras.layers.Dense(3, kernel_regularizer='l1_l2')
 In this case, the default values used are l1=0.01 and l2=0.01.
 


 Attributes
  l1   Float; L1 regularization factor.  
  l2   Float; L2 regularization factor.    Methods from_config View source 
@classmethod
from_config(
    config
)
 Creates a regularizer from its config. This method is the reverse of get_config, capable of instantiating the same regularizer from the config dictionary. This method is used by Keras model_to_estimator, saving and loading models to HDF5 formats, Keras model cloning, some visualization utilities, and exporting models to and from JSON.
 


 Arguments
  config   A Python dictionary, typically the output of get_config.   
 


 Returns   A regularizer instance.  
 get_config View source 
get_config()
 Returns the config of the regularizer. An regularizer config is a Python dictionary (serializable) containing all configuration parameters of the regularizer. The same regularizer can be reinstantiated later (without any saved state) from this configuration. This method is optional if you are just training and executing models, exporting to and from SavedModels, or using weight checkpoints. This method is required for Keras model_to_estimator, saving and loading models to HDF5 formats, Keras model cloning, some visualization utilities, and exporting models to and from JSON.
 


 Returns   Python dictionary.  
 __call__ View source 
__call__(
    x
)
 Compute a regularization penalty from an input tensor.
tf.keras.regularizers.l1_l2     View source on GitHub    Create a regularizer that applies both L1 and L2 penalties.  View aliases  Compat aliases for migration 
See Migration guide for more details. tf.compat.v1.keras.regularizers.l1_l2  
tf.keras.regularizers.l1_l2(
    l1=0.01, l2=0.01
)
 The L1 regularization penalty is computed as: loss = l1 * reduce_sum(abs(x)) The L2 regularization penalty is computed as: loss = l2 * reduce_sum(square(x))
 


 Arguments
  l1   Float; L1 regularization factor.  
  l2   Float; L2 regularization factor.   
 


 Returns   An L1L2 Regularizer with the given regularization factors.
tf.keras.regularizers.L2 A regularizer that applies a L2 regularization penalty. Inherits From: Regularizer  View aliases  Main aliases 
tf.keras.regularizers.l2 Compat aliases for migration See Migration guide for more details. tf.compat.v1.keras.regularizers.L2, tf.compat.v1.keras.regularizers.l2  
tf.keras.regularizers.L2(
    l2=0.01, **kwargs
)
 The L2 regularization penalty is computed as: loss = l2 * reduce_sum(square(x)) L2 may be passed to a layer as a string identifier: 
dense = tf.keras.layers.Dense(3, kernel_regularizer='l2')
 In this case, the default value used is l2=0.01.
 


 Attributes
  l2   Float; L2 regularization factor.    Methods from_config View source 
@classmethod
from_config(
    config
)
 Creates a regularizer from its config. This method is the reverse of get_config, capable of instantiating the same regularizer from the config dictionary. This method is used by Keras model_to_estimator, saving and loading models to HDF5 formats, Keras model cloning, some visualization utilities, and exporting models to and from JSON.
 


 Arguments
  config   A Python dictionary, typically the output of get_config.   
 


 Returns   A regularizer instance.  
 get_config View source 
get_config()
 Returns the config of the regularizer. An regularizer config is a Python dictionary (serializable) containing all configuration parameters of the regularizer. The same regularizer can be reinstantiated later (without any saved state) from this configuration. This method is optional if you are just training and executing models, exporting to and from SavedModels, or using weight checkpoints. This method is required for Keras model_to_estimator, saving and loading models to HDF5 formats, Keras model cloning, some visualization utilities, and exporting models to and from JSON.
 


 Returns   Python dictionary.  
 __call__ View source 
__call__(
    x
)
 Compute a regularization penalty from an input tensor.
tf.keras.regularizers.serialize     View source on GitHub     View aliases  Compat aliases for migration 
See Migration guide for more details. tf.compat.v1.keras.regularizers.serialize  
tf.keras.regularizers.serialize(
    regularizer
)
Module: tf.keras.utils Public API for tf.keras.utils namespace. Classes class CustomObjectScope: Exposes custom classes/functions to Keras deserialization internals. class GeneratorEnqueuer: Builds a queue out of a data generator. class OrderedEnqueuer: Builds a Enqueuer from a Sequence. class Progbar: Displays a progress bar. class Sequence: Base object for fitting to a sequence of data, such as a dataset. class SequenceEnqueuer: Base class to enqueue inputs. class custom_object_scope: Exposes custom classes/functions to Keras deserialization internals. Functions deserialize_keras_object(...): Turns the serialized form of a Keras object back into an actual object. get_custom_objects(...): Retrieves a live reference to the global dictionary of custom objects. get_file(...): Downloads a file from a URL if it not already in the cache. get_registered_name(...): Returns the name registered to an object within the Keras framework. get_registered_object(...): Returns the class associated with name if it is registered with Keras. get_source_inputs(...): Returns the list of input tensors necessary to compute tensor. model_to_dot(...): Convert a Keras model to dot format. normalize(...): Normalizes a Numpy array. pack_x_y_sample_weight(...): Packs user-provided data into a tuple. plot_model(...): Converts a Keras model to dot format and save to a file. register_keras_serializable(...): Registers an object with the Keras serialization framework. serialize_keras_object(...): Serialize a Keras object into a JSON-compatible representation. to_categorical(...): Converts a class vector (integers) to binary class matrix. unpack_x_y_sample_weight(...): Unpacks user-provided data tuple.
tf.keras.utils.CustomObjectScope     View source on GitHub    Exposes custom classes/functions to Keras deserialization internals.  View aliases  Main aliases 
tf.keras.utils.custom_object_scope Compat aliases for migration See Migration guide for more details. tf.compat.v1.keras.utils.CustomObjectScope, tf.compat.v1.keras.utils.custom_object_scope  
tf.keras.utils.CustomObjectScope(
    *args
)
 Under a scope with custom_object_scope(objects_dict), Keras methods such as tf.keras.models.load_model or tf.keras.models.model_from_config will be able to deserialize any custom object referenced by a saved config (e.g. a custom layer or metric). Example: Consider a custom regularizer my_regularizer: layer = Dense(3, kernel_regularizer=my_regularizer)
config = layer.get_config()  # Config contains a reference to `my_regularizer`
...
# Later:
with custom_object_scope({'my_regularizer': my_regularizer}):
  layer = Dense.from_config(config)

 


 Arguments
  *args   Dictionary or dictionaries of {name: object} pairs.    Methods __enter__ View source 
__enter__()
 __exit__ View source 
__exit__(
    *args, **kwargs
)
tf.keras.utils.deserialize_keras_object     View source on GitHub    Turns the serialized form of a Keras object back into an actual object.  View aliases  Compat aliases for migration 
See Migration guide for more details. tf.compat.v1.keras.utils.deserialize_keras_object  
tf.keras.utils.deserialize_keras_object(
    identifier, module_objects=None, custom_objects=None,
    printable_module_name='object'
)
tf.keras.utils.GeneratorEnqueuer     View source on GitHub    Builds a queue out of a data generator. Inherits From: SequenceEnqueuer  View aliases  Compat aliases for migration 
See Migration guide for more details. tf.compat.v1.keras.utils.GeneratorEnqueuer  
tf.keras.utils.GeneratorEnqueuer(
    sequence, use_multiprocessing=False, random_seed=None
)
 The provided generator can be finite in which case the class will throw a StopIteration exception. Used in fit_generator, evaluate_generator, predict_generator.
 


 Arguments
  generator   a generator function which yields data  
  use_multiprocessing   use multiprocessing if True, otherwise threading  
  wait_time   time to sleep in-between calls to put()  
  random_seed   Initial seed for workers, will be incremented by one for each worker.    Methods get View source 
get()
 Creates a generator to extract data from the queue. Skip the data if it is None.
 


 Yields   The next element in the queue, i.e. a tuple (inputs, targets) or (inputs, targets, sample_weights).  
 is_running View source 
is_running()
 start View source 
start(
    workers=1, max_queue_size=10
)
 Starts the handler's workers.
 


 Arguments
  workers   Number of workers.  
  max_queue_size   queue size (when full, workers could block on put())    stop View source 
stop(
    timeout=None
)
 Stops running threads and wait for them to exit, if necessary. Should be called by the same thread which called start().
 


 Arguments
  timeout   maximum time to wait on thread.join()
tf.keras.utils.get_custom_objects     View source on GitHub    Retrieves a live reference to the global dictionary of custom objects.  View aliases  Compat aliases for migration 
See Migration guide for more details. tf.compat.v1.keras.utils.get_custom_objects  
tf.keras.utils.get_custom_objects()
 Updating and clearing custom objects using custom_object_scope is preferred, but get_custom_objects can be used to directly access the current collection of custom objects. Example: get_custom_objects().clear()
get_custom_objects()['MyObject'] = MyObject

 


 Returns   Global dictionary of names to classes (_GLOBAL_CUSTOM_OBJECTS).