neulab/docprompting-conala huggingface dataset:
tf.keras.optimizers.SGD     View source on GitHub    Gradient descent (with momentum) optimizer. Inherits From: Optimizer  View aliases  Main aliases 
tf.optimizers.SGD Compat aliases for migration See Migration guide for more details. tf.compat.v1.keras.optimizers.SGD  
tf.keras.optimizers.SGD(
    learning_rate=0.01, momentum=0.0, nesterov=False, name='SGD', **kwargs
)
 Update rule for parameter w with gradient g when momentum is 0: w = w - learning_rate * g
 Update rule when momentum is larger than 0: velocity = momentum * velocity - learning_rate * g
w = w + velocity
 When nesterov=True, this rule becomes: velocity = momentum * velocity - learning_rate * g
w = w + momentum * velocity - learning_rate * g

 


 Args
  learning_rate   A Tensor, floating point value, or a schedule that is a tf.keras.optimizers.schedules.LearningRateSchedule, or a callable that takes no arguments and returns the actual value to use. The learning rate. Defaults to 0.01.  
  momentum   float hyperparameter >= 0 that accelerates gradient descent in the relevant direction and dampens oscillations. Defaults to 0, i.e., vanilla gradient descent.  
  nesterov   boolean. Whether to apply Nesterov momentum. Defaults to False.  
  name   Optional name prefix for the operations created when applying gradients. Defaults to "SGD".  
  **kwargs   Keyword arguments. Allowed to be one of "clipnorm" or "clipvalue". "clipnorm" (float) clips gradients by norm; "clipvalue" (float) clips gradients by value.    Usage: 
opt = tf.keras.optimizers.SGD(learning_rate=0.1)
var = tf.Variable(1.0)
loss = lambda: (var ** 2)/2.0         # d(loss)/d(var1) = var1
step_count = opt.minimize(loss, [var]).numpy()
# Step is `- learning_rate * grad`
var.numpy()
0.9
 
opt = tf.keras.optimizers.SGD(learning_rate=0.1, momentum=0.9)
var = tf.Variable(1.0)
val0 = var.value()
loss = lambda: (var ** 2)/2.0         # d(loss)/d(var1) = var1
# First step is `- learning_rate * grad`
step_count = opt.minimize(loss, [var]).numpy()
val1 = var.value()
(val0 - val1).numpy()
0.1
# On later steps, step-size increases because of momentum
step_count = opt.minimize(loss, [var]).numpy()
val2 = var.value()
(val1 - val2).numpy()
0.18
 Reference:  For nesterov=True, See Sutskever et al., 2013. 
 


 Raises
  ValueError   in case of any invalid argument.
Module: tf.keras.preprocessing Keras data preprocessing utils. Modules image module: Set of tools for real-time data augmentation on image data. sequence module: Utilities for preprocessing sequence data. text module: Utilities for text input preprocessing. Functions image_dataset_from_directory(...): Generates a tf.data.Dataset from image files in a directory. text_dataset_from_directory(...): Generates a tf.data.Dataset from text files in a directory. timeseries_dataset_from_array(...): Creates a dataset of sliding windows over a timeseries provided as array.
Module: tf.keras.preprocessing.image Set of tools for real-time data augmentation on image data. Classes class DirectoryIterator: Iterator capable of reading images from a directory on disk. class ImageDataGenerator: Generate batches of tensor image data with real-time data augmentation. class Iterator: Base class for image data iterators. class NumpyArrayIterator: Iterator yielding data from a Numpy array. Functions apply_affine_transform(...): Applies an affine transformation specified by the parameters given. apply_brightness_shift(...): Performs a brightness shift. apply_channel_shift(...): Performs a channel shift. array_to_img(...): Converts a 3D Numpy array to a PIL Image instance. img_to_array(...): Converts a PIL Image instance to a Numpy array. load_img(...): Loads an image into PIL format. random_brightness(...): Performs a random brightness shift. random_channel_shift(...): Performs a random channel shift. random_rotation(...): Performs a random rotation of a Numpy image tensor. random_shear(...): Performs a random spatial shear of a Numpy image tensor. random_shift(...): Performs a random spatial shift of a Numpy image tensor. random_zoom(...): Performs a random spatial zoom of a Numpy image tensor. save_img(...): Saves an image stored as a Numpy array to a path or file object. smart_resize(...): Resize images to a target size without aspect ratio distortion.
tf.keras.preprocessing.image.apply_affine_transform     View source on GitHub    Applies an affine transformation specified by the parameters given.  View aliases  Compat aliases for migration 
See Migration guide for more details. tf.compat.v1.keras.preprocessing.image.apply_affine_transform  
tf.keras.preprocessing.image.apply_affine_transform(
    x, theta=0, tx=0, ty=0, shear=0, zx=1, zy=1, row_axis=0, col_axis=1,
    channel_axis=2, fill_mode='nearest', cval=0.0, order=1
)

 


 Arguments
  x   2D numpy array, single image.  
  theta   Rotation angle in degrees.  
  tx   Width shift.  
  ty   Heigh shift.  
  shear   Shear angle in degrees.  
  zx   Zoom in x direction.  
  zy   Zoom in y direction  
  row_axis   Index of axis for rows in the input image.  
  col_axis   Index of axis for columns in the input image.  
  channel_axis   Index of axis for channels in the input image.  
  fill_mode   Points outside the boundaries of the input are filled according to the given mode (one of {'constant', 'nearest', 'reflect', 'wrap'}).  
  cval   Value used for points outside the boundaries of the input if mode='constant'.  
  order   int, order of interpolation   
 


 Returns   The transformed version of the input.
tf.keras.preprocessing.image.apply_brightness_shift     View source on GitHub    Performs a brightness shift.  View aliases  Compat aliases for migration 
See Migration guide for more details. tf.compat.v1.keras.preprocessing.image.apply_brightness_shift  
tf.keras.preprocessing.image.apply_brightness_shift(
    x, brightness
)

 


 Arguments
  x   Input tensor. Must be 3D.  
  brightness   Float. The new brightness value.  
  channel_axis   Index of axis for channels in the input tensor.   
 


 Returns   Numpy image tensor.  

 


 Raises   ValueError if brightness_range isn't a tuple.
tf.keras.preprocessing.image.apply_channel_shift     View source on GitHub    Performs a channel shift.  View aliases  Compat aliases for migration 
See Migration guide for more details. tf.compat.v1.keras.preprocessing.image.apply_channel_shift  
tf.keras.preprocessing.image.apply_channel_shift(
    x, intensity, channel_axis=0
)

 


 Arguments
  x   Input tensor. Must be 3D.  
  intensity   Transformation intensity.  
  channel_axis   Index of axis for channels in the input tensor.   
 


 Returns   Numpy image tensor.
tf.keras.preprocessing.image.array_to_img     View source on GitHub    Converts a 3D Numpy array to a PIL Image instance.  View aliases  Compat aliases for migration 
See Migration guide for more details. tf.compat.v1.keras.preprocessing.image.array_to_img  
tf.keras.preprocessing.image.array_to_img(
    x, data_format=None, scale=True, dtype=None
)
 Usage: from PIL import Image
img = np.random.random(size=(100, 100, 3))
pil_img = tf.keras.preprocessing.image.array_to_img(img)

 


 Arguments
  x   Input Numpy array.  
  data_format   Image data format, can be either "channels_first" or "channels_last". Defaults to None, in which case the global setting tf.keras.backend.image_data_format() is used (unless you changed it, it defaults to "channels_last").  
  scale   Whether to rescale image values to be within [0, 255]. Defaults to True.  
  dtype   Dtype to use. Default to None, in which case the global setting tf.keras.backend.floatx() is used (unless you changed it, it defaults to "float32")   
 


 Returns   A PIL Image instance.  

 


 Raises
  ImportError   if PIL is not available.  
  ValueError   if invalid x or data_format is passed.
tf.keras.preprocessing.image.img_to_array     View source on GitHub    Converts a PIL Image instance to a Numpy array.  View aliases  Compat aliases for migration 
See Migration guide for more details. tf.compat.v1.keras.preprocessing.image.img_to_array  
tf.keras.preprocessing.image.img_to_array(
    img, data_format=None, dtype=None
)
 Usage: from PIL import Image
img_data = np.random.random(size=(100, 100, 3))
img = tf.keras.preprocessing.image.array_to_img(img_data)
array = tf.keras.preprocessing.image.img_to_array(img)

 


 Arguments
  img   Input PIL Image instance.  
  data_format   Image data format, can be either "channels_first" or "channels_last". Defaults to None, in which case the global setting tf.keras.backend.image_data_format() is used (unless you changed it, it defaults to "channels_last").  
  dtype   Dtype to use. Default to None, in which case the global setting tf.keras.backend.floatx() is used (unless you changed it, it defaults to "float32")   
 


 Returns   A 3D Numpy array.  

 


 Raises
  ValueError   if invalid img or data_format is passed.
tf.keras.preprocessing.image.Iterator     View source on GitHub    Base class for image data iterators. Inherits From: Sequence  View aliases  Compat aliases for migration 
See Migration guide for more details. tf.compat.v1.keras.preprocessing.image.Iterator  
tf.keras.preprocessing.image.Iterator(
    n, batch_size, shuffle, seed
)
 Every Iterator must implement the _get_batches_of_transformed_samples method.
 


 Arguments
  n   Integer, total number of samples in the dataset to loop over.  
  batch_size   Integer, size of a batch.  
  shuffle   Boolean, whether to shuffle the data between epochs.  
  seed   Random seeding for data shuffling.    Methods next View source 
next()
 For python 2.x.
 


 Returns   The next batch.  
 on_epoch_end View source 
on_epoch_end()
 reset View source 
reset()
 __getitem__ View source 
__getitem__(
    idx
)
 __iter__ View source 
__iter__()
 __len__ View source 
__len__()

 


 Class Variables
  white_list_formats
tf.keras.preprocessing.image.load_img     View source on GitHub    Loads an image into PIL format.  View aliases  Compat aliases for migration 
See Migration guide for more details. tf.compat.v1.keras.preprocessing.image.load_img  
tf.keras.preprocessing.image.load_img(
    path, grayscale=False, color_mode='rgb', target_size=None,
    interpolation='nearest'
)
 Usage: image = tf.keras.preprocessing.image.load_img(image_path)
input_arr = keras.preprocessing.image.img_to_array(image)
input_arr = np.array([input_arr])  # Convert single image to a batch.
predictions = model.predict(input_arr)

 


 Arguments
  path   Path to image file.  
  grayscale   DEPRECATED use color_mode="grayscale".  
  color_mode   One of "grayscale", "rgb", "rgba". Default: "rgb". The desired image format.  
  target_size   Either None (default to original size) or tuple of ints (img_height, img_width).  
  interpolation   Interpolation method used to resample the image if the target size is different from that of the loaded image. Supported methods are "nearest", "bilinear", and "bicubic". If PIL version 1.1.3 or newer is installed, "lanczos" is also supported. If PIL version 3.4.0 or newer is installed, "box" and "hamming" are also supported. By default, "nearest" is used.   
 


 Returns   A PIL Image instance.  

 


 Raises
  ImportError   if PIL is not available.  
  ValueError   if interpolation method is not supported.
tf.keras.preprocessing.image.NumpyArrayIterator     View source on GitHub    Iterator yielding data from a Numpy array. Inherits From: Iterator, Sequence  View aliases  Compat aliases for migration 
See Migration guide for more details. tf.compat.v1.keras.preprocessing.image.NumpyArrayIterator  
tf.keras.preprocessing.image.NumpyArrayIterator(
    x, y, image_data_generator, batch_size=32, shuffle=False, sample_weight=None,
    seed=None, data_format=None, save_to_dir=None, save_prefix='',
    save_format='png', subset=None, dtype=None
)

 


 Arguments
  x   Numpy array of input data or tuple. If tuple, the second elements is either another numpy array or a list of numpy arrays, each of which gets passed through as an output without any modifications.  
  y   Numpy array of targets data.  
  image_data_generator   Instance of ImageDataGenerator to use for random transformations and normalization.  
  batch_size   Integer, size of a batch.  
  shuffle   Boolean, whether to shuffle the data between epochs.  
  sample_weight   Numpy array of sample weights.  
  seed   Random seed for data shuffling.  
  data_format   String, one of channels_first, channels_last.  
  save_to_dir   Optional directory where to save the pictures being yielded, in a viewable format. This is useful for visualizing the random transformations being applied, for debugging purposes.  
  save_prefix   String prefix to use for saving sample images (if save_to_dir is set).  
  save_format   Format to use for saving sample images (if save_to_dir is set).  
  subset   Subset of data ("training" or "validation") if validation_split is set in ImageDataGenerator.  
  dtype   Dtype to use for the generated arrays.    Methods next View source 
next()
 For python 2.x.
 


 Returns   The next batch.  
 on_epoch_end View source 
on_epoch_end()
 reset View source 
reset()
 __getitem__ View source 
__getitem__(
    idx
)
 __iter__ View source 
__iter__()
 __len__ View source 
__len__()

 


 Class Variables
  white_list_formats
tf.keras.preprocessing.image.random_brightness     View source on GitHub    Performs a random brightness shift.  View aliases  Compat aliases for migration 
See Migration guide for more details. tf.compat.v1.keras.preprocessing.image.random_brightness  
tf.keras.preprocessing.image.random_brightness(
    x, brightness_range
)

 


 Arguments
  x   Input tensor. Must be 3D.  
  brightness_range   Tuple of floats; brightness range.  
  channel_axis   Index of axis for channels in the input tensor.   
 


 Returns   Numpy image tensor.  

 


 Raises   ValueError if brightness_range isn't a tuple.
tf.keras.preprocessing.image.random_channel_shift     View source on GitHub    Performs a random channel shift.  View aliases  Compat aliases for migration 
See Migration guide for more details. tf.compat.v1.keras.preprocessing.image.random_channel_shift  
tf.keras.preprocessing.image.random_channel_shift(
    x, intensity_range, channel_axis=0
)

 


 Arguments
  x   Input tensor. Must be 3D.  
  intensity_range   Transformation intensity.  
  channel_axis   Index of axis for channels in the input tensor.   
 


 Returns   Numpy image tensor.
tf.keras.preprocessing.image.random_rotation     View source on GitHub    Performs a random rotation of a Numpy image tensor.  View aliases  Compat aliases for migration 
See Migration guide for more details. tf.compat.v1.keras.preprocessing.image.random_rotation  
tf.keras.preprocessing.image.random_rotation(
    x, rg, row_axis=1, col_axis=2, channel_axis=0, fill_mode='nearest',
    cval=0.0, interpolation_order=1
)

 


 Arguments
  x   Input tensor. Must be 3D.  
  rg   Rotation range, in degrees.  
  row_axis   Index of axis for rows in the input tensor.  
  col_axis   Index of axis for columns in the input tensor.  
  channel_axis   Index of axis for channels in the input tensor.  
  fill_mode   Points outside the boundaries of the input are filled according to the given mode (one of {'constant', 'nearest', 'reflect', 'wrap'}).  
  cval   Value used for points outside the boundaries of the input if mode='constant'.  
  interpolation_order   int, order of spline interpolation. see ndimage.interpolation.affine_transform   
 


 Returns   Rotated Numpy image tensor.
tf.keras.preprocessing.image.random_shear     View source on GitHub    Performs a random spatial shear of a Numpy image tensor.  View aliases  Compat aliases for migration 
See Migration guide for more details. tf.compat.v1.keras.preprocessing.image.random_shear  
tf.keras.preprocessing.image.random_shear(
    x, intensity, row_axis=1, col_axis=2, channel_axis=0,
    fill_mode='nearest', cval=0.0, interpolation_order=1
)

 


 Arguments
  x   Input tensor. Must be 3D.  
  intensity   Transformation intensity in degrees.  
  row_axis   Index of axis for rows in the input tensor.  
  col_axis   Index of axis for columns in the input tensor.  
  channel_axis   Index of axis for channels in the input tensor.  
  fill_mode   Points outside the boundaries of the input are filled according to the given mode (one of {'constant', 'nearest', 'reflect', 'wrap'}).  
  cval   Value used for points outside the boundaries of the input if mode='constant'.  
  interpolation_order   int, order of spline interpolation. see ndimage.interpolation.affine_transform   
 


 Returns   Sheared Numpy image tensor.
tf.keras.preprocessing.image.random_shift     View source on GitHub    Performs a random spatial shift of a Numpy image tensor.  View aliases  Compat aliases for migration 
See Migration guide for more details. tf.compat.v1.keras.preprocessing.image.random_shift  
tf.keras.preprocessing.image.random_shift(
    x, wrg, hrg, row_axis=1, col_axis=2, channel_axis=0,
    fill_mode='nearest', cval=0.0, interpolation_order=1
)

 


 Arguments
  x   Input tensor. Must be 3D.  
  wrg   Width shift range, as a float fraction of the width.  
  hrg   Height shift range, as a float fraction of the height.  
  row_axis   Index of axis for rows in the input tensor.  
  col_axis   Index of axis for columns in the input tensor.  
  channel_axis   Index of axis for channels in the input tensor.  
  fill_mode   Points outside the boundaries of the input are filled according to the given mode (one of {'constant', 'nearest', 'reflect', 'wrap'}).  
  cval   Value used for points outside the boundaries of the input if mode='constant'.  
  interpolation_order   int, order of spline interpolation. see ndimage.interpolation.affine_transform   
 


 Returns   Shifted Numpy image tensor.
tf.keras.preprocessing.image.random_zoom     View source on GitHub    Performs a random spatial zoom of a Numpy image tensor.  View aliases  Compat aliases for migration 
See Migration guide for more details. tf.compat.v1.keras.preprocessing.image.random_zoom  
tf.keras.preprocessing.image.random_zoom(
    x, zoom_range, row_axis=1, col_axis=2, channel_axis=0,
    fill_mode='nearest', cval=0.0, interpolation_order=1
)

 


 Arguments
  x   Input tensor. Must be 3D.  
  zoom_range   Tuple of floats; zoom range for width and height.  
  row_axis   Index of axis for rows in the input tensor.  
  col_axis   Index of axis for columns in the input tensor.  
  channel_axis   Index of axis for channels in the input tensor.  
  fill_mode   Points outside the boundaries of the input are filled according to the given mode (one of {'constant', 'nearest', 'reflect', 'wrap'}).  
  cval   Value used for points outside the boundaries of the input if mode='constant'.  
  interpolation_order   int, order of spline interpolation. see ndimage.interpolation.affine_transform   
 


 Returns   Zoomed Numpy image tensor.  

 


 Raises
  ValueError   if zoom_range isn't a tuple.
tf.keras.preprocessing.image.save_img     View source on GitHub    Saves an image stored as a Numpy array to a path or file object.  View aliases  Compat aliases for migration 
See Migration guide for more details. tf.compat.v1.keras.preprocessing.image.save_img  
tf.keras.preprocessing.image.save_img(
    path, x, data_format=None, file_format=None, scale=True, **kwargs
)

 


 Arguments
  path   Path or file object.  
  x   Numpy array.  
  data_format   Image data format, either "channels_first" or "channels_last".  
  file_format   Optional file format override. If omitted, the format to use is determined from the filename extension. If a file object was used instead of a filename, this parameter should always be used.  
  scale   Whether to rescale image values to be within [0, 255].  
  **kwargs   Additional keyword arguments passed to PIL.Image.save().
tf.keras.preprocessing.image.smart_resize Resize images to a target size without aspect ratio distortion. 
tf.keras.preprocessing.image.smart_resize(
    x, size, interpolation='bilinear'
)
 TensorFlow image datasets typically yield images that have each a different size. However, these images need to be batched before they can be processed by Keras layers. To be batched, images need to share the same height and width. You could simply do: size = (200, 200)
ds = ds.map(lambda img: tf.image.resize(img, size))
```

However, if you do this, you distort the aspect ratio of your images, since
in general they do not all have the same aspect ratio as `size`. This is
fine in many cases, but not always (e.g. for GANs this can be a problem).

Note that passing the argument `preserve_aspect_ratio=True` to `resize`
will preserve the aspect ratio, but at the cost of no longer respecting the
provided target size. Because <a href="../../../../tf/image/resize"><code>tf.image.resize</code></a> doesn't crop images,
your output images will still have different sizes.

#### This calls for:



```python
size = (200, 200)
ds = ds.map(lambda img: smart_resize(img, size))
```

Your output images will actually be `(200, 200)`, and will not be distorted.
Instead, the parts of the image that do not fit within the target size
get cropped out.

The resizing process is:

1. Take the largest centered crop of the image that has the same aspect ratio
as the target size. For instance, if `size=(200, 200)` and the input image has
size `(340, 500)`, we take a crop of `(340, 340)` centered along the width.
2. Resize the cropped image to the target size. In the example above,
we resize the `(340, 340)` crop to `(200, 200)`.

<!-- Tabular view -->
 <table class="responsive fixed orange">
<colgroup><col width="214px"><col></colgroup>
<tr><th colspan="2"><h2 class="add-link">Arguments</h2></th></tr>

<tr>
<td>
`x`
</td>
<td>
Input image (as a tensor or NumPy array). Must be in format
`(height, width, channels)`.
</td>
</tr><tr>
<td>
`size`
</td>
<td>
Tuple of `(height, width)` integer. Target size.
</td>
</tr><tr>
<td>
`interpolation`
</td>
<td>
String, interpolation to use for resizing.
Defaults to `'bilinear'`. Supports `bilinear`, `nearest`, `bicubic`,
`area`, `lanczos3`, `lanczos5`, `gaussian`, `mitchellcubic`.
</td>
</tr>
</table>



<!-- Tabular view -->
 <table class="responsive fixed orange">
<colgroup><col width="214px"><col></colgroup>
<tr><th colspan="2"><h2 class="add-link">Returns</h2></th></tr>
<tr class="alt">
<td colspan="2">
Array with shape `(size[0], size[1], channels)`. If the input image was a
NumPy array, the output is a NumPy array, and if it was a TF tensor,
the output is a TF tensor.
</td>
</tr>

</table>
Module: tf.keras.preprocessing.sequence Utilities for preprocessing sequence data. Classes class TimeseriesGenerator: Utility class for generating batches of temporal data. Functions make_sampling_table(...): Generates a word rank-based probabilistic sampling table. pad_sequences(...): Pads sequences to the same length. skipgrams(...): Generates skipgram word pairs.
tf.keras.preprocessing.sequence.make_sampling_table     View source on GitHub    Generates a word rank-based probabilistic sampling table.  View aliases  Compat aliases for migration 
See Migration guide for more details. tf.compat.v1.keras.preprocessing.sequence.make_sampling_table  
tf.keras.preprocessing.sequence.make_sampling_table(
    size, sampling_factor=1e-05
)
 Used for generating the sampling_table argument for skipgrams. sampling_table[i] is the probability of sampling the word i-th most common word in a dataset (more common words should be sampled less frequently, for balance). The sampling probabilities are generated according to the sampling distribution used in word2vec: p(word) = (min(1, sqrt(word_frequency / sampling_factor) /
    (word_frequency / sampling_factor)))
 We assume that the word frequencies follow Zipf's law (s=1) to derive a numerical approximation of frequency(rank): frequency(rank) ~ 1/(rank * (log(rank) + gamma) + 1/2 - 1/(12*rank)) where gamma is the Euler-Mascheroni constant.
 


 Arguments
  size   Int, number of possible words to sample.  
  sampling_factor   The sampling factor in the word2vec formula.   
 


 Returns   A 1D Numpy array of length size where the ith entry is the probability that a word of rank i should be sampled.