Utilities for Image Processors
This page lists all the utility functions used by the image processors, mainly the functional transformations used to process the images.
Most of those are only useful if you are studying the code of the image processors in the library.
Image Transformations
transformers.image_transforms.center_crop
< source >(
image: ndarray
size: typing.Tuple[int, int]
data_format: typing.Union[transformers.image_utils.ChannelDimension, str, NoneType] = None
input_data_format: typing.Union[transformers.image_utils.ChannelDimension, str, NoneType] = None
return_numpy: typing.Optional[bool] = None
)
→
np.ndarray
Parameters
-
image (
np.ndarray
) — The image to crop. -
size (
Tuple[int, int]
) — The target size for the cropped image. -
data_format (
str
orChannelDimension
, optional) — The channel dimension format for the output image. Can be one of:"channels_first"
orChannelDimension.FIRST
: image in (num_channels, height, width) format."channels_last"
orChannelDimension.LAST
: image in (height, width, num_channels) format. If unset, will use the inferred format of the input image.
-
input_data_format (
str
orChannelDimension
, optional) — The channel dimension format for the input image. Can be one of:"channels_first"
orChannelDimension.FIRST
: image in (num_channels, height, width) format."channels_last"
orChannelDimension.LAST
: image in (height, width, num_channels) format. If unset, will use the inferred format of the input image.
-
return_numpy (
bool
, optional) — Whether or not to return the cropped image as a numpy array. Used for backwards compatibility with the previous ImageFeatureExtractionMixin method.- Unset: will return the same type as the input image.
True
: will return a numpy array.False
: will return aPIL.Image.Image
object.
Returns
np.ndarray
The cropped image.
Crops the image
to the specified size
using a center crop. Note that if the image is too small to be cropped to
the size given, it will be padded (so the returned result will always be of size size
).
Converts bounding boxes from center format to corners format.
center format: contains the coordinate for the center of the box and its width, height dimensions (center_x, center_y, width, height) corners format: contains the coodinates for the top-left and bottom-right corners of the box (top_left_x, top_left_y, bottom_right_x, bottom_right_y)
Converts bounding boxes from corners format to center format.
corners format: contains the coodinates for the top-left and bottom-right corners of the box (top_left_x, top_left_y, bottom_right_x, bottom_right_y) center format: contains the coordinate for the center of the box and its the width, height dimensions (center_x, center_y, width, height)
Converts unique ID to RGB color.
transformers.image_transforms.normalize
< source >( image: ndarray mean: typing.Union[float, typing.Iterable[float]] std: typing.Union[float, typing.Iterable[float]] data_format: typing.Optional[transformers.image_utils.ChannelDimension] = None input_data_format: typing.Union[transformers.image_utils.ChannelDimension, str, NoneType] = None )
Parameters
-
image (
np.ndarray
) — The image to normalize. -
mean (
float
orIterable[float]
) — The mean to use for normalization. -
std (
float
orIterable[float]
) — The standard deviation to use for normalization. -
data_format (
ChannelDimension
, optional) — The channel dimension format of the output image. If unset, will use the inferred format from the input. -
input_data_format (
ChannelDimension
, optional) — The channel dimension format of the input image. If unset, will use the inferred format from the input.
Normalizes image
using the mean and standard deviation specified by mean
and std
.
image = (image - mean) / std
transformers.image_transforms.pad
< source >(
image: ndarray
padding: typing.Union[int, typing.Tuple[int, int], typing.Iterable[typing.Tuple[int, int]]]
mode: PaddingMode = <PaddingMode.CONSTANT: 'constant'>
constant_values: typing.Union[float, typing.Iterable[float]] = 0.0
data_format: typing.Union[transformers.image_utils.ChannelDimension, str, NoneType] = None
input_data_format: typing.Union[transformers.image_utils.ChannelDimension, str, NoneType] = None
)
→
np.ndarray
Parameters
-
image (
np.ndarray
) — The image to pad. -
padding (
int
orTuple[int, int]
orIterable[Tuple[int, int]]
) — Padding to apply to the edges of the height, width axes. Can be one of three formats:((before_height, after_height), (before_width, after_width))
unique pad widths for each axis.((before, after),)
yields same before and after pad for height and width.(pad,)
or int is a shortcut for before = after = pad width for all axes.
-
mode (
PaddingMode
) — The padding mode to use. Can be one of:"constant"
: pads with a constant value."reflect"
: pads with the reflection of the vector mirrored on the first and last values of the vector along each axis."replicate"
: pads with the replication of the last value on the edge of the array along each axis."symmetric"
: pads with the reflection of the vector mirrored along the edge of the array.
-
constant_values (
float
orIterable[float]
, optional) — The value to use for the padding ifmode
is"constant"
. -
data_format (
str
orChannelDimension
, optional) — The channel dimension format for the output image. Can be one of:"channels_first"
orChannelDimension.FIRST
: image in (num_channels, height, width) format."channels_last"
orChannelDimension.LAST
: image in (height, width, num_channels) format. If unset, will use same as the input image.
-
input_data_format (
str
orChannelDimension
, optional) — The channel dimension format for the input image. Can be one of:"channels_first"
orChannelDimension.FIRST
: image in (num_channels, height, width) format."channels_last"
orChannelDimension.LAST
: image in (height, width, num_channels) format. If unset, will use the inferred format of the input image.
Returns
np.ndarray
The padded image.
Pads the image
with the specified (height, width) padding
and mode
.
Converts RGB color to unique ID.
transformers.image_transforms.rescale
< source >(
image: ndarray
scale: float
data_format: typing.Optional[transformers.image_utils.ChannelDimension] = None
dtype: dtype = <class 'numpy.float32'>
input_data_format: typing.Union[transformers.image_utils.ChannelDimension, str, NoneType] = None
)
→
np.ndarray
Parameters
-
image (
np.ndarray
) — The image to rescale. -
scale (
float
) — The scale to use for rescaling the image. -
data_format (
ChannelDimension
, optional) — The channel dimension format of the image. If not provided, it will be the same as the input image. -
dtype (
np.dtype
, optional, defaults tonp.float32
) — The dtype of the output image. Defaults tonp.float32
. Used for backwards compatibility with feature extractors. -
input_data_format (
ChannelDimension
, optional) — The channel dimension format of the input image. If not provided, it will be inferred from the input image.
Returns
np.ndarray
The rescaled image.
Rescales image
by scale
.
transformers.image_transforms.resize
< source >(
image
size: typing.Tuple[int, int]
resample: PILImageResampling = None
reducing_gap: typing.Optional[int] = None
data_format: typing.Optional[transformers.image_utils.ChannelDimension] = None
return_numpy: bool = True
input_data_format: typing.Union[transformers.image_utils.ChannelDimension, str, NoneType] = None
)
→
np.ndarray
Parameters
-
image (
PIL.Image.Image
ornp.ndarray
ortorch.Tensor
) — The image to resize. -
size (
Tuple[int, int]
) — The size to use for resizing the image. -
resample (
int
, optional, defaults toPILImageResampling.BILINEAR
) — The filter to user for resampling. -
reducing_gap (
int
, optional) — Apply optimization by resizing the image in two steps. The biggerreducing_gap
, the closer the result to the fair resampling. See corresponding Pillow documentation for more details. -
data_format (
ChannelDimension
, optional) — The channel dimension format of the output image. If unset, will use the inferred format from the input. -
return_numpy (
bool
, optional, defaults toTrue
) — Whether or not to return the resized image as a numpy array. If False aPIL.Image.Image
object is returned. -
input_data_format (
ChannelDimension
, optional) — The channel dimension format of the input image. If unset, will use the inferred format from the input.
Returns
np.ndarray
The resized image.
Resizes image
to (height, width)
specified by size
using the PIL library.
transformers.image_transforms.to_pil_image
< source >(
image: typing.Union[numpy.ndarray, ForwardRef('PIL.Image.Image'), ForwardRef('torch.Tensor'), ForwardRef('tf.Tensor'), ForwardRef('jnp.ndarray')]
do_rescale: typing.Optional[bool] = None
input_data_format: typing.Union[transformers.image_utils.ChannelDimension, str, NoneType] = None
)
→
PIL.Image.Image
Parameters
-
image (
PIL.Image.Image
ornumpy.ndarray
ortorch.Tensor
ortf.Tensor
) — The image to convert to thePIL.Image
format. -
do_rescale (
bool
, optional) — Whether or not to apply the scaling factor (to make pixel values integers between 0 and 255). Will default toTrue
if the image type is a floating type and casting toint
would result in a loss of precision, andFalse
otherwise. -
input_data_format (
ChannelDimension
, optional) — The channel dimension format of the input image. If unset, will use the inferred format from the input.
Returns
PIL.Image.Image
The converted image.
Converts image
to a PIL Image. Optionally rescales it and puts the channel dimension back as the last axis if
needed.
ImageProcessingMixin
This is an image processor mixin used to provide saving/loading functionality for sequential and image feature extractors.
Convert a single or a list of urls into the corresponding PIL.Image
objects.
If a single url is passed, the return value will be a single object. If a list is passed a list of objects is returned.
from_dict
< source >( image_processor_dict: typing.Dict[str, typing.Any] **kwargs ) → ImageProcessingMixin
Parameters
-
image_processor_dict (
Dict[str, Any]
) — Dictionary that will be used to instantiate the image processor object. Such a dictionary can be retrieved from a pretrained checkpoint by leveraging the to_dict() method. -
kwargs (
Dict[str, Any]
) — Additional parameters from which to initialize the image processor object.
Returns
The image processor object instantiated from those parameters.
Instantiates a type of ImageProcessingMixin from a Python dictionary of parameters.
from_json_file
< source >( json_file: typing.Union[str, os.PathLike] ) → A image processor of type ImageProcessingMixin
Parameters
Returns
A image processor of type ImageProcessingMixin
The image_processor object instantiated from that JSON file.
Instantiates a image processor of type ImageProcessingMixin from the path to a JSON file of parameters.
from_pretrained
< source >( pretrained_model_name_or_path: typing.Union[str, os.PathLike] cache_dir: typing.Union[str, os.PathLike, NoneType] = None force_download: bool = False local_files_only: bool = False token: typing.Union[str, bool, NoneType] = None revision: str = 'main' **kwargs )
Parameters
-
pretrained_model_name_or_path (
str
oros.PathLike
) — This can be either:- a string, the model id of a pretrained image_processor hosted inside a model repo on
huggingface.co. Valid model ids can be located at the root-level, like
bert-base-uncased
, or namespaced under a user or organization name, likedbmdz/bert-base-german-cased
. - a path to a directory containing a image processor file saved using the
save_pretrained() method, e.g.,
./my_model_directory/
. - a path or url to a saved image processor JSON file, e.g.,
./my_model_directory/preprocessor_config.json
.
- a string, the model id of a pretrained image_processor hosted inside a model repo on
huggingface.co. Valid model ids can be located at the root-level, like
-
cache_dir (
str
oros.PathLike
, optional) — Path to a directory in which a downloaded pretrained model image processor should be cached if the standard cache should not be used. -
force_download (
bool
, optional, defaults toFalse
) — Whether or not to force to (re-)download the image processor files and override the cached versions if they exist. -
resume_download (
bool
, optional, defaults toFalse
) — Whether or not to delete incompletely received file. Attempts to resume the download if such a file exists. -
proxies (
Dict[str, str]
, optional) — A dictionary of proxy servers to use by protocol or endpoint, e.g.,{'http': 'foo.bar:3128', 'http://hostname': 'foo.bar:4012'}.
The proxies are used on each request. -
token (
str
orbool
, optional) — The token to use as HTTP bearer authorization for remote files. IfTrue
, or not specified, will use the token generated when runninghuggingface-cli login
(stored in~/.huggingface
). -
revision (
str
, optional, defaults to"main"
) — The specific model version to use. It can be a branch name, a tag name, or a commit id, since we use a git-based system for storing models and other artifacts on huggingface.co, sorevision
can be any identifier allowed by git.
Instantiate a type of ImageProcessingMixin from an image processor.
Examples:
# We can't instantiate directly the base class *ImageProcessingMixin* so let's show the examples on a
# derived class: *CLIPImageProcessor*
image_processor = CLIPImageProcessor.from_pretrained(
"openai/clip-vit-base-patch32"
) # Download image_processing_config from huggingface.co and cache.
image_processor = CLIPImageProcessor.from_pretrained(
"./test/saved_model/"
) # E.g. image processor (or model) was saved using *save_pretrained('./test/saved_model/')*
image_processor = CLIPImageProcessor.from_pretrained("./test/saved_model/preprocessor_config.json")
image_processor = CLIPImageProcessor.from_pretrained(
"openai/clip-vit-base-patch32", do_normalize=False, foo=False
)
assert image_processor.do_normalize is False
image_processor, unused_kwargs = CLIPImageProcessor.from_pretrained(
"openai/clip-vit-base-patch32", do_normalize=False, foo=False, return_unused_kwargs=True
)
assert image_processor.do_normalize is False
assert unused_kwargs == {"foo": False}
get_image_processor_dict
< source >(
pretrained_model_name_or_path: typing.Union[str, os.PathLike]
**kwargs
)
→
Tuple[Dict, Dict]
Parameters
-
pretrained_model_name_or_path (
str
oros.PathLike
) — The identifier of the pre-trained checkpoint from which we want the dictionary of parameters. -
subfolder (
str
, optional, defaults to""
) — In case the relevant files are located inside a subfolder of the model repo on huggingface.co, you can specify the folder name here.
Returns
Tuple[Dict, Dict]
The dictionary(ies) that will be used to instantiate the image processor object.
From a pretrained_model_name_or_path
, resolve to a dictionary of parameters, to be used for instantiating a
image processor of type ~image_processor_utils.ImageProcessingMixin
using from_dict
.
push_to_hub
< source >( repo_id: str use_temp_dir: typing.Optional[bool] = None commit_message: typing.Optional[str] = None private: typing.Optional[bool] = None token: typing.Union[bool, str, NoneType] = None max_shard_size: typing.Union[int, str, NoneType] = '10GB' create_pr: bool = False safe_serialization: bool = False revision: str = None **deprecated_kwargs )
Parameters
-
repo_id (
str
) — The name of the repository you want to push your image processor to. It should contain your organization name when pushing to a given organization. -
use_temp_dir (
bool
, optional) — Whether or not to use a temporary directory to store the files saved before they are pushed to the Hub. Will default toTrue
if there is no directory named likerepo_id
,False
otherwise. -
commit_message (
str
, optional) — Message to commit while pushing. Will default to"Upload image processor"
. -
private (
bool
, optional) — Whether or not the repository created should be private. -
token (
bool
orstr
, optional) — The token to use as HTTP bearer authorization for remote files. IfTrue
, will use the token generated when runninghuggingface-cli login
(stored in~/.huggingface
). Will default toTrue
ifrepo_url
is not specified. -
max_shard_size (
int
orstr
, optional, defaults to"10GB"
) — Only applicable for models. The maximum size for a checkpoint before being sharded. Checkpoints shard will then be each of size lower than this size. If expressed as a string, needs to be digits followed by a unit (like"5MB"
). -
create_pr (
bool
, optional, defaults toFalse
) — Whether or not to create a PR with the uploaded files or directly commit. -
safe_serialization (
bool
, optional, defaults toFalse
) — Whether or not to convert the model weights in safetensors format for safer serialization. -
revision (
str
, optional) — Branch to push the uploaded files to.
Upload the image processor file to the 🤗 Model Hub.
Examples:
from transformers import AutoImageProcessor
image processor = AutoImageProcessor.from_pretrained("bert-base-cased")
# Push the image processor to your namespace with the name "my-finetuned-bert".
image processor.push_to_hub("my-finetuned-bert")
# Push the image processor to an organization with the name "my-finetuned-bert".
image processor.push_to_hub("huggingface/my-finetuned-bert")
register_for_auto_class
< source >( auto_class = 'AutoImageProcessor' )
Register this class with a given auto class. This should only be used for custom image processors as the ones
in the library are already mapped with AutoImageProcessor
.
This API is experimental and may have some slight breaking changes in the next releases.
save_pretrained
< source >( save_directory: typing.Union[str, os.PathLike] push_to_hub: bool = False **kwargs )
Parameters
-
save_directory (
str
oros.PathLike
) — Directory where the image processor JSON file will be saved (will be created if it does not exist). -
push_to_hub (
bool
, optional, defaults toFalse
) — Whether or not to push your model to the Hugging Face model hub after saving it. You can specify the repository you want to push to withrepo_id
(will default to the name ofsave_directory
in your namespace). -
kwargs (
Dict[str, Any]
, optional) — Additional key word arguments passed along to the push_to_hub() method.
Save an image processor object to the directory save_directory
, so that it can be re-loaded using the
from_pretrained() class method.
to_dict
< source >(
)
→
Dict[str, Any]
Returns
Dict[str, Any]
Dictionary of all the attributes that make up this image processor instance.
Serializes this instance to a Python dictionary.
to_json_file
< source >( json_file_path: typing.Union[str, os.PathLike] )
Save this instance to a JSON file.
to_json_string
< source >(
)
→
str
Returns
str
String containing all the attributes that make up this feature_extractor instance in JSON format.
Serializes this instance to a JSON string.