ARTIFICIAL INTELLIGENCE FOR SEMI-AUTOMATED DYNAMIC COMPRESSION OF IMAGES

In non-limiting examples of the present disclosure, systems, methods and devices for determining image compression optimums are provided. An image may be processed with a machine learning model that has been trained to identify object types in digital images. A first object and a first object type of the first object may be identified in the image. A first compressed version of the image may be generated, wherein the first compressed version has a first storage size. The first object and the first object type of the first object may be identified in the first compressed version of the image. A second compressed version of the image may be generated based on the identification of the first object and the first object type in the first compressed version of the image. The second compressed version may have a smaller storage size than the first storage size.

BACKGROUND

Personal computing devices with cameras are now ubiquitous. As the cameras on those computing devices have become more sophisticated, the storage costs associated with storing the visual media generated by those devices has risen dramatically. While much of the visual media generated by computing devices can be stored in the cloud, it is expensive to do so. The vast majority of visual media generated by users does not need to be full quality and resolution to maintain its enjoyment and usefulness.

It is with respect to this general technical environment that aspects of the present technology disclosed herein have been contemplated. Furthermore, although a general environment has been discussed, it should be understood that the examples described herein should not be limited to the general environment identified in the background.

SUMMARY

This summary is provided to introduce a selection of concepts in a simplified form that are further described below in the Detailed Description section. This summary is not intended to identify key features or essential features of the claimed subject matter, nor is it intended to be used as an aid in determining the scope of the claimed subject matter. Additional aspects, features, and/or advantages of examples will be set forth in part in the description which follows and, in part, will be apparent from the description or may be learned by practice of the disclosure.

Non-limiting examples of the present disclosure describe systems, methods and devices for determining image compression optimums. One or more object detection and/or object classification machine learning models may be applied to a digital image. One or more objects, object types, and/or specific objects may be identified and classified in the digital image. One or more tags may be associated with the digital image. The one or more tags may comprise a description of the objects, object types, and/or specific objects that were identified and classified. The digital image may be compressed utilizing one or more compression engines. Exemplary compression engines that may be applied to a digital image may include a channel agnostic pixel compression engine, a channel agnostic bit compression engine, a color channel down-sampling engine, and a color channel bit compression engine. In some examples, the color channel down-sampling engine and/or the color channel bit compression engine may be selectively applied to the background of a digital image (e.g., not to objects identified in a digital image).

A compressed version of a digital image may be processed by the one or more machine learning models to determine whether the same objects, object types, and/or specific objects can be identified and classified in the compressed version of the digital image. If the same objects are identified and classified in the compressed version of the digital image, the digital image may be compressed further until one or more of the same objects, object types, and/or specific objects can no longer be matched to the original digital image. The original digital image and the compressed versions of the digital image form a compression pyramid. One or more digital images in the compression pyramid may be surfaced for selection by a user. A selected digital image may replace an original digital image in storage thereby significantly reducing the amount of storage space needed to store digital visual media.

DETAILED DESCRIPTION

Examples of the disclosure provide systems, methods, and devices for using artificial intelligence to identify visual media compression optimums. As described herein, visual media may comprise digital images and/or digital videos. Users may store visual media on one or more computing devices. In some instances, visual media may be stored in one or more cloud-based services. The computing devices and/or cloud-based storage services may be associated with user accounts from which visual media is received. The computing devices and/or cloud-based storage services may have limited storage space and/or the user accounts may only subscribe to a limited amount of storage space.

A visual media optimization service may perform operations associated with reducing the amount of storage space that is required to store visual media. The visual media optimization service may be executed all or in part as a cloud-based service. In other examples, the visual optimization service may be executed all or in part on one or more local computing devices. Although the visual media optimization service is primarily discussed herein as processing digital images, it should be understood that it may process digital videos utilizing the same or similar operations.

The visual media optimization service may receive an indication to process a digital image. The indication may include a new digital image being uploaded to a cloud-based storage service, a new digital image being generated, and/or a manual input from a user, for example. Once the indication to process a digital image is received, the visual media optimization service may process the digital image with one or more machine learning models. The one or more machine learning models may comprise one or more image processing neural networks and/or optical character recognition models. In some examples, the one or more machine learning models may have been trained to identity and classify objects and object types (e.g., persons, text, buildings, landmarks, animals, etc.) in digital images. In additional examples, the one or more machine learning models may have been trained to identify and classify specific objects within object types (e.g., specific persons, specific buildings, specific landmarks, specific letters, specific words, etc.).

Upon identifying one or more objects, object types, and/or specific objects in a digital image, the visual media optimization service may associate one or more tags with the digital image. The one or more tags may comprise metadata that comprises a description of the one or more objects, object types, and/or specific objects that were identified by the one or more machine learning models.

The visual media optimization service may generate one or more compressed versions of a digital image. In some examples, the visual media optimization service may generate the compressed versions of the digital image contemporaneously with one another. In other examples, the visual media optimization service may generate the compressed versions of the digital image sequentially. For example, the visual media optimization service may only generate a next compressed version of a digital image upon first determining that the object tags for a previously generated compressed version of the digital image match the object tags for the original digital image. That is, the visual media optimization service may apply one or more machine learning models to each compressed version of a digital image. The one or more machine learning models may comprise the same machine learning models that were applied to the original digital image.

The visual media optimization service may determine whether the same objects, object types, and/or specific objects can be identified and classified in compressed versions of a digital image. Once the machine learning models have processed a compressed version of a digital image, the visual media optimization service may apply one or more tags to the compressed digital image. The one or more tags may comprise metadata that comprises a description of the one or more objects, object types, and/or specific objects that were identified by the one or more machine learning models. The tags of a compressed digital image may then be compared with the tags for an original digital image. If the tags match (e.g., there are the same tags and corresponding object description for the compressed digital image as for the original digital image), the visual media optimization service may generate one or more additional compressed versions of the digital images that have a smaller storage size than the previously compressed digital image. This process may be repeated until the tags for a compressed digital image no longer match the tags from the original digital image. The original digital image and each of the compressed digital images comprise a digital image pyramid.

Once a determination is made that tags for a compressed digital image do not match the tags for an original digital image, the visual media optimization service may surface a plurality of the digital images from the digital image pyramid for user selection. For example, the visual media optimization service may surface the compressed version of the digital image that was one step above the compressed digital image for which the tags did not match the original digital image, one or more compressed digital images toward the middle of the compression pyramid, and in some examples, the original digital image and/or the compressed version of the digital image for which the tags did not match the original digital image. In some examples, the visual media optimization service may surface the compressed digital images with an indication of how compressed they are in relation to the original image. In some examples, the indication may comprise text and/or values describing the original size of the digital image and/or a compressed digital image. In other examples, the indication may comprise a ratio of the size of a compressed version of a digital image compared to the size of the original digital image. The size may be indicated in pixel number or storage size (e.g., megabytes, gigabytes). A user may then select which of the digital images to save in the place of the original digital image. In some examples, the visual media optimization service may surface additional compressed image versions if a determination is made that the user is not satisfied with any of the previously surfaced images. For example, an input may be received to display one or more additional image choices. In some examples, the input may comprise an indication to display higher quality images. In other examples, the input may comprise an indication to display lower quality images. Thus, based on the type of input from a user, a binary search and image selection mechanism may be utilized to identify a desired image quality for a user.

The visual media optimization service may save the compression point and/or the identities of the compression engines that were utilized in compressing a user-selected image, and in some examples, utilize that information to automatically compress images for that user account to the same or similar compression point. For example, if the visual media optimization service surfaces two compression choices for an image (e.g., a first compression choice with a higher number of pixels, and a second compression choice with a lower number of pixels), and a selection is made of the second compression choice, the visual media optimization service may surface images with fewer pixels (e.g., pixel counts similar to the second compression choice) when a next compression choice needs to be made. Similarly, the visual media optimization service may save the compression point and/or the identities of the compression engines that were utilized in compressing a user-selected image, and in some examples, utilize that information when surfacing images for selection in the future. For example, the visual media optimization service may surface compressed digital images that have compression points that are similar to the selected digital image and/or digital images that were compressed using the same or similar compression engines as the selected digital image.

The visual media optimization service may apply one or more compression engines to a digital image in generating a compression pyramid of images. Exemplary compression engines that may be applied to a digital image in generating a compression pyramid of images may include a channel agnostic pixel compression engine, a channel agnostic bit compression engine, a color channel down-sampling engine, and a color channel bit compression engine.

A channel agnostic pixel compression engine may perform operations associated with compressing an image by reducing the height and width of an image. In reducing the height and width of an image, a channel agnostic pixel compression engine may simply reduce the number of pixels included in an image, while maintaining the ratio of pixels per color channel. In examples, a minimum compression size utilizing the channel agnostic pixel compression image may be determined based on a display size of a device on which an image is surfaced. For example, if an image is surfaced on a smart phone with a screen size of X, the minimum pixel compression size will be less than for a tablet device with a screen size of X+Y.

A channel agnostic bit compression engine may perform operations associated with compressing an image by reducing the number of bits in an image. In reducing the number of bits in an image, a channel agnostic bit compression engine may reduce the number of bits by the same number in each color channel (e.g., 24 bits per color channel to 16 bits per color channel, 16 bits per color channel to 8 bits per color channel).

A color channel down-sampling engine may perform operations associated with compressing an image by down-sampling the red and blue (and alpha if present) color channels in a red, green, blue color space (RGB color space), and/or down-sampling the U and V color channels in a YUV color space. In some examples, a color channel down-sampling engine may only be applied to the background of a digital image. That is, the objects that are identified in a digital image by one or more machine learning models may be classified as foreground and the rest of a digital image may be classified as background. A color channel down-sampling engine may then be selectively applied to the background of the digital image.

A color channel bit compression engine may perform operations associated with compressing a digital image by reducing the number of bits in a digital image in the red channel and blue channel (and alpha channel if present) of the RGB color space, and U and V color channels of the YUV color space. According to some aspects, a determination as to the amount to reduce the bit number in one or more of the red channel, blue channel, U channel, and/or V channel may be determined based on analyzing a 3D histogram of a digital image. For example, a color channel that contains more information may be reduced less than a color channel that contains less information based on analysis of a 3D histogram. In some examples, a color channel bit compression engine may only be applied to the background of a digital image. That is, the objects that are identified in a digital image by one or more machine learning models may be classified as foreground and the rest of a digital image may be classified as background. A color channel bit compression engine may then be selectively applied to the background of the digital image.

The systems, methods, and devices described herein provide technical advantages for determining visual media compression optimums. Storage costs associated with storing visual media on local computing devices and in the cloud are greatly reduced via the mechanisms described herein. Digital images may be compressed to the point where the important features (e.g., objects identified via machine learning models) are still readily discernable, while some of the background information may not be as clear as in an original image. However, because users typically care the most about important features of images, and because the compression engines described herein still provide high quality and high-resolution results, the storage size of acceptable digital images can be significantly reduced.

FIG.1is a schematic diagram illustrating an example distributed computing environment100for determining image compression optimums. Computing environment100includes initial image sub-environment102, network and processing sub-environment120, optimization engines sub-environment130, and image selection sub-environment148.

Network and processing sub-environment120includes network122, server computing device124, media store126, and compression optimization engines128. Any and all of the computing devices described herein may communicate with one another via a network, such as network122. Server computing device124is illustrative of a computing device that may host a visual media optimization service. Although the visual media optimization service is primarily described herein as being cloud-based, the visual media optimization service may reside and/or perform one or more operations on local computing devices (e.g., computing device104A/104B). The visual media optimization service may be associated with media store126. Media store126may store digital visual media (e.g., images, videos). For ease of explanation, the current disclosure primarily includes discussion of digital images, although it should be understood that operations discussed herein may be applied to digital images and digital videos. Visual media store126may store visual media associated with a plurality of user accounts (e.g., a user account associated with computing device104A/104B, user accounts associated with smart phones, user accounts associated with digital cameras, etc.). In some examples, users may manually push visual media from one or more computing devices to media store126for storage. In other examples, one or more devices and/or user accounts may automatically sync visual media to media store126for storage. Although media store126is illustrated as being cloud-based, media store126may comprise local storage on a local computing device, such as computing device104A/104B.

Each user account that media store126hosts may have same or different storage requirements and/or storage allowances. For example, a user may subscribe to a visual media storage service that includes media store126, and the subscription may include access to a specified amount of storage space (e.g., 5 gigabytes, 20 gigabytes, etc.). The visual media optimization service, which may be included in the visual media storage service, or which may be separate from the visual media storage service, may perform operations associated with compressing visual media files included in media store126such that users may optimize the amount of visual media they can store in association with a storage account, while at the same time storing the visual media in an acceptable resolution and/or quality. That is, the visual media optimization service assists users in striking a balance between image/video quality, and the amount of data (e.g., number, size, length) of images/video that are stored in association with user storage accounts.

The visual media optimization service includes compression optimization engines128, which are illustrated individually in optimization engines sub-environment130. Optimization engines sub-environment130includes object identification engine132, machine learning model selection engine134, object tagging engine136, tag comparison engine138, channel agnostic pixel compression engine140, channel agnostic bit compression engine142, color channel down-sampling engine144, color channel bit compression engine146, and YUV conversion engine147. Compression optimization engines128may be comprised in a single machine learning model, or a plurality of machine learning models.

The visual media optimization engine may receive an indication to compress an image (or video) in media store126. The indication to compress the image may comprise receiving the image (e.g., the image being initially received by media store126), a manual input (e.g., an input from a visual media application that specifies that the image should be compressed), or an automated input (e.g., settings for a user account may dictate that new images uploaded to a user account are compressed at various times or intervals). Once the indication to compress an image is received, the visual media optimization engine may utilize one or more of compression optimization engines128in compressing the image.

Object identification engine132may comprise one or more machine learning models that have been trained to identify and classify objects into type categories. For example, object identification engine132may receive a digital image, perform one or more preprocessing operations on the digital image, and process the digital image with a neural network. An output layer of the neural network may identify portions of the digital image that include objects and classify those objects into category types. The category types may include: person type, building type, landscape type, automobile type, animal type, etc.

Machine learning model selection engine134may perform operations associated with selecting and/or applying a machine learning model to objects in a digital image for further classification. For example, if object identification engine132identifies a person type object in a digital image, machine learning model selection engine134may select a person identification neural network for processing at least the portion of the digital image corresponding to the person object to further determine a specific identity of the person. In another example, if object identification engine132identifies a building type object in a digital image, machine learning model selection engine134may select a building identification neural network for processing at least the portion of the digital image corresponding to the building object to further determine a specific identity of the building (e.g., Eiffel Tower, Empire State Building).

Upon identifying one or more objects, object types, and/or specific objects in a digital image via application of object identification engine132and/or machine learning model selection engine134, object tagging engine136may associate one or more metadata tags that include that information with the digital image. For example, if object identification engine identifies a person type object in a digital image, a building type object in the digital image, and a text type object in the digital image; a first machine learning model selected by machine learning model selection engine134determines that the person type object corresponds to Jane Doe, a second machine learning model selected by machine learning model selection engine134determines that the building type object corresponds to the Eiffel Tower; and a third machine learning model selected by machine learning model selection engine134determines that the text type object corresponds to “Paris Cafe”, object tagging engine136may associate that information with the digital image as a plurality of metadata tags (e.g., a first tag for the person object type and specific person, a second tag for the building object type and specific building, a third tag for the text object type and specific text).

According to examples, object identification engine132, machine learning selection engine134, and object tagging engine136may be applied to each version of an image in a compression pyramid. For example, each of those engines may be applied to an original, full-size image, and one or more compressed versions of that original image. For example, if the image discussed above is compressed a first time to a reduced storage size, object identification engine132, machine learning model selection engine134, and object tagging engine136may be applied to that compressed version of the digital image. The tags that are associated with the compressed image via object tagging engine136may then be compared against the tags from the original image. Because the quality/size of the compressed image is less than the original image, one or more machine learning models may not be able to identify one of the objects or object types to a threshold value of accuracy (e.g., 90% accuracy, 95% accuracy) that were identified in the original digital image.

According to examples, the visual media optimization engine may compress an image a plurality of times, creating a pyramid of compressed images of different storage sizes. The visual media optimization engine may select a compressed image that has the smallest storage size in which all of the original tags were still identified/applied as an optimal digital image for saving. For example, if an original digital image has three tags that were identified and applied to it (e.g., person, building, text), a first compressed digital image with a smaller storage size than the original image has the same three tags that were identified and applied to it (e.g., person, building, text), and a third compressed digital image with a smaller storage size than the first compressed digital image has only two tags that were identified and applied to it (e.g., person, building), the visual media optimization engine may select the second digital image as the optimum digital image for saving. Tag comparison engine138may perform operations associated with comparing the tags from each of the images in a compression pyramid, and determining which images include the same tags and which images are missing tags from the original image.

In some examples, in generating a pyramid of compressed images, each progressively smaller image (e.g., based on storage space of an image) may be generated after determining that each of the original objects and tags were identified in a previously compressed image. Thus, to conserve processing resources associated with compressing many different versions of an image and processing the different compressed versions with machine learning models, the compression operations and subsequent machine learning processing of those compressed versions may be performed sequentially until an image is identified that does not include all of the original tags. In other examples, where processing costs are less of a concern, a plurality of compressed versions of a digital image may be generated contemporaneously and subsequently processed with object identification engine132, machine learning model selection engine134, object tagging engine136, and/or tag comparison engine138.

In some examples, a version of a digital image that is identified by the visual media optimization engine as being the optimum digital image for saving may be automatically saved as a replacement image for the original image. In other examples, once a version of a digital image has been identified by the visual media optimization engine as being the optimum digital image for saving, that version of the digital image may be surfaced with one or more other versions of that digital image such that a user may manually select which image to save. For example, where the digital images are being compressed sequentially, the visual media optimization engine may stop generating compressed versions of an image once the optimum digital image has been identified, and present the optimum digital image with the original image and/or one or more previously compressed versions of the digital image (e.g., images with larger storage sizes than the optimum digital image) for selection by a user. Once an image has been selected by a user as the image to save to media store126, that image may replace the original image, or in the case that the user selects the original image, the original image may be maintained in media store126.

In creating a pyramid of compressed images, one or more compression engines may be utilized. For example, one or more of channel agnostic pixel compression engine140, channel agnostic bit compression engine142, color channel down-sampling engine144, and color channel bit compression engine146may be utilized in compressing one or more digital images for tagging and tag comparison to identify an optimum digital image for saving.

Channel agnostic pixel compression engine140may perform operations associated with compressing an image by reducing the height and width of an image. In reducing the height and width of an image, channel agnostic pixel compression engine140may simply reduce the number of pixels included in an image, while maintaining the ratio of pixels per color channel.

Channel agnostic bit compression engine142may perform operations associated with compressing an image by reducing the number of bits in an image. In reducing the number of bits in an image, channel agnostic bit compression engine142may reduce the number of bits by the same number in each color channel (e.g., 24 bits per color channel to 16 bits per color channel).

Color channel down-sampling engine144may perform operations associated with compressing an image by down-sampling the red and blue (and alpha if present) color channels in a red, green, blue color space (RGB color space), and/or down-sampling the U and V color channels in a YUV color space.

In the RGB color space, the green channel may maintain its sampling (e.g., maintain green at 1), while the red channel may be down-sampled by a factor of X (e.g., 2, 3, 4), and the blue channel may be down-sampled by a factor of Z (e.g., 2, 3, 4). In some examples, X and Z may be the same factor. In other examples X and Z may be different factors. The sampling of the green color channel may be maintained because it typically includes the maximum information as compared to the red and blue color channels.

In a specific example, if the down-sampling ratio for R:G:B is 2:1:3 (e.g., green channel is left where it is, red channel is down-sampled by a factor of 2 ×2, and blue channel is down-sampled by a factor of 3×3), a group of 108 pixels (R 1-36, G 1-36, B 1-36) in the original RGB color space may be converted to a group of 49 pixels (R′ 1-9, G′ 1-36, and B′ 1-4) in the modified R′G′B′ space, which is approximately a 2.2× reduction in storage space.

Alternatively, if the down-sampling ratio of R:G:B was selected as 3:1:5 based on the formulae for conversion of RGB to gray-scale (e.g., green channel is left where it is, red channel is down-sampled by a factor of 3×3, and blue channel is down-sampled by a factor of 5×5), a group of 675 pixels (R 1-225, G 1-225, B 1-225) in the original RGB color space may be converted to a group of 259 pixels (R′ 1-25, G′ 1-225, B′ 1-9), which is approximately a 2.6× reduction in storage space.

In examples, YUV conversion engine147may be applied to the RGB color space, and the RGB color space may be converted to a YUV color space (e.g., YUV411, YUV420). In such examples, the U color channel (blue projection) may be down-sampled by a factor of X (e.g., 2, 3, 4), and the V color channel (red projection) may be down-sampled by a factor of X (e.g., 2, 3, 4). That is, the U and V color channels (the chrominance components) are down-sampled by the same factor, but that factor need not only be 2. The sampling of the Y component (luma component) may be maintained because it typically includes the maximum information as compared to the U and V color channels.

In a specific example, if the down-sampling ratio for Y:U:V is 1:3:3, then a group of 11 pixels (Y 1-9, U, V) is the representation of 27 pixels in the RGB color space (R 1-9, G 1-9, B 1-9), which is approximately a 2.45× reduction in storage space.

In another example, if the down-sampling ratio for Y:U:V is 1:4:4, then a group of 18 pixels (Y 1-16, U, V) is the representation of 48 pixels in the RGB color space (R 1-16, G 1-16, B 1-16), which is approximately a 2.67× reduction in storage space.

Color channel bit compression engine146may perform operations associated with compressing a digital image by reducing the number of bits in a digital image in the red channel and the blue channel (and alpha channel if present) of the RGB color space, and U and V color channels of the YUV color space.

In compressing a digital image utilizing color channel bit compression engine146, different color channels may have their bits reduced by different factors. For example, in the RGB color space, the red color channel bits may be reduced by a factor of X bits (2, 3, 4, 5, 6), and the blue channel bits may be reduced by a factor of Z bits (2, 3, 4, 5, 6). In some examples, X and Z may be the same factor. In other examples, X and Z may be different factors. The bits in the green channel may be maintained because that channel typically includes the maximum information as compared to the red and blue color channels.

In a specific example of a 36-bit original image in the RBG color space (e.g., 12 bits for each color channel—R 12 bits, G 12 bits, B 12 bits), if the bit representation in the red color channel is reduced by a factor of 2, the bit representation in the blue color channel is reduced by a factor of 3, and the bit representation in the green color channel is kept where it is, the new bit representation by color channel is (R 6 bits, G 12 bits, B 4 bits). Thus, there is approximately a 1.64× reduction in storage space (36-bit to-22 bit).

In examples, YUV conversion engine147may be applied to the RGB color space, and the RGB color space may be converted to a YUV color space (e.g., YUV411, YUV420). In such examples, the U color channel bits may be reduced by a factor X (e.g., 2, 3, 4) and the V color channel bits may be reduced by a factor of X (e.g., 2, 3, 4). That is, the bits in the U and V color channels may be reduced by the same factor. The bits in the Y component may be maintained because it typically includes the maximum information as compared to the U and V color channels.

In a specific example of a 48-bit original image in the YUV space (e.g., 16 bits for each channel—Y 16 bits, U 16 bits, V 16 bits), if the bit representation in the U and V color channels are reduced by a factor of 2, and the bit representation in the Y color channel is kept where it is, the new bit representation by color channel is (Y 16 bits, U 8 bits, V 8 bits). Thus, there is approximately a 1.5× reduction in storage space (48-bit to 32-bit).

In compressing a digital image, color channel down-sampling engine144may be applied to the entire image or a portion of the digital image. In some examples, color channel down-sampling engine144may be applied selectively to the background of a digital image. For example, each of the objects that are identified in a digital image with object identification engine132may be tagged or otherwise distinguished from the remainder of the digital image. In some examples, the objects in a digital image identified with object identification engine132may be marked as foreground and the remainder of the digital image may be marked as background. Color channel down-sampling engine144may thus be applied to the background of the image, while not being applied to the foreground. Thus, the quality/resolution of the important objects in a digital image may not necessarily be reduced via application of color channel down-sampling engine144.

In compressing a digital image, color channel bit compression engine146may be applied to the entire image or a portion of the digital image. In some examples, color channel bit compression engine146may be applied selectively to the background of a digital image. For example, each of the objects that are identified in a digital image with object identification engine132may be tagged or otherwise distinguished from the remainder of the digital image. In some examples, the objects in a digital image identified with object identification engine132may be marked as foreground and the remainder of the digital image may be marked as background. Color channel bit compression engine146may thus be applied to the background of the image, while not being applied to the foreground. Thus, the quality/resolution of the important objects in a digital image may not necessarily be reduced via application of color channel bit compression engine146.

In the current example, a visual media application is displayed on computing device104A. In some examples, the media storage application may be synced to media store126. The media storage application includes a plurality of image thumbnails (thumbnail108A, thumbnail110A, thumbnail112A, thumbnail114A) on a right display side106A of computing device104A. A full original image116of thumbnail108A is displayed on a left display side of computing device104A.

In this example, an indication has been received by the visual media optimization engine to compress original image116. That is, window118, which states “Reduce Image Size?” with selectable options for “Yes” and “No” has been surfaced, and the “Yes” option has been selected. As such, the visual media optimization service may apply object identification engine132, machine learning model selection engine134, and object tagging engine136to original image116and identify and tag one or more objects and object types in original image116. For example, the three persons in original image116may be tagged as person objects and/or specific persons. Similarly, the mountains in original image116may be tagged as mountain objects and/or specific mountains. Similarly, the sky in original image116may be tagged as a sky object.

The visual media optimization engine may apply one or more of the compression engines (e.g., channel agnostic pixel compression engine140, channel agnostic bit compression engine142, color channel down-sampling engine144, color channel bit compression engine146) to original image116, and generate a first compressed version of original image116that has a first reduced storage size that is less than the storage size of original image116. Object identification engine132, machine learning model selection engine134, and object tagging engine136may then be applied to the first compressed version of original image116. Tag comparison engine138may then compare the tags that have been associated with the first compressed version of original image116with the tags of original image116. If each of the tags are the same, original image116may be compressed even further. If one or more tags are missing on first compressed version of original image116compared to original image116, the visual media optimization engine may determine that original image116should not be compressed further. This process may be repeated until a compressed version of original image is generated for which the tags do not match the tags from original image116. That is, when the machine learning models can no longer identify each of the objects and object types in an image to within a threshold value of accuracy (e.g., 90%), the tags will no longer match and there is a high likelihood that a user will not select that image as a good enough replacement for an original image. When this happens, one or more of the compressed images, and in some cases the original image, may then be surfaced, and a user may manually select which image to save to media store126. The one or more compressed images that are surfaced may or may not include the last image that was compressed (e.g., the image that not all of the objects and/or object types could be identified in).

Computing device104B in image selection sub-environment148may be the same or a different computing device as computing device104A. Computing device104B includes the four image thumbnails (thumbnail108B, thumbnail110B, thumbnail112B, thumbnail114B) on the right display side106B of the visual media application. The left side150of the visual media application displays four versions of original image116. Each of the versions of original image116displayed on the left side150of the visual media application may differ in storage size, pixel number, height and/or width, color space (e.g., RGB vs YUV), bit number, color channel bit number, sampling, and/or color channel sampling based on the one or more compression engines that were utilized to generate each compressed version of the digital image. A user may select which of the four versions to save in media store126. In this example, this selection may be made via window152, which includes selectable elements for saving one of the four versions to media store126. In other examples, a user may interact with one of the four displayed images. Other selection mechanisms may be utilized.

In some examples, the visual media optimization service may save information related to which image was selected and utilize that information in determining which compression engines to apply in the future for that user. For example, if a user selected an image that was reduced to storage size X, there may be a higher likelihood that the user will select images of approximately that size in the future. As such, more images of approximately that size may be surfaced for the user account to select from in the future. Similarly, if the user selected a version of the digital image that was compressed with color channel bit compression engine146applied only to the background, there may be a higher likelihood that the user will select images that have been compressed utilizing that engine applied to the background in the future. Thus, the visual media optimization engine may tailor its compression techniques to each specific user account that is associated with media store126.

FIG.2is a simplified block diagram of a computing environment200illustrating the compression of an image utilizing a color channel agnostic pixel compression engine and a color channel agnostic bit compression engine in the RGB color space. Computing environment200includes original image sub-environment202, compressed image sub-environment226, object recognition neural network250, same objects detected element252, further compression element254, and revert to previous compression element256.

Original image sub-environment202includes original digital image204. Bit number by channel element206indicates that original digital image204is a 48-bit image. The bit number of original image204is exemplary and it should be understood that the color channel agnostic bit compression engine may be utilized on images of any bit size. Red channel bit element210indicates that the red color channel has 16 bits. Green channel bit element212indicates that the blue color channel has 16 bits. Blue channel bit element214indicates that the blue channel has 16 bits.

Pixel ratio by channel element216indicates that original digital image204is 1080 pixels wide and 608 pixels high. The pixel number and ratio of original digital image204is exemplary and it should be understood that the color channel agnostic pixel compression engine may be utilized on images of any pixel number and ratio. Red channel pixel ratio element220, green channel pixel ratio element222, and blue channel pixel ratio element224indicate that original digital image204has a color channel pixel ratio of R: 2.5, G: 5, B: 1.

In this example, one or more machine learning models (e.g., image neural networks) have been applied to original digital image204. Those one or more machine learning models may have been trained to identify objects and object types in images. The one or more machine learning models have identified objects207and may have classified objects207as mountains. In some examples, the one or more machine learning models may have classified objects207as specific mountains (e.g., Grand Teton, Middle Teton, South Teton). The one or more machine learning models have identified objects208,209and210and may have classified those objects as persons. In some examples, the one or more machine learning models may have classified the persons corresponding to objects208,209and210as specific persons (e.g., John Doe, Jane Doe). The visual media optimization service may tag original digital image204with a description of the objects, object types, and/or specific objects that were identified by the one or more machine learning models.

Moving from original image sub-environment202to compressed image sub-environment226, the color channel agnostic pixel compression engine and the color channel agnostic bit compression engine have been applied to original digital image204and resulted in compressed digital image228. Compressed bit number by channel element230indicates that compressed digital image228has been compressed from 48-bit to 24-bit, and the bit value of each color channel has been reduced by half. That is, the color channel agnostic bit compression engine has equally reduced the bit value in each of the red, green, and blue channels. Specifically, compressed red channel bit element234indicates that the red color channel of compressed digital image228has 8 bits (reduced from 16), compressed green channel bit element236indicates that the green color channel of compressed digital image228has 8 bits (reduced from 16), and compressed blue channel bit element238indicates that the blue color channel of compressed digital image228also has 8 bits (reduced from 16).

Compressed pixel number element232indicates that the pixel number has been reduced from 1080W×608H in original digital image204to 540W×304H in compressed digital image228. That is, the color channel agnostic pixel compression engine has reduced the pixel count by half (equally in the height and the width of the digital image). Additionally, compressed pixel ratio by channel element240indicates that in reducing the pixel count in compressed digital image228, the ratio of pixels in each color channel has remained the same as in original digital image204. Specifically, compressed red channel pixel ratio element244, compressed green channel pixel ratio246, and compressed blue channel ratio element248indicate that compressed digital image228has a color channel pixel ratio of R: 2.5, G: 5, B: 1.

One or more machine learning models are applied to compressed digital image228. The one or more machine learning models may have been trained to identify objects and object types in images. In this example, the one or more machine learning models are illustrated as object recognition neural network250. Once object recognition neural network250has processed compressed image228, the visual media optimization service may tag compressed image228with a description of the objects, object types, and/or specific objects that were identified by object recognition neural network250. A comparison may then be made between the object tags of original digital image204and compressed digital image228. This is illustrated by same objects detected element252. If a determination is made that the same objects were detected/identified, the digital image may be compressed further (e.g., the bit number and/or pixel number may be reduced further) and the processing with the neural network and comparison of tags may be repeated. This is indicated by further compression element254. Alternatively, if a determination is made that the same objects were not detected/identified (e.g., if tags are missing for any of objects207,208,209,210), the visual media optimization service may determine that a previous compression of the digital image is the optimal digital image, and/or that compressed digital image228is as far as visual media optimization service is going to compress original digital image204. This is indicated by revert to previous compression element256. The visual media optimization service may then cause one or more versions of the digital image to be surfaced for selection by a user. A selected version of the digital image may then be saved to a visual media store.

FIG.3is a simplified block diagram of a computing environment300illustrating the compression of an image utilizing a color channel down-sampling engine in the RGB color space. Computing environment300includes original image sub-environment302, compressed image sub-environment326, down-sampling element325, object recognition neural network350, same objects detected element352, further compression element354, and revert to previous compression element356.

Original image sub-environment302includes original digital image304. Pixels per channel element316includes red channel pixel number element320, green channel pixel number element322, and blue channel pixel number element324. Red channel pixel number element320indicates that there 218,880 pixels in the red color channel of original digital image304. Green channel pixel number element322indicates that there are 218,880 pixels in the green color channel of original digital image304. Blue channel pixel number element324indicates that there are 218,880 pixels in the blue color channel of original digital image304. It should be understood that the pixel numbers in each of the color channels are provided for exemplary purposes and the color channel down-sampling engine may be applied to color channels that include any number of pixels.

In this example, one or more machine learning models (e.g., image neural networks) have been applied to original digital image304. Those one or more machine learning models may have been trained to identify objects and object types in images. The one or more machine learning models have identified objects307and may have classified objects307as mountains. In some examples, the one or more machine learning models may have classified objects307as specific mountains. The one or more machine learning models have identified objects308,309and310and may have classified those objects as persons. In some examples, the one or more machine learning models may have classified the persons corresponding to objects308,309and310as specific persons. The visual media optimization service may tag original digital image304with a description of the objects, object types, and/or specific objects that were identified by the one or more machine learning models.

Down-sampling element325illustrates the down-sampling ratio that is applied to the color channels of original digital image304in compressing it with the color channel down-sampling engine. Specifically, the down-sampling ratio that is applied is a down-sampling of the red color channel by a factor of 2 (e.g., 2×2), leaving the green color channel where it is (e.g., factor of 1), and down-sampling the red color channel by a factor of 3 (e.g., 3×3).

Although the color channel down-sampling engine is illustrated inFIG.3as being applied to the red, green, blue color space, the color channel down-sampling engine may be applied to the YUV color space, although original digital image304would first have to be converted to the YUV color space. Specifically, the U and V color channels may be down-sampled equally by the same factor (e.g., factor of 2, factor of 3, factor of 4), while leaving the Y color channel where it is.

The result of the downs-sampling is indicated in compressed image sub-environment326. Compressed image sub-environment326includes compressed digital image328and compressed pixels per channel element340. Compressed pixels per channel element340includes compressed red channel pixel number element344, compressed green channel pixel number element346, and compressed blue channel pixel number element348. Compressed red channel pixel number element344indicates that the result of down-sampling the red color channel by a factor of two has resulted in a reduction in pixels in that channel from 218,880 pixels to 54,720 pixels (e.g., 2×2 reduction). Compressed green channel pixel number element346indicates that there was no down-sampling applied to that color channel, resulting in no reduction of pixels in that channel (e.g., 218,880 pixels). Compressed blue channel pixel number element348indicates that the result of down-sampling the blue color channel by a factor of three has resulted in a reduction in pixels in that channel from 218,880 pixels to 24,320 pixels (e.g., 3×3 reduction).

One or more machine learning models are applied to compressed image328. The one or more machine learning models may have been trained to identify objects and object types in images. In this example, the one or more machine learning models are illustrated as object recognition neural network350. Once object recognition neural network350has processed compressed image328, the visual media optimization service may tag compressed image328with a description of the objects, object types, and/or specific objects that were identified by object recognition neural network350. A comparison may then be made between the object tags of original digital image304and compressed digital image328. This is illustrated by same objects detected element352. If a determination is made that the same objects were detected/identified, the digital image may be compressed further (e.g., the red and blue color channels may be down-sampled by additional factors, one or more other compression techniques may be applied to original digital image304to generate a compressed image of smaller storage size than compressed digital image328) and the processing with the neural network and comparison of tags may be repeated. This is indicated by further compression element354. Alternatively, if a determination is made that the same objects were not detected/identified (e.g., if tags are missing for any of objects307,308,309,310), the visual media optimization service may determine that a previous compression of the digital image is the optimal digital image, and/or that compressed digital image328is as far as visual media optimization service is going to compress original digital image304. The visual media optimization service may then cause one or more versions of the digital image to be surfaced for selection by a user. A selected version of the digital image may then be saved to a visual media store.

FIG.4is a simplified block diagram of a computing environment400illustrating the compression of an image utilizing a color channel down-sampling engine in the YUV color space. Computing environment400includes original image sub-environment402, compressed image sub-environment426, down-sampling element425, object recognition neural network450, same objects detected element452, further compression element454, and revert to previous compression element456.

Original image sub-environment402includes original digital image504. Pixels per channel element416includes Y channel pixel number element420, U channel pixel number element422, and V channel pixel number element424. Y channel pixel number element420indicates that there are 9 pixels in the Y component/channel of original digital image404. U channel pixel number element422indicates that there are 9 pixels in the U color channel of original digital image404. V channel pixel number element424indicates that there are 9 pixels in the V color channel of original digital image404. It should be understood that the pixel numbers in each of the channels (Y channel/component, U channel, V channel) are provided for exemplary purposes and the color channel down-sampling engine may be applied to channels that include any number of pixels.

In this example, one or more machine learning models (e.g., image neural networks) have been applied to original digital image504. Those one or more machine learning models may have been trained to identify objects and object types in images. The one or more machine learning models have identified object407and may have classified object407as a building object. The one or more machine learning models may have further classified object407as a specific building type object (e.g., Eiffel Tower). The one or more machine learning models have identified object408and may have classified object408as a person type object. The one or more machine learning models may have further classified object408as a specific person (e.g., John Doe). The one or more machine learning models may have identified object409and classified object408as a cloud type object. The visual media optimization service may tag original digital image404with a description of the objects, object types, and/or specific objects that were identified by the one or more machine learning models.

Down-sampling element425illustrates the down-sampling ratio that is applied to the color channels of original digital image404, after it has been converted to the YUV color space, in compressing it with the color channel down-sampling engine. Specifically, the down-sampling ratio that is applied is Y: 1, U:3 (e.g., 3×3), V: 3 (e.g., 3×3). Thus, the Y channel/component is left as it is and the U channel and the V channel are down-sampled equally by a factor of 3 (e.g., 3×3).

The result of the down-sampling is indicated in compressed image sub-environment426. Compressed image sub-environment426includes compressed digital image428and compressed pixels per channel element440. Compressed pixels per channel element440includes compressed Y channel pixel number element444, compressed U channel pixel number element446, and compressed V channel pixel number element448. Compressed U channel pixel number element446indicates that the result of down-sampling the U color channel by a factor of three has resulted in a reduction in pixels in that channel from 9 pixels to 1 pixel (e.g., 3×3 reduction). Compressed V channel pixel number element448indicates that the result of down-sampling the V color channel by a factor of three has resulted in a reduction in pixels in that channel from 9 pixels to 1 pixel (e.g., 3×3 reduction). Compressed Y channel pixel number element444indicates that there was no down-sampling applied to that channel/component, resulting in no reduction of pixels in that channel (e.g., 9 pixels).

One or more machine learning models are applied to compressed image428. The one or more machine learning models may have been trained to identify objects and object types in images. In this example, the one or more machine learning models are illustrated as object recognition neural network450. Once object recognition neural network450has processed compressed image428, the visual media optimization service may tag compressed image428with a description of the objects, object types, and/or specific objects that were identified by object recognition neural network450. A comparison may then be made between the object tags of original digital image404and compressed digital image428. This is illustrated by same objects detected element452. If a determination is made that the same objects were detected/identified, the digital image may be compressed further (e.g., U and V channels may be down-sampled by additional factors, one or more other compression techniques may be applied to original digital image404and/or compressed digital image428to generate a compressed image of smaller storage size than compressed digital image428) and the processing with the neural network and comparison of tags may be repeated. This is indicated by further compression element454. Alternatively, if a determination is made that the same objects were not detected/identified (e.g., if tags are missing for any of objects407,408, or409), the visual media optimization service may determine that a previous compression of the digital image is the optimal digital image, and/or that compressed digital image428is as far as visual media optimization service is going to compress original digital image404. The visual media optimization service may then cause one or more versions of the digital image to be surfaced for selection by a user. A selected version of the digital image may then be saved to a visual media store.

FIG.5is a simplified block diagram of a computing environment500illustrating the compression of an image utilizing a color channel bit compression engine in the RGB color space. Computing environment500includes original image sub-environment502, compressed image sub-environment526, object recognition neural network550, same objects detected element552, further compression element554, and revert to previous compression element556.

Original image sub-environment502includes original digital image504. Original digital image504is a 36-bit image as indicated by bit element508. The bit number of original image504is exemplary and it should be understood that the color channel bit compression engine may be utilized on images of any bit number. Bit number by channel element506includes red channel bit element510, green channel bit element512, and blue channel bit element514. Red channel bit element510indicates that the red color channel of original digital image504has 12 bits. Green channel bit element512indicates that the green channel of original digital image504has 12 bits. Blue channel bit element514indicates that the blue channel of original digital image504has 12 bits.

In this example, one or more machine learning models (e.g., image neural networks) have been applied to original digital image504. Those one or more machine learning models may have been trained to identify objects and object types in images. The one or more machine learning models have identified object515and may have classified object515as a building type object. The one or more machine learning models may have classified object515as a specific object (e.g., Eiffel Tower). The one or more machine learning models have also identified object516and may have classified object516as a person type object. The one or more machine learning models may have classified object516as a specific person (e.g., John Doe). The one or more machine learning models have also identified object517and may have classified object517as a cloud type object. The visual media optimization service may tag original digital image504with a description of the objects, object types, and/or specific objects that were identified by the one or more machine learning models.

Bit compression element518indicates the factor that the color channel bit compression engine compresses the bit size of the red color channel by in compressing original digital image504to compressed digital image528. That is, the red color channel is reduced from 12 bits to 6 bits (a factor of 2). Bit compression element520indicates the factor that the color channel bit compression engine compresses the bit size of the blue color channel by in compressing original digital image504to compressed digital image528. That is, the blue color channel is reduced from 12 bits to 4 bits (a factor of 3). The bit compression factors illustrated and described in relation toFIG.5are exemplary and it should be understood that the color channel bit compression engine may compress the red and blue color channels by different factors. Additionally, the compression factors for the red channel and the blue channel need not necessarily be different from one another. As described herein, the color channel bit compression engine keeps the green channel bit value of the original image because that channel includes the most information.

Although the color channel bit compression engine is illustrated inFIG.5as being applied to the red, green, blue color space, the color channel bit compression engine may be applied to the YUV color space, although original digital image504would first have to be converted to the YUV color space. Specifically, the bit numbers in the U and V color channels may be reduced by a same factor (e.g., factor of 2, factor of 3, factor of 4), while leaving the Y color channel where it is.

Moving from original image sub-environment502to compressed image sub-environment526, the color channel bit compression engine has been applied to original digital image504and resulted in compressed digital image528. Compressed digital image508is a 22-bit image as indicated by bit element532. Compressed bit number by channel element530includes compressed red channel bit element534, compressed green channel bit element536, and compressed blue channel bit element538. Compressed red channel element534indicates that the red color channel of compressed digital image528has 6 bits (compressed down by a factor of 2 from 12 bits). Compressed green channel element536indicates that the green color channel of compressed digital image528still has 12 bits (the same as the green channel in original digital image504). Compressed blue channel element538indicates that the blue color channel of compressed digital image528has 4 bits (compressed down by a factor of 3 from 12 bits).

One or more machine learning models are applied to compressed digital image528. The one or more machine learning models may have been trained to identify objects and object types in images. In this example, the one or more machine learning models are illustrated as object recognition neural network550. Once object recognition neural network550has processed compressed image528, the visual media optimization service may tag compressed image528with a description of the objects, object types, and/or specific objects that were identified by object recognition neural network550. A comparison may then be made between the object tags of original digital image504and compressed digital image528. This is illustrated by same objects detected element552. If a determination is made that the same objects were detected/identified, the digital image may be compressed further (e.g., the bit numbers in the blue and/or red channel may be reduced further, one or more other compression models may be applied to digital image504and/or digital image528) and the processing with the neural network and comparison of tags may be repeated. This is indicated by further compression element554. Alternatively, if a determination is made that the same objects were not detected/identified (e.g., if tags are missing for any of objects515,516,517), the visual media optimization service may determine that a previous compression of the digital image is the optimal digital image for saving, and/or that compressed digital image528is as far as the visual media optimization service is going to compress original digital image504. This is indicated by revert to previous compression element556. The visual media optimization service may then cause one or more versions of the digital image to be surfaced for selection by a user. A selected version of the digital image may be saved to a visual media store.

FIG.6is an exemplary method600for determining image compression optimums. The method600begins at a start operation and flow moves to operation602.

At operation602a digital image is processed with a machine learning model that has been trained to identify object types in digital images. The machine learning model may comprise one or more image processing neural networks. In additional examples, the machine learning model may comprise an optical character recognition model.

From operation602flow continues to operation604where a first object and a first object type are identified in the digital image based on the processing of the digital image with the machine learning model. The identification may be based on the machine learning model determining to a threshold value of accuracy (e.g., 90% accuracy, 95% accuracy) that the first object is present, and to a threshold value of accuracy (e.g., 90% accuracy, 95% accuracy) that the first object is of the first object type. Examples of the first object type may include person object type, face object type, text object type animal object type, building object type, mountain object type, etc.

From operation604flow continues to operation606where a first compressed version of the digital image having a first storage size is generated. The first compressed version of the digital image may be generated via application of one or more compression models to the digital image. Examples of compression models that may be applied include a channel agnostic pixel compression engine, a channel agnostic bit compression engine, a color channel down-sampling engine, and a color channel bit compression engine.

From operation606flow continues to operation608where the first object and the first object type of the first object are identified in the first compressed version of the digital image based on processing the first compressed version of the digital image with the machine learning model. The identification may be based on the machine learning model determining to a threshold value of accuracy (e.g., 90% accuracy, 95% accuracy) that the first object is present, and to a threshold value of accuracy (e.g., 90% accuracy, 95% accuracy) that the first object is of the first object type.

From operation608flow continues to operation610where a second compressed version of the digital image with a second storage size that is less than the first storage size is generated based on the identification of the first object and the first object type in the first compressed version of the digital image with the machine learning model. The second compressed version of the digital image may be generated via application of one or more compression models to the digital image. Examples of compression models that may be applied include a channel agnostic pixel compression engine, a channel agnostic bit compression engine, a color channel down-sampling engine, and a color channel bit compression engine.

From operation610flow moves to an end operation and the method600ends.

FIG.7is another exemplary method700for determining image compression optimums. The method700begins at a start operation and flow moves to operation702.

At operation702a first version of a digital image is processed with a first machine learning model that has been trained to classify objects of a first object type, and a second machine learning model that has been trained to classify objects of a second object type. The first machine learning model may comprise an image neural network and/or an optical character recognition model. The second machine learning model may comprise an image neural network and/or an optical character recognition model. In examples, the first version of the digital image may comprise an original (e.g., uncompressed image). In other examples, the first version of the digital image may comprise a digital image that has been compressed from an original image.

From operation702flow continues operation704where a first object is classified as the first object type in the first version of the digital image based on the processing, and a second object is classified as the second object type in the first version of the digital image based on the processing. Examples of the object types include person object type, face object type, text object type, animal object type, building object type, mountain object type, etc.

From operation704flow continues to operation706where a first tag comprising a description of the first object type is applied to the first version of the digital image. The first tag may comprise metadata that is associated with the digital image.

From operation706flow continues to operation708where a second tag comprising a description of the second object type is applied to the first version of the digital image. The second tag may comprise metadata that is associated with the digital image.

From operation708flow continues to operation710where the first version of the digital image is compressed with a first compression model. Examples of compression models that may be applied include a channel agnostic pixel compression engine, a channel agnostic bit compression engine, a color channel down-sampling engine, and a color channel bit compression engine.

From operation710flow continues to operation712where the compressed digital image is processed with the first machine learning model.

From operation712flow continues to operation714where a determination is made based on the processing of the compressed image that the first object cannot be classified as the first object type to within a threshold value of accuracy. The threshold value of accuracy may be a percentage (e.g., 90%, 95%).

From operation714flow continues to operation716where a determination is made based on the processing of the compressed image that the second object can be classified as the first object type to within a threshold value of accuracy. The threshold value of accuracy may be a percentage (e.g., 90%, 95%).

From operation716flow continues to operation718where the first version of the digital image is selected as an optimum compression of the digital image.

From operation718flow moves to an end operation and the method700ends.

FIGS.8and9illustrate a mobile computing device800, for example, a mobile telephone, a smart phone, wearable computer (such as smart eyeglasses), a tablet computer, an e-reader, a laptop computer, or other AR compatible computing device, with which embodiments of the disclosure may be practiced. With reference toFIG.8, one aspect of a mobile computing device800for implementing the aspects is illustrated. In a basic configuration, the mobile computing device800is a handheld computer having both input elements and output elements. The mobile computing device800typically includes a display805and one or more input buttons810that allow the user to enter information into the mobile computing device800. The display805of the mobile computing device800may also function as an input device (e.g., a touch screen display). If included, an optional side input element815allows further user input. The side input element815may be a rotary switch, a button, or any other type of manual input element. In alternative aspects, mobile computing device800may incorporate more or fewer input elements. For example, the display805may not be a touch screen in some embodiments. In yet another alternative embodiment, the mobile computing device800is a portable phone system, such as a cellular phone. The mobile computing device800may also include an optional keypad835. Optional keypad835may be a physical keypad or a “soft” keypad generated on the touch screen display. In various embodiments, the output elements include the display805for showing a graphical user interface (GUI), a visual indicator820(e.g., a light emitting diode), and/or an audio transducer825(e.g., a speaker). In some aspects, the mobile computing device800incorporates a vibration transducer for providing the user with tactile feedback. In yet another aspect, the mobile computing device800incorporates input and/or output ports, such as an audio input (e.g., a microphone jack), an audio output (e.g., a headphone jack), and a video output (e.g., a HDMI port) for sending signals to or receiving signals from an external device.

FIG.9is a block diagram illustrating the architecture of one aspect of a mobile computing device. That is, the mobile computing device900can incorporate a system (e.g., an architecture)902to implement some aspects. In one embodiment, the system902is implemented as a “smart phone” capable of running one or more applications (e.g., browser, e-mail, calendaring, contact managers, messaging clients, games, and media clients/players). In some aspects, the system902is integrated as a computing device, such as an integrated personal digital assistant (PDA) and wireless phone.

One or more application programs966may be loaded into the memory962and run on or in association with the operating system964. Examples of the application programs include phone dialer programs, e-mail programs, personal information management (PIM) programs, word processing programs, spreadsheet programs, Internet browser programs, messaging programs, and so forth. The system902also includes a non-volatile storage area968within the memory962. The non-volatile storage area968may be used to store persistent information that should not be lost if the system902is powered down. The application programs966may use and store information in the non-volatile storage area968, such as e-mail or other messages used by an e-mail application, and the like. A synchronization application (not shown) also resides on the system902and is programmed to interact with a corresponding synchronization application resident on a host computer to keep the information stored in the non-volatile storage area968synchronized with corresponding information stored at the host computer. As should be appreciated, other applications may be loaded into the memory962and run on the mobile computing device900, including instructions for providing and operating visual media optimization applications.

The system902has a power supply970, which may be implemented as one or more batteries. The power supply970might further include an external power source, such as an AC adapter or a powered docking cradle that supplements or recharges the batteries.

The system902may also include a radio interface layer972that performs the function of transmitting and receiving radio frequency communications. The radio interface layer972facilitates wireless connectivity between the system902and the “outside world,” via a communications carrier or service provider. Transmissions to and from the radio interface layer972are conducted under control of the operating system964. In other words, communications received by the radio interface layer972may be disseminated to the application programs966via the operating system964, and vice versa.

The visual indicator820may be used to provide visual notifications, and/or an audio interface974may be used for producing audible notifications via the audio transducer825. In the illustrated embodiment, the visual indicator820is a light emitting diode (LED) and the audio transducer825is a speaker. These devices may be directly coupled to the power supply970so that when activated, they remain on for a duration dictated by the notification mechanism even though the processor960and other components might shut down for conserving battery power. The LED may be programmed to remain on indefinitely until the user takes action to indicate the powered-on status of the device. The audio interface974is used to provide audible signals to and receive audible signals from the user. For example, in addition to being coupled to the audio transducer825, the audio interface974may also be coupled to a microphone to receive audible input, such as to facilitate a telephone conversation. In accordance with embodiments of the present disclosure, the microphone may also serve as an audio sensor to facilitate control of notifications, as will be described below. The system902may further include a video interface976that enables an operation of an on-board camera830to record still images, video stream, and the like.

A mobile computing device900implementing the system902may have additional features or functionality. For example, the mobile computing device900may also include additional data storage devices (removable and/or non-removable) such as, magnetic disks, optical disks, or tape. Such additional storage is illustrated inFIG.9by the non-volatile storage area968.

Data/information generated or captured by the mobile computing device900and stored via the system902may be stored locally on the mobile computing device900, as described above, or the data may be stored on any number of storage media that may be accessed by the device via the radio interface layer972or via a wired connection between the mobile computing device900and a separate computing device associated with the mobile computing device900, for example, a server computer in a distributed computing network, such as the Internet. As should be appreciated such data/information may be accessed via the mobile computing device900via the radio interface layer972or via a distributed computing network. Similarly, such data/information may be readily transferred between computing devices for storage and use according to well-known data/information transfer and storage means, including electronic mail and collaborative data/information sharing systems.

FIG.10is a block diagram illustrating physical components (e.g., hardware) of a computing device1000with which aspects of the disclosure may be practiced. The computing device components described below may have computer executable instructions for determining image compression optimums. In a basic configuration, the computing device1000may include at least one processing unit1002and a system memory1004. Depending on the configuration and type of computing device, the system memory1004may comprise, but is not limited to, volatile storage (e.g., random access memory), non-volatile storage (e.g., read-only memory), flash memory, or any combination of such memories. The system memory1004may include an operating system1005suitable for running one or more visual media and/or visual media storage applications. The operating system1005, for example, may be suitable for controlling the operation of the computing device1000. Furthermore, embodiments of the disclosure may be practiced in conjunction with a graphics library, other operating systems, or any other application program and is not limited to any particular application or system. This basic configuration is illustrated inFIG.10by those components within a dashed line1008. The computing device1000may have additional features or functionality. For example, the computing device1000may also include additional data storage devices (removable and/or non-removable) such as, for example, magnetic disks, optical disks, or tape. Such additional storage is illustrated inFIG.10by a removable storage device1009and a non-removable storage device1010.

As stated above, a number of program modules and data files may be stored in the system memory1004. While executing on the processing unit1002, the program modules1006(e.g., visual media optimization service1020) may perform processes including, but not limited to, the aspects, as described herein. Object classification engine1011may perform one or more operations associated with processing a digital image and identifying and/or classifying one or more objects, object types, and/or specific objects in the digital image. Tag comparison engine1013may perform one or more operations associated with comparing tags of a compressed version of a digital image with tags from a non-compressed version of a digital image. Channel agnostic compression engine1015may perform one or more operations associated with compressing a digital image by reducing the height and width of a digital image and/or by down-sampling each color channel of a digital image by the same factor. Color channel compression engine1017may perform one or more operations associated with down-sampling the red and blue channels in a RGB color space by different factors and leaving the green channel where it was and/or reducing the U and V channels in a YUV color space by the same factor and leaving the Y component where it was. Color channel compression engine1017may additionally or alternatively perform one or more operations associated with reducing the pixel ratio of the red and blue channels by different factors in a RBG color space while leaving the green channel the same and/or reducing the pixel ratio of the U and V channels by the same factor in a YUV color space while leaving the Y component the same.

The computing device1000may also have one or more input device(s)1012such as a keyboard, a mouse, a pen, a sound or voice input device, a touch or swipe input device, etc. The output device(s)1014such as a display, speakers, a printer, etc. may also be included. The aforementioned devices are examples and others may be used. The computing device1000may include one or more communication connections1016allowing communications with other computing devices1050. Examples of suitable communication connections1016include, but are not limited to, radio frequency (RF) transmitter, receiver, and/or transceiver circuitry; universal serial bus (USB), parallel, and/or serial ports.

The term computer readable media as used herein may include computer storage media. Computer storage media may include volatile and nonvolatile, removable and non-removable media implemented in any method or technology for storage of information, such as computer readable instructions, data structures, or program modules. The system memory1004, the removable storage device1009, and the non-removable storage device1010are all computer storage media examples (e.g., memory storage). Computer storage media may include RAM, ROM, electrically erasable read-only memory (EEPROM), flash memory or other memory technology, CD-ROM, digital versatile disks (DVD) or other optical storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other article of manufacture which can be used to store information and which can be accessed by the computing device1000. Any such computer storage media may be part of the computing device1000. Computer readable media and computer storage media as described herein does not include transitory media such as a carrier wave or other propagated or modulated data signal. Computer readable storage device means hardware and does not include transitory media such as a carrier wave or other propagated or modulated data signal.

FIG.11illustrates one aspect of the architecture of a system for processing data received at a computing system from a remote source, such as a personal/general computer1104, tablet computing device1106, or mobile computing device1108, as described above. Content displayed at server device1102may be stored in different communication channels or other storage types. For example, various documents may be stored using a directory service1122, a web portal1124, a mailbox service1126, an instant messaging store1128, or a social networking site1130. The program modules1006may be employed by a client that communicates with server device1102, and/or the program modules1006may be employed by server device1102. The server device1102may provide data to and from a client computing device such as a personal/general computer1104, a tablet computing device1106and/or a mobile computing device1108(e.g., a smart phone) through a network1115. By way of example, the computer system described above may be embodied in a personal/general computer1104, a tablet computing device1106and/or a mobile computing device1108(e.g., a smart phone). Any of these embodiments of the computing devices may obtain content from the store1116, in addition to receiving graphical data useable to be either pre-processed at a graphic-originating system, or post-processed at a receiving computing system.

The description and illustration of one or more aspects provided in this application are not intended to limit or restrict the scope of the disclosure as claimed in any way. The aspects, examples, and details provided in this application are considered sufficient to convey possession and enable others to make and use the best mode of claimed disclosure. The claimed disclosure should not be construed as being limited to any aspect, example, or detail provided in this application. Regardless of whether shown and described in combination or separately, the various features (both structural and methodological) are intended to be selectively included or omitted to produce an embodiment with a particular set of features. Having been provided with the description and illustration of the present disclosure, one skilled in the art may envision variations, modifications, and alternate aspects falling within the spirit of the broader aspects of the general inventive concept embodied in this application that do not depart from the broader scope of the claimed disclosure. The various embodiments described above are provided by way of illustration only and should not be construed to limit the claims attached hereto. Those skilled in the art will readily recognize various modifications and changes that may be made without following the example embodiments and applications illustrated and described herein, and without departing from the true spirit and scope of the following claims.