SYSTEM AND METHOD FOR CORRECTING DISTORTED IMAGES

In an example, an image may be identified. Object detection may be performed on the image to identify a region including a distorted representation of an object. The region may be masked to generate a masked image including a masked region corresponding to the object. Using a machine learning model, the masked region may be replaced with an undistorted representation of the object to generate a modified image.

BACKGROUND

Many platforms exist that allow users to share content including images, videos, etc. However, some images and/or videos may be visually distorted. For example, straight lines in an image appear to be curved or deformed. Such visual distortions may be associated with a lower image quality, and may provide for a negative user experience of users viewing the distorted images and/or videos.

DETAILED DESCRIPTION OF EXAMPLE EMBODIMENTS

Subject matter will now be described more fully hereinafter with reference to the accompanying drawings, which form a part hereof, and which show, by way of illustration, specific example embodiments. This description is not intended as an extensive or detailed discussion of known concepts. Details that are well known may have been omitted, or may be handled in summary fashion.

The following provides a discussion of some types of scenarios in which the disclosed subject matter may be utilized and/or implemented.

One or more systems and/or techniques for correcting distortions in video streams and/or images are provided. In accordance with one or more of the techniques provided herein, a video stream correction system is provided, which may correct at least some distortions in a first video stream to generate a second video stream (corresponding to a corrected version of the first video stream, for example). In some examples, the first video stream may be captured with a camera that has a wide or ultrawide field of view, which may introduce one or more distortions (e.g., at least one of fish-eye lens distortion, barrel distortion, etc.) to the first video stream. Using the techniques provided herein, the one or more distortions may be corrected using a machine learning model (e.g., a trained machine learning model) that may predict pixel values in a contextually aware and/or dynamic manner. As compared to some systems that correct distortions merely using a (static) rule-based approach and/or using a known object's dimensions as a reference point (e.g., scaling sections of a video frame using a scaling factor calculated based upon a known length of an object and a distorted length of the object), using the machine learning model according to the techniques provided herein may result in dynamically correcting distortions of the first video stream with increased accuracy.

In accordance with some embodiments of the present disclosure, the second video stream may be shown to one or more participants a video call (e.g., a conference call, a virtual reality communication session, etc.). In an example, the second video stream may comprise a desktop view of a desk (to provide the one or more participants with an additional perspective, for example). The techniques of the present disclosure may be used to replace distorted representations of objects on the desktop with undistorted (e.g., corrected) representations of the objects (e.g., the undistorted representations of the objects may be included in the second video stream). The systems that correct distortions using the rule-based approach may not be able to correct at least some of the distorted representations of the objects. For example, the rule-based approach may not be used to accurately correct a distorted representation of an object with a height that exceeds a threshold height, whereas using the techniques provided herein, the distorted representation of the object (with the height exceeding the threshold height) may be accurately corrected (e.g., the distorted representation may be replaced with an undistorted representation of the object) using the machine learning model in accordance with the techniques provided herein.

FIGS.1A-1Eillustrate examples of a system101for correcting distortions in a video stream.FIG.1Aillustrates a first client device106(e.g., a laptop, a smartphone, a tablet, a wearable device, etc.) associated with a user108providing a first video stream102. The first video stream102may be provided to the system101(e.g., the first video stream102may be transmitted to a server of the system101). In some examples, the first video stream102may comprise a real-time representation of a view of a first camera104associated with the first client device106. In an example, the first camera104may be a camera that is mounted on and/or embedded in the first client device106. Alternatively and/or additionally, the first camera104may be a standalone camera (e.g., the first camera104may be a security camera and/or a different type of camera, such as a webcam and/or an external camera, that may or may not be mounted on the first client device106). The first camera104may be connected to the first client device106via a wired or wireless connection.

In some examples, the first video stream102may be sent by the first client device106in association with a video call established between the first client device106and a second client device (not shown). In an example, a communication system may establish the video call in response to receiving a request to establish the video call from the first client device106and/or the second client device. In an example, the first client device106may correspond to a caller and/or originator of the video call and the second client device may correspond to a receiver and/or destination of the video call, or vice versa. When the video call is established, a video corresponding to the first video stream102(captured using the first camera104) may be displayed on the second client device (such that a user of the second client device is able to see the video while conversing with the user108over the video call, for example). Other users and/or client devices (in addition to the first client device106and/or the second client device, for example) may be participants of the video call. In some examples, the video call may be implemented in a virtual reality environment (e.g., a metaverse).

In some examples, an angle114of a field of view associated with the first camera104and/or the first video stream102may be at least a threshold angle associate with a wide or ultrawide field of view. In an example, the first camera104may correspond to a wide or ultrawide camera associated with the wide or ultrawide field of view (e.g., the first camera104may capture a wider field of view relative to some cameras). The threshold angle may be 90 degrees, 100 degrees, 110 degrees, 120 degrees, or other value. In an example in which the first client device106and/or the first camera104is on and/or over a desktop of a desk110, the field of view (e.g., the wide or ultrawide field of view) associated with the first camera104may encompass (i) at least a portion of the desktop of the desk110and/or (ii) a face of the user108of the first client device106. Accordingly, the first video stream102captured by the first camera104may show (i) at least a portion of the desktop of the desk110and/or (ii) the face of the user108. However, the first video stream102may comprise one or more distortions (e.g., fish-eye lens distortion, barrel distortion, and/or one or more other distortions that may be found in images captured using the wide or ultrawide field of view). Using the techniques provided herein, a video stream correction system (e.g., the system101) may correct at least some distortions in the first video stream102, and/or may provide a corrected version of the first video stream102for display on the second client device.

An embodiment of correcting distortions in a video stream is illustrated by an exemplary method200ofFIG.2, and is further described in conjunction with the system101ofFIGS.1A-1E. At202, the video stream correction system may receive the first video stream102. At204, the video stream correction system may analyze a first video frame of the first video stream102to identify a region of interest of the first video frame. In an example, the region of interest may correspond to a region targeted by the video stream correction system for distortion correction (e.g., the video stream correction system may focus on detecting and/or correcting distorted objects in the region of interest).

FIG.1Billustrates an exemplary method for determination of the region of interest (shown with reference number134). In an example, the region of interest134may be determined using a region of interest determination model122. The first video frame (shown with reference number120) may be input to the region of interest determination model122, which may analyze the first video frame120to determine boundaries of the region of interest134. The region of interest determination model122may comprise a convolutional context-aware model. The region of interest determination model122may comprise a plurality of layers. In an example, each layer of one, some and/or all layers of the plurality of layers may be a dense layer, such as a fully connected dense layer. In some examples, an activation function of the region of interest determination model122may comprise a rectified linear unit. In some examples, a loss function of the region of interest determination model122may comprise Mean Absolute Error (MAE) and/or Mean Squared Error (MSE). In an example, the plurality of layers comprises a first layer124(e.g., an input layer), a second layer126(e.g., a convolutional neural network (CNN) layer connected to the input layer), a third layer128(e.g., an attention layer connected to the CNN layer), a fourth layer130(e.g., a dense layer connected to the attention layer), and/or a fifth layer132(e.g., an output layer connected to the dense layer), wherein the region of interest134may be output by the fifth layer132.FIG.1Cillustrates an example of the first video frame120and the region of interest134of the first video frame120. The region of interest134may comprise a representation of the desktop of the desk110.

Returning back to the flow diagram ofFIG.2, at206the video stream correction system may perform object detection on the region of interest134of the first video frame120to identify a first region (in the region of interest134, for example) comprising a distorted representation of a first object. In an example shown inFIG.1D, an object detection module118may perform object detection on the region of interest134to identify objects comprising at least one of a desk plant142, a phone144, a keyboard146, a mouse148, eye-glasses150, a mug152, etc. At least some of the objects may be distorted in the first video frame120(e.g., the keyboard146is shown having an abnormal shape with angled sides different from an actual rectangular shape of the keyboard146). In an example, the first object may correspond to the keyboard146, and the first region may correspond to a region, of the first video frame120, that is occupied by the keyboard146. In some examples, the first region (occupied by the keyboard146, for example) may be determined by performing object segmentation (e.g., instance segmentation) on the first object (e.g., the keyboard146).

At208ofFIG.2, the video stream correction system may mask the first region to generate a first masked image comprising a masked region corresponding to the first object. For example, masking the first region may comprise setting some or all pixels of the first region to a predefined color (e.g., a predefined pixel value), such as at least one of white, black, etc. Accordingly, in some examples, the first masked image may comprise the masked region, having the predefined color, in place of the first object.

At210ofFIG.2, using a first machine learning model, the video stream correction system may replace the masked region with an undistorted representation of the first object to generate a modified image (e.g., a corrected image). For example, the modified image may be generated, using the first machine learning model, to comprise the undistorted representation of the first object in place of the masked region. In an example, the video stream correction system may regenerate, using the first machine learning model, pixels of the masked region of the first masked image to generate the modified image comprising the undistorted representation of the first object. In some examples, the modified image is generated (using the first machine learning model, for example) based upon the distorted representation of the first object.

FIG.1Eillustrates the first machine learning model (shown with reference number164) being used to generate the modified image (shown with reference number166). For example, the distorted representation (shown with reference number160) of the first object and the first masked image (shown with reference number162) comprising the masked region (shown with reference number170) may be input to the first machine learning model164, which may use the distorted representation160of the first object and the first masked image162to generate the modified image166. For example, the first machine learning model164may be used to regenerate pixels of the masked region170of the first masked image162to generate the undistorted representation (shown with reference number168) of the first object (e.g., the keyboard146). In the example shown inFIG.1E, the distorted representation160of the first object corresponds to a region, of the first video frame120, comprising the keyboard146. In some examples, the first machine learning model164may be trained to generate the undistorted representation168of the first object (e.g., the keyboard146) in a contextually aware manner.

In some examples, the first machine learning model164is trained to predict, in a contextually aware manner, pixel values (e.g., colors) for pixels of a masked region of a masked image, wherein a first loss function of the first machine learning model164may be based upon (i) an original image (e.g., an image that is partially masked to generate the masked image) and/or (ii) a reference image (e.g., a reference image of an object, in the original image, that is masked to generate the masked image).

FIGS.3A-3Billustrate examples of a system301for training the first machine learning model164. InFIG.3A, a region of an original image302may be masked to generate a second masked image304with a second masked region306. The second masked region306may correspond to a second object (e.g., an apple) in the original image302(e.g., the second object in the original image302may be masked to generate the second masked image304). For example, some or all pixels of the second object in the original image302may be set to a predefined color (e.g., a predefined pixel value), such as at least one of white, black, etc., to generate the second masked image304. Alternatively and/or additionally, pixels of the original image302may be masked randomly and/or according to one or more predefined rules. In an example, a predefined proportion of the original image302(e.g., the predefined proportion may be between 15% of the original image302and 25% of the original image302) may be masked to generate the second masked image304.

In some examples, the first machine learning model164may replace the second masked region306of the second masked image304with a representation312of the second object to generate an output image310. For example, the first machine learning model164may be used to regenerate pixels of the second masked region306of the second masked image304to generate the representation312of the second object (e.g., the apple). In some examples, the output image310may be generated (using the first machine learning model164) based upon an object reference image308associated with the second object (e.g., the object reference image308may correspond to a reference image of the second object). For example, the output image310may be input to the first machine learning model164.

In some examples, using the original image302and/or the object reference image308, the first machine learning model164learns first contextual awareness information associated with a context of a pixel given its surroundings. For example, for each pixel of one, some and/or all pixels of the original image302and/or the object reference image308, the first contextual awareness information may comprise a relationship of the pixel to the pixel's surroundings (e.g., the pixel's surroundings may correspond to pixels neighboring the pixel, such as pixels within a threshold distance of the pixel). In an example, the first contextual awareness information may comprise a relationship (e.g., a contextual relationship) between a color of the pixel and one or more colors of the pixel's surroundings, wherein the relationship may be determined by the first machine learning model164based upon a pixel value (e.g., a magnitude of the pixel value) of the pixel and/or one or more pixel values (e.g., magnitudes of the one or more pixel values) of the pixel's surroundings. Alternatively and/or additionally, the first contextual awareness information may comprise shapes and/or edges of objects and/or areas, which the first machine learning model164may learn based upon a gradient of a pixel change between the pixel and the pixel's surroundings. The first machine learning model164may be used to predict, based upon the first contextual awareness information, pixel values (e.g., indicative of pixel colors) of pixels of the second masked region306of the second masked image304to generate the representation312of the second object (e.g., the apple). For example, the representation312may be generated according to the predicted pixel values.

In some examples, the first loss function of the first machine learning model164may be used to determine a first difference between (i) a pixel value of a pixel in the original image302and (ii) a pixel value of a corresponding pixel in the output image310(e.g., the pixel value may correspond to a predicted pixel value predicted using the first machine learning model164). In some examples, the first machine learning model164may be updated based upon the first difference and/or the first loss function. In an example, one or more weights and/or parameters of the first machine learning model164may be modified based upon the first difference. In an example, the first loss function may be used to determine one or more differences (e.g., a plurality of differences, each corresponding to a difference between an original pixel value of the original image302and a corresponding predicted pixel value of the output image310) comprising the first difference, and a first loss value may be determined based upon the one or more differences. The one or more weights and/or parameters of the first machine learning model164may be modified based upon the first loss value. In some examples, the first machine learning model164comprises a first neural network model. In some examples, the first contextual awareness information (learned by the first machine learning model164, for example), the first difference and/or the first loss value may be propagated (according to the first loss function, for example) across at least some of the first neural network model.

In some examples, the first machine learning model164is trained using a second machine learning model. In some examples, the second machine learning model is used to determine a context score associated with the output image310, and the first machine learning model164may be trained based upon the context score.

InFIG.3B, the second machine learning model (shown with reference number322) is used to generate a fault flag representation324based upon the output image310(output by the first machine learning model164). In an example, the second machine learning model322is used to classify each pixel of one, some and/or all pixels of the output image310as (i) an original pixel (e.g., a pixel from the original image302) or (ii) a generated pixel (e.g., a pixel generated using the first machine learning model164). For example, a pixel of the output image310may be classified as an original pixel based upon a determination (e.g., a determination by the second machine learning model322) that the pixel is contextually related to the pixel's surroundings (e.g., a determination that the pixel is contextually suitable given the pixel's surroundings), wherein the pixel's surroundings may correspond to pixels neighboring the pixel, such as pixels within a threshold distance of the pixel. Alternatively and/or additionally, a pixel of the output image310may be classified as a generated pixel based upon a determination (e.g., a determination by the second machine learning model322) that the pixel is not contextually related to the pixel's surroundings (e.g., a determination that the pixel is not contextually suitable given the pixel's surroundings).

In some examples, the fault flag representation324may be generated based upon a plurality of classifications (e.g., original pixel classifications and/or generated pixel classifications), of pixels in the output image310, determined using the second machine learning model322. For example, to generate the fault flag representation324, pixels of the output image310that are classified as original pixels may be set to a first color (e.g., black) and pixels of the output image310that are classified as generated pixels may be set to a second color (e.g., white). In some examples, the context score may be determined based upon the plurality of classifications (and/or the fault flag representation324) determined using the second machine learning model322. In an example, the context score may be a function of (i) a first quantity of pixels, of the output image310, that are classified as original pixels (e.g., the first quantity of pixels may equal a quantity of pixels, in the fault flag representation324, that are set to black) and/or (ii) a second quantity of pixels, of the output image310, that are classified as generated pixels (e.g., the second quantity of pixels may equal a quantity of pixels, in the fault flag representation324, that are set to white), wherein an increase of the first quantity of pixels and/or a decrease of the second quantity of pixels may correspond to an increase of the context score. In an example, one or more operations (e.g., mathematical operations) may be performed using the first quantity of pixels and/or the second quantity of pixels to determine the context score.

In some examples, using the output image310, the second machine learning model322learns second contextual awareness information associated with a context of a pixel given its surroundings. For example, for each pixel of one, some and/or all pixels of the output image310, the second contextual awareness information may comprise a relationship of the pixel to the pixel's surroundings (e.g., the pixel's surroundings may correspond to pixels neighboring the pixel, such as pixels within a threshold distance of the pixel). In an example, the second contextual awareness information may comprise a relationship (e.g., a contextual relationship) between a color of the pixel and colors of the pixel's surroundings. The relationship may be determined by the second machine learning model322based upon a pixel value (e.g., a magnitude of the pixel value) of the pixel and/or one or more pixel values (e.g., magnitudes of the one or more pixel values) of the pixel's surroundings. Alternatively and/or additionally, the second contextual awareness information may comprise shapes and/or edges of objects and/or areas, which the second machine learning model322may learn based upon a gradient of a pixel change between the pixel and the pixel's surroundings. The second machine learning model322may be used to determine the context score, the plurality of classifications and/or the fault flag representation324based upon the second contextual awareness information.

In some examples, a second loss function of the second machine learning model322may be used to determine a second loss value. The second loss value may be based upon a difference between (i) actual generated pixels of the output image310(e.g., the actual generated pixels may correspond to pixels, of the output image310, that were generated using the first machine learning model164and/or were not a part of the original image302) and (ii) pixels, of the output image310, that are classified as generated pixels by the second machine learning model322. In some examples, the second machine learning model322may be updated based upon the second loss value (and/or the second loss function). In an example, one or more weights and/or parameters of the second machine learning model322may be modified based upon the second loss value (and/or the second loss function). In some examples, the second machine learning model322comprises a second neural network model. In some examples, the second contextual awareness information (learned by the second machine learning model322, for example) and/or the second loss value may be propagated (according to the second loss function, for example) across at least some of the second neural network model. In some examples, the second loss function of the second machine learning model322may be connected to a generator mask of the second machine learning model322.

In some examples, the first machine learning model164may be trained based upon the context score (and/or based upon the plurality of classifications) determined using the second machine learning model322. In an example, one or more weights and/or parameters of the first machine learning model164may be modified based upon the context score (and/or based upon the plurality of classifications).

In some examples, acts shown in and/or discussed with respect toFIGS.3A-3Bmay relate to a (single) training iteration for training the first machine learning model164. In some examples, multiple training iterations may be performed to train the first machine learning model164. For example, the multiple training iterations may be performed using different images (e.g., including images different than the original image302) and/or different objects (e.g., including objects different than the second object). In some examples, each iteration of the multiple training iterations may be performed using one or more of the techniques shown in and/or discussed with respect toFIGS.3A-3B. In some examples, training iterations may be performed on the first machine learning model164periodically, and/or upon receiving (new) training information. Alternatively and/or additionally, training iterations may cease being performed on the first machine learning model164in response to determining that the first machine learning model164is sufficiently trained. In an example, the determination that the first machine learning model164is sufficiently trained may be based upon (i) a determination that a loss value (e.g., the first loss value) associated with an original image (e.g., the original image302) and/or an output image (e.g., the output image310) generated by the first machine learning model164in a training iteration does not meet (e.g., is smaller than) a threshold loss value, and/or (ii) a determination that a context score (determined by the second machine learning model322) of an output image (e.g., the output image310) generated by the first machine learning model164meets (e.g., exceeds) a threshold context score.

It may be appreciated that training the first machine learning model164using the techniques provided herein may result in the first machine learning model164learning to regenerate masked pixels (e.g., generate pixels of the undistorted representation168of the first object) in a contextually aware manner (e.g., the first machine learning model164learns to identify contextual awareness of a pixel to the pixel's surroundings), thereby enabling the first machine learning model164to generate undistorted representations of distorted objects (e.g., the undistorted representation168of the first object) with increased accuracy. In an example, using the first masked image162and/or the distorted representation160of the first object, the first machine learning model164may learn third contextual awareness information associated with a context of a pixel given its surroundings. For example, for each pixel of one, some and/or all pixels of the first masked image162and/or the distorted representation160of the first object, the third contextual awareness information may comprise a relationship of the pixel to the pixel's surroundings (e.g., the pixel's surroundings may correspond to pixels neighboring the pixel, such as pixels within a threshold distance of the pixel). In an example, the third contextual awareness information may comprise a relationship (e.g., a contextual relationship) between a color of the pixel and colors of the pixel's surroundings (e.g., the relationship may be determined based upon a pixel value of the pixel and/or one or more pixel values of the pixel's surroundings). Alternatively and/or additionally, the third contextual awareness information may comprise shapes and/or edges of objects and/or areas (which may be learned based upon a gradient of a pixel change between the pixel and the pixel's surroundings, for example). The first machine learning model164may be used to predict, based upon the third contextual awareness information, pixel values (e.g., indicative of pixel colors) of pixels of the masked region170of the first masked image162to generate the undistorted representation168of the first object (e.g., the keyboard146). For example, the undistorted representation168may be generated according to the predicted pixel values.

Referring back toFIG.1E, in some examples, the modified image166may be generated to include an undistorted representation of each object of one, some and/or all of the objects identified in the region of interest134. For example, the modified image166may comprise a plurality of (corrected) representations comprising an undistorted representation of the desk plant142, an undistorted representation of the phone144, an undistorted representation of the mouse148, an undistorted representation of the eye-glasses150, and/or an undistorted representation of the mug152. In some examples, each representation of the plurality of representations may be generated using one or more of the techniques provided herein with respect to generating the undistorted representation168of the first object (e.g., the keyboard146).

Returning back toFIG.2, at212, the video stream correction system may generate a second video stream comprising the modified image166. In an example, the modified image166may correspond to a video frame of the second video stream. In some examples, the second video stream may correspond to a corrected version of the first video stream102. For example, each video frame of one, some and/or all video frames of the first video stream102may be modified to generate a corrected video frame (using one or more of the techniques provided herein with respect to generating the modified image166, for example), and the corrected video frame may be included in the second video stream. For example, corrected video frames, such as the modified image166, may be compiled by the video stream correction system to generate the second video stream. The second video stream may be displayed on the second client device (during the video call, for example). In an example, the second video stream may be displayed on the second client device in response to establishing the video call. In some examples, the second video stream may comprise a desktop view (e.g., a view of the desktop of the desk110) with undistorted (e.g., corrected) representations of one or more objects on the desktop of the desk110(e.g., at least one of the desk plant142, the phone144, the keyboard146, the mouse148, the eye-glasses150, the mug152, etc.). In this way, the user of the second client device may be able to view the desktop of the desk110(while conversing with the user108over the video call, for example).

In accordance with some embodiments, a machine learning model provided herein (e.g., at least one of the region of interest determination model122, the first machine learning model164, the second machine learning model322, etc.) may comprise at least one of a tree-based model, a machine learning model used to perform linear regression, a machine learning model used to perform logistic regression, a decision tree model, a support vector machine (SVM), a Bayesian network model, a k-Nearest Neighbors (kNN) model, a K-Means model, a random forest model, a machine learning model used to perform dimensional reduction, a machine learning model used to perform gradient boosting, a neural network model (e.g., a deep neural network model and/or a convolutional neural network model), etc.

In accordance with some embodiments, at least some of the present disclosure may be performed and/or implemented automatically and/or in real time. For example, at least some of the present disclosure may be performed and/or implemented such that in response to receiving the first video frame120of the first video stream102, the modified image166(e.g., which may be a corrected version of the first video frame120) is output by the video stream correction system and/or is displayed on the second client device quickly (e.g., instantly) and/or in real time.

It may be appreciated that the techniques provided herein may be used for correcting distortions in an image (e.g., at least one of a single frame, a photograph, a graphical object, etc.). An embodiment of correcting distortions in an image is illustrated by an exemplary method400ofFIG.4. At402, an image may be identified. At404, object detection may be performed on the image to identify a region comprising a distorted representation of an object (e.g., a desk plant, a phone, a keyboard, a mouse, eye-glasses, a mug, or other object). At406, the region may be masked to generate a masked image (e.g., the first masked image162) comprising a masked region (e.g., the masked region170) corresponding to the object. At408, the masked region may be replaced, using a machine learning model (e.g., the first machine learning model164), with an undistorted representation of the object to generate a modified (e.g., corrected) image. The modified image may be displayed on a client device.

FIG.5is an interaction diagram of a scenario500illustrating a service502provided by a set of computers504to a set of client devices510(e.g., UEs) via various types of transmission mediums. The computers504and/or client devices510may be capable of transmitting, receiving, processing, and/or storing many types of signals, such as in memory as physical memory states.

The computers504of the service502may be communicatively coupled together, such as for exchange of communications using a transmission medium506. The transmission medium506may be organized according to one or more network architectures, such as computer/client, peer-to-peer, and/or mesh architectures, and/or a variety of roles, such as administrative computers, authentication computers, security monitor computers, data stores for objects such as files and databases, business logic computers, time synchronization computers, and/or front-end computers providing a user-facing interface for the service502.

Likewise, the transmission medium506may comprise one or more sub-networks, such as may employ different architectures, may be compliant or compatible with differing protocols and/or may interoperate within the transmission medium506. Additionally, various types of transmission medium506may be interconnected (e.g., a router may provide a link between otherwise separate and independent transmission medium506).

In scenario500ofFIG.5, the transmission medium506of the service502is connected to a transmission medium508that allows the service502to exchange data with other services502and/or client devices510. The transmission medium508may encompass various combinations of devices with varying levels of distribution and exposure, such as a public wide-area network and/or a private network (e.g., a virtual private network (VPN) of a distributed enterprise).

In the scenario500ofFIG.5, the service502may be accessed via the transmission medium508by a user512of one or more client devices510, such as a portable media player (e.g., an electronic text reader, an audio device, or a portable gaming, exercise, or navigation device); a portable communication device (e.g., a camera, a phone, a wearable or a text chatting device); a workstation; and/or a laptop form factor computer. The respective client devices510may communicate with the service502via various communicative couplings to the transmission medium508. As a first such example, one or more client devices510may comprise a cellular communicator and may communicate with the service502by connecting to the transmission medium508via a transmission medium507provided by a cellular provider. As a second such example, one or more client devices510may communicate with the service502by connecting to the transmission medium508via a transmission medium509provided by a location such as the user's home or workplace (e.g., a WiFi (Institute of Electrical and Electronics Engineers (IEEE) Standard 802.11) network or a Bluetooth (IEEE Standard 802.15.1) personal area network). In this manner, the computers504and the client devices510may communicate over various types of transmission mediums.

FIG.6presents a schematic architecture diagram600of a computer504that may utilize at least a portion of the techniques provided herein. Such a computer504may vary widely in configuration or capabilities, alone or in conjunction with other computers, in order to provide a service such as the service502.

The computer504may comprise one or more processors610that process instructions. The one or more processors610may optionally include a plurality of cores; one or more coprocessors, such as a mathematics coprocessor or an integrated graphical processing unit (GPU); and/or one or more layers of local cache memory. The computer504may comprise memory602storing various forms of applications, such as an operating system604; one or more computer applications606; and/or various forms of data, such as a database608or a file system. The computer504may comprise a variety of peripheral components, such as a wired and/or wireless network adapter614connectible to a local area network and/or wide area network; one or more storage components616, such as a hard disk drive, a solid-state storage device (SSD), a flash memory device, and/or a magnetic and/or optical disk reader.

The computer504may comprise a mainboard featuring one or more communication buses612that interconnect the processor610, the memory602, and various peripherals, using a variety of bus technologies, such as a variant of a serial or parallel AT Attachment (ATA) bus protocol; a Uniform Serial Bus (USB) protocol; and/or Small Computer System Interface (SCI) bus protocol. In a multibus scenario, a communication bus612may interconnect the computer504with at least one other computer. Other components that may optionally be included with the computer504(though not shown in the schematic architecture diagram600ofFIG.6) include a display; a display adapter, such as a graphical processing unit (GPU); input peripherals, such as a keyboard and/or mouse; and a flash memory device that may store a basic input/output system (BIOS) routine that facilitates booting the computer504to a state of readiness.

The computer504may operate in various physical enclosures, such as a desktop or tower, and/or may be integrated with a display as an “all-in-one” device. The computer504may be mounted horizontally and/or in a cabinet or rack, and/or may simply comprise an interconnected set of components. The computer504may comprise a dedicated and/or shared power supply618that supplies and/or regulates power for the other components. The computer504may provide power to and/or receive power from another computer and/or other devices. The computer504may comprise a shared and/or dedicated climate control unit620that regulates climate properties, such as temperature, humidity, and/or airflow. Many such computers504may be configured and/or adapted to utilize at least a portion of the techniques presented herein.

FIG.7presents a schematic architecture diagram700of a client device510whereupon at least a portion of the techniques presented herein may be implemented. Such a client device510may vary widely in configuration or capabilities, in order to provide a variety of functionality to a user such as the user512. The client device510may be provided in a variety of form factors, such as a desktop or tower workstation; an “all-in-one” device integrated with a display708; a laptop, tablet, convertible tablet, or palmtop device; a wearable device mountable in a headset, eyeglass, earpiece, and/or wristwatch, and/or integrated with an article of clothing; and/or a component of a piece of furniture, such as a tabletop, and/or of another device, such as a vehicle or residence. The client device510may serve the user in a variety of roles, such as a workstation, kiosk, media player, gaming device, and/or appliance.

The client device510may comprise one or more processors710that process instructions. The one or more processors710may optionally include a plurality of cores; one or more coprocessors, such as a mathematics coprocessor or an integrated graphical processing unit (GPU); and/or one or more layers of local cache memory. The client device510may comprise memory701storing various forms of applications, such as an operating system703; one or more user applications702, such as document applications, media applications, file and/or data access applications, communication applications such as web browsers and/or email clients, utilities, and/or games; and/or drivers for various peripherals. The client device510may comprise a variety of peripheral components, such as a wired and/or wireless network adapter706connectible to a local area network and/or wide area network; one or more output components, such as a display708coupled with a display adapter (optionally including a graphical processing unit (GPU)), a sound adapter coupled with a speaker, and/or a printer; input devices for receiving input from the user, such as a keyboard711, a mouse, a microphone, a camera, and/or a touch-sensitive component of the display708; and/or environmental sensors, such as a global positioning system (GPS) receiver719that detects the location, velocity, and/or acceleration of the client device510, a compass, accelerometer, and/or gyroscope that detects a physical orientation of the client device510. Other components that may optionally be included with the client device510(though not shown in the schematic architecture diagram700ofFIG.7) include one or more storage components, such as a hard disk drive, a solid-state storage device (SSD), a flash memory device, and/or a magnetic and/or optical disk reader; and/or a flash memory device that may store a basic input/output system (BIOS) routine that facilitates booting the client device510to a state of readiness; and a climate control unit that regulates climate properties, such as temperature, humidity, and airflow.

The client device510may comprise a mainboard featuring one or more communication buses712that interconnect the processor710, the memory701, and various peripherals, using a variety of bus technologies, such as a variant of a serial or parallel AT Attachment (ATA) bus protocol; the Uniform Serial Bus (USB) protocol; and/or the Small Computer System Interface (SCI) bus protocol. The client device510may comprise a dedicated and/or shared power supply718that supplies and/or regulates power for other components, and/or a battery704that stores power for use while the client device510is not connected to a power source via the power supply718. The client device510may provide power to and/or receive power from other client devices.

FIG.8is an illustration of a scenario800involving an example non-transitory machine-readable medium802. The non-transitory machine-readable medium802may comprise processor-executable instructions812that when executed by a processor816cause performance (e.g., by the processor816) of at least some of the provisions herein. The non-transitory machine-readable medium802may comprise a memory semiconductor (e.g., a semiconductor utilizing static random access memory (SRAM), dynamic random access memory (DRAM), and/or synchronous dynamic random access memory (SDRAM) technologies), a platter of a hard disk drive, a flash memory device, or a magnetic or optical disc (such as a compact disk (CD), a digital versatile disk (DVD), or floppy disk). The example non-transitory machine-readable medium802stores computer-readable data804that, when subjected to reading806by a reader810of a device808(e.g., a read head of a hard disk drive, or a read operation invoked on a solid-state storage device), express the processor-executable instructions812. In some embodiments, the processor-executable instructions812, when executed cause performance of operations, such as at least some of the example method200ofFIG.2and/or at least some of the example method400ofFIG.4, for example. In some embodiments, the processor-executable instructions812are configured to cause implementation of a system, such as at least some of the example system101ofFIGS.1A-1Eand/or at least some of the example system301ofFIGS.3A-3B, for example.

Various operations of embodiments are provided herein. In an embodiment, one or more of the operations described may constitute computer readable instructions stored on one or more computer readable media, which if executed by a computing device, will cause the computing device to perform the operations described. The order in which some or all of the operations are described should not be construed as to imply that these operations are necessarily order dependent. Alternative ordering may be implemented without departing from the scope of the disclosure. Further, it will be understood that not all operations are necessarily present in each embodiment provided herein. Also, it will be understood that not all operations are necessary in some embodiments.

Also, although the disclosure has been shown and described with respect to one or more implementations, alterations and modifications may be made thereto and additional embodiments may be implemented based upon a reading and understanding of this specification and the annexed drawings. The disclosure includes all such modifications, alterations and additional embodiments and is limited only by the scope of the following claims. The specification and drawings are accordingly to be regarded in an illustrative rather than restrictive sense. In particular regard to the various functions performed by the above described components (e.g., elements, resources, etc.), the terms used to describe such components are intended to correspond, unless otherwise indicated, to any component which performs the specified function of the described component (e.g., that is functionally equivalent), even though not structurally equivalent to the disclosed structure. In addition, while a particular feature of the disclosure may have been disclosed with respect to only one of several implementations, such feature may be combined with one or more other features of the other implementations as may be desired and advantageous for any given or particular application.