Patent ID: 12219180

DETAILED DESCRIPTION

In the current era of big data, data is becoming more and more abundant. Running intricate algorithms on the full dataset incurs tremendous computational burden. For example, conventional image compression algorithm can involve tremendous computational burden. Image compression is generally performed using lossy compression or lossless compression. Lossy image compression allows for some distortions in the decompressed data in exchange for a higher compression rate, while lossless image compression requires that there be no quality loss in the decompressed data. Lossless compression is particularly important in those applications where the original image content cannot be altered (e.g., to enable further processing, archiving etc.).

In conventional image compression systems, lossless compression algorithms include the steps of transforming the image, predicting the pixel value, context modeling, and entropy coding. In lossless compression, the pixel-value prediction process takes into account the difference between actual pixel values and the predicted pixel values. The pixel-value prediction process generally utilizes the surrounding pixels to predict the target pixel value.

The context modeling process takes the error in prediction (also called residuals) and clusters error values with similar statistical characteristics based on some context information derived from surrounding pixels. Context modeling in image compression algorithms, also known as error context modeling, builds or trains a model for the residual for each pixel with respect to a context. The context model built using image compression systems is typically in the form of a decision tree. Context modeling, is an important component in image compression systems. However, it is also an expensive component in terms of time and resource utilization costs. For example, in conventional image processing systems, context models are built using all available context-residual pairs (e.g., a context-residual pair for each pixel in an image), which is resource intensive and incurs a tremendous computational cost.

By way of example, Free Lossless Image Format (FLIF) is a particular type of lossless image compression algorithm used in an image compression system. FLIF utilizes a context model called Meta-Adaptive Near-zero Integer Arithmetic Coding (MANIAC). The MANIAC context model is a decision tree based error context model. The MANIAC context model builds a tree using the context for each pixel (derived from surrounding pixels) and the difference in prediction values. Each leaf node of the MANIAC tree maintains a frequency table, which is subsequently used to entropy encode the residuals. The tree is a clustering model where data points from the training set are navigated through the tree into one of the leaf nodes. As the tree is built or trained, each leaf node contains a cluster of data points from the training set. Building the MANIAC tree is an important, but a very time-consuming step in the entire image compression process. In particular, in order to build the MANIAC tree, the conventional context modeling process iterates over all available context-residual pairs. This training procedure can take around 45% of the encoding time.

Accordingly, embodiments of the present disclosure are directed towards reducing the time and resource utilization used to perform context modeling (e.g., for image compression). The embodiments of the present disclosure utilize actively-learned context modeling to build a context model using a portion of the data that has similar performance to a context model generated with all the data. In this regard, a context model is generated using context-residual pairs associated with only a portion of pixels. Using only a portion of context-residual pairs to generate a context model enables a more efficient generation of context models, thereby increasing the efficiency and resource utilization of performing image compression.

At a high level, to actively train a context model using context-residual pairs associated with only a portion of pixels, an iterative process is performed to selectively identify suitable context-residual pairs for generating the context model. In each iteration, context-residual pairs associated with additional pixels can be selected to build an updated context model until a threshold size of subset of data is obtained. To select pixels and/or context-residual pairs to add to the subset of data used to generate the context model, various methods can be used.

For example, one embodiment involves selecting pixels by evaluating the likelihood of prediction of the remaining data in the training set and updating the subset of data with the largest likelihood of prediction value over the entire dataset. Determining the likelihood of prediction involves determining the value of the likelihood function of probability. To determine the likelihood of prediction, the residuals are used as the observed data and the context model is used as the probability. Updating the subset with a pixel that provides the largest likelihood of prediction can result in a low cross entropy value over the entire residual data. A low cross entropy value over the entire residual data is desirable. The smaller the cross entropy is, a more efficient compression can be obtained. A low cross entropy indicates that the probability function is very accurate over the whole data. In some examples, the subset is updated with only one context-residual pair (corresponding to the pixel having the largest likelihood of prediction value over the entire dataset). In another example, the subset is updated with only a certain number or a batch size of context-residual pairs (for e.g. corresponding to a certain number of pixels having the largest likelihood of prediction over the entire dataset). In some embodiments, the likelihood of prediction value may have to be calculated for each pixel absent in the subset.

Another embodiment involves selecting a higher percentage of pixels from leaf nodes in the context model that have a higher entropy value. Entropy values generally refer to values that indicate an extent of diversity of context associated with pixels in a cluster that represent a leaf node in a context model. As described herein, leaf nodes associated with a greater or higher entropy value indicate a greater diversity of context associated with the pixels contained in the cluster therein. If the entropy value of the leaf node is low, then the context-residual pairs in the leaf nodes cluster share similarities which would lead to a desirable context model. If the entropy value of the leaf node is high, then the context-residual pairs in the leaf node's cluster don't share similarities. The entropy allows the actively-learned context modeling method to determine whether the selected portion of data is diverse. The context model built using a diverse portion of data has low redundancy. It should be understood that other methods of updating the subset can also be used. The subset of data is iteratively updated and an updated context model is generated during each iteration until the size of the subset of data attains a threshold subset of data size (e.g., data associated with a predetermined number of pixels). When the subset of data attains or reaches the threshold subset of data size, the final updated subset of data is used to build the final context model.

In operation, to perform actively-learned context modeling, an initial subset of data from a training dataset is selected (e.g., at random). Such selected data can include context-residual pairs corresponding with any number of pixels. The training dataset can include context-residual pairs corresponding with each pixels, for example, in an image. The actively-learned context modeling method builds or trains an initial context model using the initial subset of data (e.g., context-residual pairs) and then selects additional data to update the subset with. As described, the actively-learned context modeling can select data to update the subset using different techniques that can provide the lowest cross entropy value with the updated context model over the entire residual data. In one embodiment, the actively-learned context modeling evaluates the likelihood of prediction of the remaining data in the training set and updates the subset of data with the largest likelihood of prediction. In another embodiment, it updates the subset of data by selecting higher percentage of data from leaf nodes in the context model that have a higher entropy value. The entropy allows the actively-learned context modeling method to determine whether the selected portion of data is diverse. The context model built using a diverse portion of data has low redundancy. A desirable context model can have a low cross entropy over the entire dataset.

The actively-learned context modeling method continues to iteratively update the subset of data and generate updated context models therefrom until the size of the subset of data attains a threshold subset of data size (e.g., data associated with a predetermined number of pixels). When the subset of data attains or reaches the threshold subset of data size, the final updated subset of data is used to build the final context model. The threshold subset of data size can be a number of pixels allowed for training or a number of data allowed for training (an allowable data value). For example, a threshold subset of data size can be a predetermined amount of pixels from the training set or training data. In another example, threshold subset of data size can be a number based on how fast the process should be.

Advantageously, iteratively refining the subset of data by selectively updating the subset (e.g., by selecting data based, at least in part, on the likelihood of prediction or entropy values of the residuals in leaf nodes) allows the image compression system to build or train a context model having a lower redundancy, thereby enabling the context model to have similar performance to a context model built using all the data (e.g., context-residual pairs associated with each pixel of an image). Further, utilizing only a portion of available data (e.g., context-residual pairs associated with only a portion of pixels of an image) enables a more efficient process, resulting in reduction of computing resources needed to perform image compression.

It should be understood that the embodiments of the present technology can be used in any system that uses context modeling and is not limited to systems using image processing with error context modeling.

Exemplary Environment for Image Compression

Turning toFIG.1,FIG.1is a diagram of an environment used to perform actively-learned context modeling, according to embodiments of the present disclosure. It should be understood that this and other arrangements described herein are set forth only as examples. Other arrangements and elements (e.g., machines, interfaces, functions, orders, and groupings of functions, etc.) can be used in addition to or instead of those shown, and some elements may be omitted altogether for the sake of clarity. Further, many of the elements described herein are functional entities that may be implemented as discrete or distributed components or in conjunction with other components, and in any suitable combination and location. Various functions described herein as being performed by one or more entities may be carried out by hardware, firmware, and/or software. For instance, some functions may be carried out by a processor executing instructions stored in memory as further described with reference toFIG.9.

The system100is an example of a suitable architecture for implementing certain aspects of the present disclosure. In one embodiment, the system100includes, among other components not shown, an image compression system102, a server104, and a user device106. Each of the image compression system102, server104, and user device106shown inFIG.1can comprise one or more computer devices, such as the computing device900ofFIG.9, discussed below. As shown inFIG.1, the image compression system102, the server104, and the user device106can communicate via a network108, which may include, without limitation, one or more local area networks (LANs) and/or wide area networks (WANs). Such networking environments are commonplace in offices, enterprise-wide computer networks, intranets, and the Internet. It should be understood that any number of user devices and servers may be employed within the system100within the scope of the present disclosure. Each may comprise a single device or multiple devices cooperating in a distributed environment. For instance, the image compression system102could be provided by multiple devices collectively providing the functionality of the image compression system102as described herein. Additionally, other components not shown may also be included within the network environment.

It should be understood that any number of user devices, servers, and other components can be employed within the operating environment100within the scope of the present disclosure. Each can comprise a single device or multiple devices cooperating in a distributed environment.

User device106can be any type of computing device capable of being operated by a user. For example, in some implementations, user device106is the type of computing device described in relation toFIG.9. By way of example and not limitation, a user device106may be embodied as a personal computer (PC), a laptop computer, a mobile device, a smartphone, a tablet computer, a smart watch, a wearable computer, a personal digital assistant (PDA), an MP3 player, a global positioning system (GPS) or device, a video player, a handheld communications device, a gaming device or system, an entertainment system, a vehicle computer system, an embedded system controller, a remote control, an appliance, a consumer electronic device, a workstation, any combination of these delineated devices, or any other suitable device.

The user device106can include one or more processors, and one or more computer-readable media. The computer-readable media may include computer-readable instructions executable by the one or more processors. The instructions may be embodied by one or more applications, such as application120shown inFIG.1. Application120is referred to as a single application for simplicity, but its functionality can be embodied by one or more applications in practice. As indicated above, the other user devices can include one or more applications similar to application120.

The application(s) may generally be any application capable of facilitating actively-learned context modeling (e.g., via the exchange of information between the user devices and the server104). In some implementations, the application(s) comprises a web application, which can run in a web browser, and could be hosted at least partially on the server-side of environment100. In addition, or instead, the application(s) can comprise a dedicated application, such as an application having image processing functionality. In some cases, the application is integrated into the operating system (e.g., as a service). It is therefore contemplated herein that “application” be interpreted broadly.

In accordance with embodiments herein, the application120can either initiate the actively-learned context modeling module114or facilitate actively-learned context modeling via a set of operations initiated, for example, based on a user selection. In embodiments, an image(s) is selected that will be processed using actively-learned context modeling. Additionally in some embodiments, users can select suitable entropy values or threshold subset of data sizes or any other values that can be used to determine a desirable context model in the actively-learned context modeling. The actively-learned context modeling module114will perform the actively-learned context modeling method described herein. The actively-learned context modeling module114performs an iterative process to selectively identify suitable context-residual pairs associated with only a portion of pixels. An iterative process is performed to selectively identify suitable context-residual pairs for generating the context model. In each iteration, pixels, the actively-learned context modeling module114selects one or more context-residual pairs to build an updated context model until a threshold size of subset of data is obtained. The pixels and/or context-residual pairs are selected based on likelihood of prediction of the remaining data in the training set or by based on entropy values of each leaf node in the context model. It should be understood that other methods of updating the subset can also be used. A desirable context model will have low cross entropy over the entire dataset. The threshold subset of data size can be a number of pixels allowed for training. For example, threshold subset of data size can be a predetermined amount of pixels from the training set or training data. In another example, threshold subset of data size can be a number based on how fast the process should be. In some embodiments, the application120can initiate multiple operations to effectuate actively-learned context modeling during image processing. For example, the application120can initiate multiple processes using the actively-learned context modeling modules114on a burst compression containing multiple images. In operation, a user can indicate or provide an image to process. The application120can initiate the image transformation module110in the image compression system102and/or provide the image to the image transformation module110. The image transformation module110obtains an image (e.g., from user device106or a data store) and transforms the image to decorrelate the color channels. Upon image transformation, the prediction module112utilizes the surrounding pixels to predict the target pixel value for each pixel. The difference between the targeted value and the actual value of each pixel is the residual value of the pixels.

To perform actively-learned context modeling, the actively-learned context modeling module114selects an initial subset of data from the training set (e.g. at random). Such selected data can include context-residual pairs corresponding with any number of pixels. The training dataset can include context-residual pairs corresponding with each pixels, for example, in an image. The actively-learned context modeling module114builds or trains an initial context model using the initial subset of data (e.g., context-residual pairs). An iterative process is performed to selectively identify suitable context-residual pairs for generating the context model. In each iteration, the actively-learned context modeling module114selects one or more context-residual pairs to build an updated context model until a threshold size of subset of data is obtained. The actively-learned context modeling module114updates the subset by selecting one or more context-residual pairs using different methods.

One embodiment involves selecting context-residual pairs by evaluating the likelihood of prediction of the remaining data in the training set and updating the subset of data with the pixel or the context-residual pair having the largest likelihood of prediction value over the entire dataset. Determining the likelihood of prediction involves determining the value of likelihood function of probability. To determine the likelihood of prediction, the residuals are used as the observed data and the context model is used as the probability. Updating the subset with a pixel that provides the largest likelihood of prediction can result in a low cross entropy value over the entire residual data. A low cross entropy value over the entire residual data is desirable. The smaller the cross entropy is, the better compression can be obtained. A low cross entropy indicates that the probability function is very accurate over the whole data. In some examples, the subset is updated with only one context-residual pair (corresponding to the pixel having the largest likelihood of prediction value over the entire dataset). In another example, the subset is updated with a batch size of context-residual pairs (for e.g. corresponding to the pixels having the largest likelihood of prediction over the entire dataset). In some embodiments, the likelihood of prediction value may have to be calculated for each pixel in each iteration. In other embodiments, the likelihood of prediction is calculated for a portion of pixels not included in the subset of data.

Another embodiment involves selecting a higher percentage of pixels from leaf nodes in the context model that have a higher entropy value. Entropy values generally refer to values that indicate an extent of diversity of context associated with pixels in a cluster that represent a leaf node in a context model. As described herein, leaf nodes associated with a greater or higher entropy value indicate a greater diversity of context associated with the pixels contained in the cluster therein. If the entropy value of the leaf node is low, then the context-residual pairs in the leaf nodes cluster share similarities which would lead to a desirable context model. If the entropy value of the leaf node is high, then the context-residual pairs in the leaf node's cluster don't share similarities. The entropy allows the actively-learned context modeling method to determine whether the selected portion of data is diverse. The context model built using a diverse portion of data has low redundancy. It should be understood that other methods of updating the subset can also be used.

The actively-learned context modeling module114continues to iteratively update the subset of data and generate updated context models therefrom until the size of the subset of data is attains a threshold subset of data size (e.g., data associated with a predetermined number of pixels). When the subset of data attains or reaches the threshold subset of data size, the final updated subset of data is used to build the final context model. The threshold subset of data size can be a number of pixels allowed for training or a number of data allowed for training (an allowable data value). For example, a threshold subset of data size can be a predetermined amount of pixels from the training set or training data. In another example, threshold subset of data size can be a number based on how fast the process should be.

The final updated subset of data that is identified by the actively-learned context modeling module114can build a context model that has similar performance to a context model built with all the data. The entropy coder module116encodes the context model generated by the actively-learned context modeling module114.

As described herein, server104can facilitate built a context model using the actively-learned context modeling module114. Server104includes one or more processors, and one or more computer-readable media. The computer-readable media includes computer-readable instructions executable by the one or more processors. The instructions may optionally implement one or more components of the actively-learned context modeling module114, described in additional detail herein. At a high level, the actively-learned context modeling module114performs an iterative process to selectively identify suitable context-residual pairs associated with only a portion of pixels. An iterative process is performed to selectively identify suitable context-residual pairs for generating the context model. In each iteration, pixels, the actively-learned context modeling module114selects context-residual pairs associated with the highest likelihood of prediction or selects a higher percentage of context-residual pairs associated with leaf nodes in the context model having a higher entropy value. It should be understood that other methods of updating the subset can also be used.

For cloud-based implementations, the instructions on server104may implement one or more components of the actively-learned context modeling module114, and application120may be utilized by a user to interface with the functionality implemented on server(s)104. In some cases, application120comprises a web browser. In other cases, server104may not be required. For example, the components of the actively-learned context modeling module114may be implemented completely on a user device, such as user device106. In this case, the actively-learned context modeling module114may be embodied at least partially by the instructions corresponding to application120. Therefore, the actively-learned context modeling module114can operate on a server, such as server104, or on a user device, such as user device106or partially on both.

These components may be in addition to other components that provide further additional functions beyond the features described herein. The image compression system102can be implemented using one or more devices, one or more platforms with corresponding application programming interfaces, cloud infrastructure, and the like. While the image compression system102is shown separate from the user device106in the configuration ofFIG.1, it should be understood that in other configurations, some or all of the functions of the image compression system102can be provided on the user device106.

System Using Actively-Learned Context Modeling

With reference to figuresFIG.2andFIG.3,FIG.3is a block diagram illustrating an exemplary image compression system300using actively-learned context modeling, in accordance with one embodiment of the present disclosure, andFIG.2is a diagram of an exemplary image200used in the image compression system. Turning toFIG.3, the image compression system obtains an image320to be compressed. The image320can be a Red-Green-Blue (RGB) source image file. In one example, the image transformation module322uses reversible Luma, Chroma Orange, and Chroma Green (YCoCg) color transformation to decorrelate the color channels. After the image compression system300transforms the image320to different color channels304, the image compression system300processes the color channels304through the rest of the modules in the image compression system300one after another following the order of Luma (Y), Chroma Orange (Co), and Chroma Green (Cg).FIG.2illustrates an exemplary image200in one of the color channels. The image200is made up of pixels204-264. The image compression system300calculates the residuals312for each pixel in the image200by calculating the difference309between the actual value304of the pixel and the predicted value310of the pixel. To calculate the residual312, the image compression system300first uses the prediction module302to predict the target pixel value for each pixel204-264. For example, to predict the target value for pixel232marked as “?” inFIG.2, the prediction module302will take the median of surrounding pixels. For example, the prediction module302will take into consideration the pixel located towards the top220of the target pixel232marked as (T), the pixel located towards the left256of the target pixel232marked as (L), and the pixel located towards the top left264of the target pixel232marked as (TL). One way of determining the median is by the following using the following pixel information: the top pixel T (232), the left pixel L (256), and the gradient of the top and left pixels determined by T+L−TL. It should be understood that a combination of other surrounding pixels can be used to predict the target pixel232.

After predicting the pixel values for each pixel204-264, the difference309between predicted pixel value310and actual value304is determined to determine the residual value312for each pixel204-265. This difference or error value between the actual value304of the pixel and the predicted pixel value310of the pixel is also called a residual value312.

In one embodiment, after determining the residual values312for the pixels, the image compression system300utilizes an actively-learned context modeling module311to obtain a context model for the residuals312. The actively-learned context modeling module311will initiate the actively-learned context modeling method described herein. The actively-learned context modeling module311identifies a subset of data from a training set to build or train a context model that can obtain a similar performance as a context model built or trained using all the data. The training set can include the context values313and residual values312for pixels in the image. The data can consist of any other information as well or any combination of different data as well.

In some embodiments, the actively-learned context modeling module311selects an initial subset of data from a training set. In some embodiments, a subset of data may be randomly selected. For example, if the total pixels in an image are pixels 1-10, then context-residual pairs for pixels 1, 2, 7, and 9 are randomly selected as the initial subset of data. The initially selected subset of data is used to build or train an initial context model. For example, the rest of the pixels not included in the subset are then navigated into one of the leaf nodes in the context model depending on the context for each pixel and the decision nodes of the context model. For example, after the context model is built or trained first leaf node has a cluster that includes context-residual pairs for pixels 1, 3, and 4. The second leaf node has a cluster that includes context-residual pairs for pixel 2. The third leaf node has a cluster that includes context-residual pairs for pixels 5 and 7. The fourth leaf node has a cluster that includes context-residual pairs for pixels 8 and 9.

After the initial context model is built, the actively-learned context modeling module311can select data or context-residual pairs from the training set to update the subset. For example, in one embodiment, the actively-learned context modeling module calculates the likelihood of prediction of the remaining pixels not in the subset and updates the subset with the pixel having the largest likelihood of prediction value of the remaining data in the training set. In another embodiment, the actively-learned context modeling module determines the entropy value of each leaf node in the tree and updates the subset based on the entropy value of each leaf node in the tree. For example, a higher percentage of data is selected from leaf nodes in the context model that have a higher entropy value. If the entropy value of the leaf node is low, then the context-residual pairs in the leaf nodes cluster share similarities which would lead to a desirable context model. If the entropy value of the leaf node is high, then the context-residual pairs in the leaf node's cluster don't share similarities. It should be understood that other methods of selecting data to update the subset can also be used. The updated subset of data can then be used to generate an updated context model. The actively-learned context modeling module311continues to iteratively update the subset of data and generate updated context models therefrom until the size of the subset of data attains a threshold subset of data size (e.g., data associated with a predetermined number of pixels). When the subset of data attains or reaches the threshold subset of data size, the final updated subset of data is used to build the final context model. The threshold subset of data size can be a number of pixels allowed for training or a number of data allowed for training (an allowable data value). For example, a threshold subset of data size can be a predetermined amount of pixels from the training set or training data. In another example, threshold subset of data size can be a number based on how fast the process should be.

By selecting only a portion of the data to build or train the context model, the actively-learned context modeling module311may provide a speedup of the image compression system300while slightly reducing compression rates i.e. the output bitstream will be slightly longer, or slightly larger compressed file size. In this example, the actively-learned context modeling module311is used after obtaining the residual values312and before encoding the context model using entropy encoding module316. It should be understood that the actively-learned context modeling module311can be used anywhere in the image compression system300or in any algorithm using a context modeling module.

When the final subset of data is identified, the context model built or trained using the final subset of data is provided to the entropy coder module316. The entropy coder module316will entropy code the context model and represent the image in an efficient manner to prepare it for transmission. The entropy encoded data travels through the bitstream318and transmitted over a channel. For example, the entropy encoded data travels to a server, user device, or a cloud computing service, or the like. After it is transmitted, a decoder performs reverse operations illustrated in system300to obtain the image.

With reference toFIG.4,FIG.4are exemplary images400illustrating an implementation of actively-learned context modeling in accordance with various embodiment of the present disclosure.

In one embodiment, the actively-learned context modeling module311selects an initial portion of pixels at random404that are indicated by non-grayed boxes. The portions of the pixels in404that are non-gray are the pixels in the initial subset that the actively-learned context modeling module311selects initially to build the context model. These initial pixels404for the initial subset can be selected randomly. With that selected subset of data of pixels, an initial context model ƒθis generated404and the initial context model is updated using the remaining pixels408not in the subset404. As illustrated in408, the context model built or trained using pixels not in the initial subset404. The430,434,438,440pixels in408are the pixels in the initial subset. The context model is built or trained using pixels not in the initial subset404that are not430,434,438,440pixels in408. After the context model is built, a pixel416is selected to update the subset. In this example, the subset includes in the initial non-gray pixels from404, and a new pixel416is selected and the subset is updated with the new pixel416. Note that the new pixel416is a pixel that doesn't have more information than the rest of the pixels418,420,424. The actively-learned context modeling module311can use any process to select context-residual pairs to update the subset that can provide the lowest cross entropy value with the updated context model over the entire residual data. For example, the actively-learned context modeling module311can use either the likelihood of prediction or entropy value of residuals in leaf nodes in the context model to select a pixel or a subset of pixels to update the subset. With the updated subset of data412, the actively-learned context modeling module311builds or trains a new context model.

FIG.5illustrates an exemplary context model500built using only a portion of data for a single image in accordance with one embodiment of the present disclosure. In one example, the context model500is a decision tree. The decision tree500includes decision nodes504,508,512. The tree is a clustering model where candidate data points are navigated through the decision nodes504,508,512to navigate into one of the leaf nodes516,520,524,528.

In one embodiment, the decision tree500clusters a portion of data determined by the actively-learned context modeling module311into leaf nodes516,520,524,528based on the context or property values and the decision node. For example, leaf node516includes cluster530a, leaf node520includes cluster530b, leaf node524includes cluster530c, and leaf node528includes cluster530d. Each pixel is navigated into one of the leaf nodes and joins the cluster of data in the leaf node.

In this example, each pixel in the cluster530a,530b,530c,530dis represented by a context-residual pair (c, ϵ), where c corresponds to the context information313for a pixel and ϵ corresponds to the residual value312for the pixel. In another example, c can correspond to specific property values for the pixel. It should be understood that other information and combinations of the information can be used to build the tree500.

Exemplary Method Implementing Actively-Learned Context Modeling Using Algorithm 1

FIG.6is a flow diagram illustrating an exemplary method600for implementing actively-learned context modeling in accordance with one embodiment of the present disclosure. A processing devices such as a user device, a server, a cloud computing service or the like implements the exemplary method600. The actively-learned context modeling module of an image compression system can initiate the actively-learned context modeling method600described herein. In one example, actively-learned context modeling enables identification of a portion of data to build or train a context model that can obtain a similar performance as a context model built using all the data. In embodiments, the actively-learned context modeling enables identification of the portion of data using a greedy approach. This approach involves exhaustively comparing evaluating the likelihood of prediction function values of the remaining data not in the subset, and selecting the one with largest likelihood of prediction value. One example of method600is based on Algorithm 1 further described below.

In one embodiment of a processing device implementing the method600at block604selects an initial subset of data or portion from a training set corresponding to an image to be compressed. In one example, the initial subset of data is selected at random from the training set. The training set can be include data such as context and residuals pairs for each pixel. The initial subset of data can be selected at random or using a predetermined method or using an algorithm. It could also be selected based on data from the image such as the context, residual or the like. For example, the initial subset of data can be selected to be diverse so that the elements have low redundancy. Some of the data in the initial subset of data can be selected using a combination of the above. In one example, the initial subset of data is determined by an algorithm or provided by a user.

Continuing withFIG.6, the processing device implementing the method600at block608, builds or trains or generates an initial context model based on the initial subset of data. The context model is in the form of a decision tree having a set of leaf nodes.

The processing device implementing the method600at block612determines a likelihood of prediction for the context model on a portion of residual data, for example, for residual data of pixels absent or not included in the subset. In another example, the processing device at block612determines the likelihood of prediction value for each pixel not in the subset. The processing device uses the likelihood of prediction to select context-residual pairs with the highest likelihood of prediction value to obtain a low cross entropy value over the entire residual data. In one example, in every iteration, only the context-residual pair having the highest likelihood of prediction value is selected and added to the subset. In another example, a certain number or a batch size of context-residual pairs having the highest likelihood of prediction value can be selected and added to the subset.

Continuing withFIG.6, the processing device implementing the method600at block616, updates the subset of data with the context-residual pair having the largest likelihood of prediction. The context-residual pair is selected from the data absent in the subset of data. In another example, the processing device implementing the method600at block616, updates the subset of data with a batch size of context-residual pairs having the highest likelihood of prediction value (e.g., the 5 context-residual pairs having the highest likelihood of prediction value are selected and used to update the subset). The batch size of context-residual pairs can be provided or determined.

The processing device implementing the method600at block628determines whether the size of the subset of data is less than a threshold subset of data size. If the size of the subset of data is less than the threshold subset of data size in block628, then the processing device implementing the method600goes back to block608and builds a context model with the updated subset. If the size of the updated subset of data is determined to be not less than the threshold subset of data size in block628, then the processing device implementing the method600at block630builds the final context model using the updated subset of data.

Exemplary Method Implementing Actively-Learned Context Modeling Using Algorithm 2

FIG.7is a flow diagram illustrating an exemplary method700for implementing actively-learned context modeling in accordance with one embodiment of the present disclosure. A processing device such as a user device, server, cloud computing service or the like can implement the exemplary method700. The actively-learned context modeling module of an image compression system can initiate the actively-learned context modeling method700described herein. One example of method700is based on Algorithm 2 further described below.

In one embodiment of the method700, a processing device implementing the method700at block704selects an initial subset of data or portion of data from the training dataset or training set. The training dataset corresponds to an image to be compressed. The subset of data corresponds to a subset of data of pixels of the image. The initial subset of data can be selected at random or using a predetermined method or using an algorithm.

Continuing withFIG.7, the processing device implementing the method700at block708builds or train or generate a context model based on the initial subset of data. The processing device implementing the method700at block712determines an entropy value for each leaf node of the context model. The entropy value indicates an extent of diversity of context associated with the corresponding leaf node. The processing device implementing the method700at block716samples or selects data from each leaf node of the context model based on the entropy value of the leaf node. In one example, the subset of data is updated based on the calculated entropy value. For example, if the entropy value for the leaf node in the context model has a high entropy value, then a higher percentage of the data is selected from the leaf node having a higher entropy value and lower percentage of data is selected from the leaf node having a lower entropy value. The subset is then updated with this selected data. For example, if a leaf node has a higher entropy value, then a higher percentage of the updated subset of data will come from that leaf node. In one example, the method700updates the subset each time with a batch of data determined by a batch size. For example, if batch size is 5, then during each iteration 5 context-residual pairs are added to the subset. A higher percentage of the context-residual pairs in the batch will come from leaf nodes with a higher entropy value and a lower percentage of the context-residual pairs in the batch will come from leaf nodes with a lower entropy value.

Continuing withFIG.7, the processing device implementing the method700at block720updates the subset of data with the sampled data from block716. The method processing device implementing the method700at block724determines whether the updated subset of data is less than a threshold subset of data size. The threshold subset of data size is a threshold value that can be predetermined, can be provided, or can be calculated or determined. If the size of the updated subset of data is determined to be less than the threshold subset of data size in block724, then the processing device implementing the method700continues back to block708to build a context model using the updated subset of data. If the size of the updated subset of data is determined to be not less than the threshold subset of data size in block724, then the processing device implementing the method700at block728builds a final context model using the updated subset of data.

Exemplary Algorithms Using Actively-Learned Context Modeling

Algorithm 1

In one embodiment, Algorithm 1 provides one example of an algorithm that can be used to implement actively-learned context modeling. The actively-learned context modeling module can initiate the actively-learned context modeling method described in Algorithm 1. Algorithm 1 can be used to identify or select a portion or a subset of data from a training set such that the context model is generated or trained with the selected portion and obtains a similar performance to a context model generated or trained using all the data. Algorithm 1 can be seen in Table 1:

TABLE 1Algorithm 1: Active context modelingRequire: training set= {(ci, ϵi)}i=1N, context model pθ,selection budget M1:select a training pair at random and update2:while || < M do3:train a context model pθusing4:k = arg minjlog pθ(ϵj| cj), ∀(cj, ϵj) ∉5:←∪ {(ck, ϵk)}6:end while7:train the final context model pθusing

According to Algorithm 1, given a training set, Budget M, context model ƒθand training procedure, a subset of data S can be determined or found using steps 3-6 of Algorithm 1 such that a subset S is selected of size M, where the subset contains context-residuals pairs with the highest likelihood of prediction. The selected subset of data is updated with context-residual pairs having the highest likelihood prediction values, which will decrease the cross entropy on entire residual data of the updated model trained with updated subset. The low cross entropy over the entire residual data can be determined by Σ−pilog p(ƒ{circumflex over (θ)}(ci)), where pirefers to probability of pixel i and log p (ƒ{circumflex over (θ)}(ci)) is the likelihood of prediction value.

As seen in Algorithm 1, in one example, a training set or data and a budget M is provided, determined, predetermined, or calculated. The value M is also the threshold subset of data size and can be a number of pixels allowed for training or a number of data allowed for training (an allowable data value). For example, M can be a predetermined amount of pixels from the training set or training data. In another example, M can be a number based on how fast the process should be. The budget M is the size of final subset. In another example, the budget could be compression speed, which is determined by the subset size. An initial subset of data is selected by the actively-learned context modeling module311(Step 1 of Algorithm 1). S is the size of the current subset of data and M is the size of the final subset. In one example, the subset of data S is called a training pair and all the context-residual pairs of data are the training set. The subset of data S can either be selected at random, through a parameter, through an input, through an algorithm, through predetermined methods or through any other method. In one example, a training pair is selected at random and the subset of data S is updated (Step 1 of Algorithm 1).

In one example, while the size of the current subset of data S is less than the budget M or a threshold subset of data size, steps 3-6 of Algorithm 1 are performed (Step 2 of Algorithm 1). The steps 3-6 of the algorithm include building or training a context model using the subset of data S (Step 3 of Algorithm 1), selecting a context-residual pair to update the subset with (Step 4 of Algorithm 1) and updating the subset with the context-residual pair (Step 5 of Algorithm 1).

Steps 3-6 of the algorithm include performing mathematical analysis on the context model after each iteration to determine the likelihood of prediction that represents a number of bits required to encode each residual in the context model. Data points with high likelihood are encoded efficiently using the updated context model. The likelihood of prediction can be determined by calculating the log probability for each residual in the entire training set not including the data in the subset. The cross entropy of a context model over the entire dataset can be determined by Σ−pilog p(ƒ{circumflex over (θ)}(ci)). The negative log probability value of the entire training set is equivalent to the cross entropy.

For example, in step 4 of the Algorithm, the log probability for each residual in the entire training set not in the subset is calculated. In one example, Algorithm 1 iterates until the log probability over the entire training set is as high as possible when it is trained on the selected subset of data S. The log probability is the log of value of the sample probability function evaluated at a given data point. The subset S is selected such that the log probability over the entire training set is as high as possible when it is trained on the selected subset S as is noted in Equation 5 below:
S=arg maxθ˜P(θ|S)└Σi=1Nlog(Pi|ci;θ[I(ϵi)])┘  Equation (5)Such that S⊆D, |S|=M,where P(θ|S) indicates the distribution of parameters trained with the selected subset of data S. For example, for a MANIAC tree, θ represents the tree structure. Therefore, when S reaches size M (Step 6 of Algorithm 1), Algorithm 1 can identify a final subset that includes the pairs with the largest likelihood of prediction value over the entire residual data. The final subset that has the largest likelihood of prediction value over the entire residual data will provide a low cross entropy value. The smaller the cross entropy is, the better compression can be obtained.

With continued reference to Algorithm 1, if the size of the subset of data is less than a budget M (Step 2 of Algorithm 1), then the subset of data is updated with the context-residual pair having the lowest likelihood of prediction value (Step 5 of Algorithm 1). The actively-learned context modeling module311trains or builds the final context model using the final updated subset of data S if the size of the subset of data is more than the budget M (Steps 6 and 7 of Algorithm 1).

In some examples, the subset is updated with only one context-residual pair (corresponding to the pixel having the largest likelihood of prediction). In another example, the subset is updated with only a batch size of context-residual pairs (corresponding to the pixels having the largest likelihood of prediction). In some embodiments, the likelihood of prediction value is calculated for each pixel absent or not in the subset. In other embodiments, the likelihood of prediction is calculated for a portion of pixels absent or not in the subset. Algorithm 2 provides a method to avoid determining values for each pixel and instead uses entropy values of leaf nodes to update the subset.

Algorithm 2

An exemplary actively-learned context modeling module of an image compression system actively-learned context modeling method using can implement Algorithm 2 in a system that uses FLIF image compression. It should be understood that Algorithm 2 can be implemented in any other system as well. Algorithm 2 allows the actively-learned context modeling method to not evaluate the likelihood of prediction at each iteration as seen in Algorithm 1. Instead, in Algorithm 2, the actively-learned context modeling method selects a batch B of data points at each step to add to the subset. In order to avoid the situation that the selected batch B is closely related, the actively-learned context modeling method can force the selected batch B to be diverse at each step so that the elements in the selected batch B have low redundancy. The actively-learned context modeling method iterates until the subset is of a certain size. Algorithm 2 can be seen in Table 2:

TABLE 2Algorithm 2: Active context modeling for FLIFRequire: training set= {(ci, ϵi)}i=1N, MANIAC tree pθ,selection budget M, batch size B1:select a subsetat random, || < M2:while || < M do3:train a MANIAC tree pθusing4:∀(cj, ϵj) ∉, navigate (cj, ϵj) through pθ5:estimate the entropy Hkfor each leaf node k6:for each leaf node k do7:sample⁢Bk=B·Hk∑kHk⁢data⁢points⁢uniformly8:updateto include newly sampled data9:end for10:end while11:train the final context model pθusing

Algorithm 2 uses a training set D for a MANIAC tree. It should be understood that Algorithm 2 can be used in any context model. In one embodiment of Algorithm 2, a budget M and a batch size B is either provided or calculated or predetermined. M is the total size of the final subset of data. For example, if the image includes 1 million pixels, then the budget M could be 1% sampling of a 1 million pixel image. That is 1% of 1 Million=10 k. Therefore, in this example, the final subset size M of the 1 Million pixel image is 10 k. It should be understood that any other percentage can be used or any method can be used to determine or assign to the final subset. B is the batch size of context-residual pairs that are selected in each iteration and added to the subset. For example, if the batch size B is 5, then in each iteration 5 context-residuals pairs are selected and added to the subset. In this algorithm the size B should be smaller than the final subset size M. For example, if the batch size B is 1 k context-residual pairs that means 1K context-residual pairs are added in every iteration. Therefore, it might take 10 iterations to reach a size of 10 k (final subset size M). Initially, a subset of data S is selected, for example at random (Step 1 of Algorithm 2). It should be understood that the subset of data S can be selected using other methods as well, such as picking certain type of data or using an algorithm to select the subset of data or the like or a combination. The size of initial subset of data S selected in step 1 of Algorithm 2 is less than the budget M.

The value M is also the threshold subset of data size and can be a number of pixels allowed for training or a number of data allowed for training (an allowable data value). For example, M can be a predetermined amount of pixels from the training set or training data. In another example, M can be a number based on how fast the process should be. The subset of data S is iteratively grown until M is reached (Step 2 of Algorithm 2). The tree is grown or the context model is trained or the context model is built until reach maximum allowed target M. B refers to the size of the updated subset of data.

Continuing with Algorithm 2, while the size of the subset of data S is less than the M value, the initial context model is built or trained using the initial subset of data (Step 3 of Algorithm 2). In step 3 of Algorithm 2, an initial context model {circumflex over (θ)}=(ƒ, S) is given, where {circumflex over (θ)} is the context model and(ƒ, S) refers to obtaining the context model using subset S.

Continuing with Algorithm 2, steps 5-8 are used to identify data from the rest of the training set to add to the subset in each iteration. In Step 5 of Algorithm 2, an entropy for each leaf node can be estimated or determined or calculated. An entropy value Hk for residuals in each leaf node k is determined (Step 5 of Algorithm 2). In one example, if the weighted entropy Hn of the leaf node is low that can indicate that the residuals in the leaf node are similar or the residuals are concentrated.

After determining the entropy value of each leaf node, steps 6-8 will sample only a certain number of context-residual pairs from each leaf node based on the entropy value. For example, if batch size is 5, that means steps 6-8 will select 5 context-residual pairs. However, a higher percentage of the batch Bkwill come from a leaf node with a higher entropy value (Hk/total Hk). As such, the sampled batch from each leaf node Bk is based on weighed entropy of the leaf node as noted in Step 7 of Algorithm 2.

Therefore, Bkdata points will be selected from each leaf node. In one example, Algorithm 2 allows the selected batch of data to cover as many modes of the data distribution as possible (e.g., selecting samples from each leaf node so that all leaf nodes are covered). As such, the number of data being selected from each leaf node k is proportional to the entropy of the corresponding leaf node. This means that the clusters with higher diversity will have more data sampled. Given the batch size B, each leaf node k is sampled:

Bk=B·Hk∑Hk
(Step 7 of Algorithm 2), where B is the given batch size, Hkis the calculated entropy value for each leaf node, and Bk is the sample size of node k.

Continuing with step 8 of Algorithm 2, the actively-learned context modeling method will update the subset of data with B amount of context-residual pairs that is distributed across all the leaf nodes as noted in steps 6 and 7 of Algorithm 2. Therefore, the subset of data in the leaf node can provide an optimal solution (Step 8 of Algorithm 2).

The steps of sampling the data for each leaf node and updating the new subset of data S to include the sampling is then reiterated for each leaf node until the size of the subset S reaches budget M (Steps 7 and 8 of Algorithm 2).

When the size of the subset of data S is not less than the budget M, the final context model is built using the updated subset of data S (Step 11 of Algorithm 2). The final context model is updated: {circumflex over (θ)}=(ƒ, S) in step 11 of Algorithm 2.

By selectively updating the subset with a certain percentage of data based on the entropy value of the context model over the residuals in each leaf node, a low cross entropy can be obtained over the entire dataset.

Exemplary Implementation of Actively-Learned Context Modeling

Actively-learned context modeling can be implemented in any algorithm. In one example, actively-learned context modeling can be used in a FLIF image compression system. FLIF uses a decision tree based context model called MANIAC tree. The tree can be interpreted as a clustering model where candidate data points are navigated through the tree into one of the leaf nodes. Given the clustering results, a batch of candidate data can be selected and added for training at the next step by updating the subset of data which will be used in next iteration's training. In one example, a batch of data can be selected that are distributed across all leaf nodes. The selected batch can cover as many modes of the data distribution as possible. The number of data being selected from each leaf node can be proportional to the entropy of the corresponding leaf node.

In one example, actively-learned context modeling can be used in compression algorithms for single images or for burst compressions.FIG.8illustrates an exemplary implementation for a burst compression using actively-learned context modeling in accordance with one embodiment of the present disclosure. For example, in a system that uses an algorithm such as FLIF for burst compression, the decision tree808a,808b,808ccan be interpreted as a clustering model. In a burst compression, trained tree from the previous frame802a,802b,802cmay be available. Since the frames802a,802b,802ccontain similar content, the tree808a,808b,808cfrom the previous frame802a,802b,802ccan be used as a proxy to cluster the current frame802a,802b,802c. A batch of training pairs can be selected from the leaf nodes of the tree808a,808b, and808c. In one example, for the first frame802a, the training pairs can be selected uniformly804aat random since no previous frame exists. For the following frames802band802c, the training pairs can be selected using active sampling804b,804c. In one example, after performing one iteration using actively-learned context modeling, a context model tree808ais built and the entropy for each leaf node is calculated. For uniform sampling808a, the training pair can be selected randomly from the training set. For active sampling808b,808b, the next iteration of the training pair comes from prior information of the training pairs. For example, in active sampling808b,808c, the next iteration of training pair of the tree can include the ratio of the entropy determined for the leaf node. For example, for the initial training pair, an initial training pair is selected at random and the entropy of each leaf node is calculated. Therefore, in active sampling808b,808c, the pixels can be actively selected based on the knowledge of the tree built using prior training pairs.

Exemplary Operating Environment

Having described implementations of the present disclosure, an exemplary operating environment in which embodiments of the present technology may be implemented is described below in order to provide a general context for various aspects of the present disclosure. Referring toFIG.9, an exemplary operating environment for implementing embodiments of the present technology is shown and designated generally as computing device900. Computing device900is but one example of a suitable computing environment and is not intended to suggest any limitation as to the scope of use or functionality of the technology described herein. Neither should the computing device900be interpreted as having any dependency or requirement relating to any one or combination of components illustrated.

The technology may be described in the general context of computer code or machine-useable instructions, including computer-executable instructions such as program modules, being executed by a computer or other machine, such as a personal data assistant or other handheld device. Generally, program modules including routines, programs, objects, components, data structures, etc., refer to code that perform particular tasks or implement particular abstract data types. The technology described herein may be practiced in a variety of system configurations, including hand-held devices, consumer electronics, general-purpose computers, more specialty computing devices, etc. The technology described herein may also be practiced in distributed computing environments where tasks are performed by remote-processing devices that are linked through a communications network.

With reference toFIG.9, computing device900includes bus910that directly or indirectly couples the following devices: memory912, one or more processors914, one or more presentation components916, input/output (I/O) ports918, input/output components920, and illustrative power supply922. Bus910represents what may be one or more busses (such as an address bus, data bus, or combination thereof). Although the various blocks ofFIG.9are shown with lines for the sake of clarity, in reality, delineating various components is not so clear, and metaphorically, the lines would more accurately be grey and fuzzy. For example, one may consider a presentation component such as a display device to be an I/O component. Also, processors have memory. The inventors recognize that such is the nature of the art, and reiterate that the diagram ofFIG.9is merely illustrative of an exemplary computing device that can be used in connection with one or more embodiments of the present disclosure. Distinction is not made between such categories as “workstation,” “server,” “laptop,” “hand-held device,” etc., as all are contemplated within the scope ofFIG.9and reference to “computing device.”

Computing device900typically includes a variety of computer-readable media. Computer-readable media can be any available media that can be accessed by computing device900and includes both volatile and nonvolatile media, removable and non-removable media. By way of example, and not limitation, computer-readable media may comprise computer storage media and communication media. Computer storage media includes both volatile and nonvolatile, removable and non-removable media implemented in any method or technology for storage of information such as computer-readable instructions, data structures, program modules or other data. Computer storage media includes, but is not limited to, RAM, ROM, EEPROM, flash memory or other memory technology, CD-ROM, digital versatile disks (DVD) or other optical disk storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other medium which can be used to store the desired information and which can be accessed by computing device900. Computer storage media does not comprise signals per se. Communication media typically embodies computer-readable instructions, data structures, program modules or other data in a modulated data signal such as a carrier wave or other transport mechanism and includes any information delivery media. The term “modulated data signal” means a signal that has one or more of its characteristics set or changed in such a manner as to encode information in the signal. By way of example, and not limitation, communication media includes wired media such as a wired network or direct-wired connection, and wireless media such as acoustic, RF, infrared and other wireless media. Combinations of any of the above should also be included within the scope of computer-readable media.

Memory912includes computer storage media in the form of volatile and/or nonvolatile memory. The memory may be removable, non-removable, or a combination thereof. Exemplary hardware devices include solid-state memory, hard drives, optical-disc drives, etc. Computing device900includes one or more processors that read data from various entities such as memory912or I/O components920. Presentation component(s)916present data indications to a user or other device. Exemplary presentation components include a display device, speaker, printing component, vibrating component, etc.

I/O ports918allow computing device900to be logically coupled to other devices including I/O components920, some of which may be built in. Illustrative components include a microphone, joystick, game pad, satellite dish, scanner, printer, wireless device, etc. The I/O components920may provide a natural user interface (NUI) that processes air gestures, voice, or other physiological inputs generated by a user. In some instance, inputs may be transmitted to an appropriate network element for further processing. A NUI may implement any combination of speech recognition, touch and stylus recognition, facial recognition, biometric recognition, gesture recognition both on screen and adjacent to the screen, air gestures, head and eye-tracking, and touch recognition associated with displays on the computing device900. The computing device900may be equipped with depth cameras, such as, stereoscopic camera systems, infrared camera systems, RGB camera systems, and combinations of these for gesture detection and recognition. Additionally, the computing device900may be equipped with accelerometers or gyroscopes that enable detection of motion.

Aspects of the present technology have been described in relation to particular embodiments, which are intended in all respects to be illustrative rather than restrictive. Alternative embodiments will become apparent to those of ordinary skill in the art to which the present technology pertains without departing from its scope.

Having identified various components utilized herein, it should be understood that any number of components and arrangements may be employed to achieve the desired functionality within the scope of the present disclosure. For example, the components in the embodiments depicted in the figures are shown with lines for the sake of conceptual clarity. Other arrangements of these and other components may also be implemented. For example, although some components are depicted as single components, many of the elements described herein may be implemented as discrete or distributed components or in conjunction with other components, and in any suitable combination and location. Some elements may be omitted altogether. Moreover, various functions described herein as being performed by one or more entities may be carried out by hardware, firmware, and/or software, as described below. For instance, various functions may be carried out by a processor executing instructions stored in memory. As such, other arrangements and elements (e.g., machines, interfaces, functions, orders, and groupings of functions) can be used in addition to or instead of those shown.

Embodiments described herein may be combined with one or more of the specifically described alternatives. In particular, an embodiment that is claimed may contain a reference, in the alternative, to more than one other embodiment. The embodiment that is claimed may specify a further limitation of the subject matter claimed.

The subject matter of embodiments of the technology is described with specificity herein to meet statutory requirements. However, the description itself is not intended to limit the scope of this patent. Rather, the inventors have contemplated that the claimed subject matter might also be embodied in other ways, to include different steps or combinations of steps similar to the ones described in this document, in conjunction with other present or future technologies. Moreover, although the terms “step” and/or “block” may be used herein to connote different elements of methods employed, the terms should not be interpreted as implying any particular order among or between various steps herein disclosed unless and except when the order of individual steps is explicitly described.

For purposes of this disclosure, the word “including” has the same broad meaning as the word “comprising,” and the word “accessing” comprises “receiving,” “referencing,” or “retrieving.” Further, the word “communicating” has the same broad meaning as the word “receiving,” or “transmitting” facilitated by software or hardware-based buses, receivers, or transmitters using communication media described herein. In addition, words such as “a” and “an,” unless otherwise indicated to the contrary, include the plural as well as the singular. Thus, for example, the constraint of “a feature” is satisfied where one or more features are present. Also, the term “or” includes the conjunctive, the disjunctive, and both (a or b thus includes either a or b, as well as a and b).

For purposes of a detailed discussion above, embodiments of the present disclosure are described with reference to a distributed computing environment; however, the distributed computing environment depicted herein is merely exemplary. Components can be configured for performing certain embodiments, where the term “configured for” can refer to “programmed to” perform particular tasks or implement particular abstract data types using code. Further, while embodiments of the present disclosure may generally refer to the technical solution environment and the schematics described herein, it is understood that the techniques described may be extended to other implementation contexts.

From the foregoing, it will be seen that this technology is one well adapted to attain all the ends and objects set forth above, together with other advantages which are obvious and inherent to the system and method. It will be understood that certain features and subcombinations are of utility and may be employed without reference to other features and subcombinations. This is contemplated by and is within the scope of the claims.