Patent ID: 12248794

DETAILED DESCRIPTION

In the following description, reference is made to the accompanying drawings that form a part hereof, and in which is shown by way of illustration specific embodiments which may be practiced. These embodiments are described in sufficient detail to enable those skilled in the art to practice the invention, and it is to be understood that other embodiments may be utilized and that structural, logical and electrical changes may be made without departing from the scope of the present invention. The following description of example embodiments is, therefore, not to be taken in a limited sense, and the scope of the present invention is defined by the appended claims.

The functions or algorithms described herein may be implemented in software in one embodiment. The software may consist of computer executable instructions stored on computer readable media or computer readable storage device such as one or more non-transitory memories or other type of hardware-based storage devices, either local or networked. Further, such functions correspond to modules, which may be software, hardware, firmware or any combination thereof. Multiple functions may be performed in one or more modules as desired, and the embodiments described are merely examples. The software may be executed on a digital signal processor, ASIC, microprocessor, or other type of processor operating on a computer system, such as a personal computer, server or other computer system, turning such computer system into a specifically programmed machine.

The functionality can be configured to perform an operation using, for instance, software, hardware, firmware, or the like. For example, the phrase “configured to” can refer to a logic circuit structure of a hardware element that is to implement the associated functionality. The phrase “configured to” can also refer to a logic circuit structure of a hardware element that is to implement the coding design of associated functionality of firmware or software. The term “module” refers to a structural element that can be implemented using any suitable hardware (e.g., a processor, among others), software (e.g., an application, among others), firmware, or any combination of hardware, software, and firmware. The term, “logic” encompasses any functionality for performing a task. For instance, each operation illustrated in the flowcharts corresponds to logic for performing that operation. An operation can be performed using, software, hardware, firmware, or the like. The terms, “component,” “system,” and the like may refer to computer-related entities, hardware, and software in execution, firmware, or combination thereof. A component may be a process running on a processor, an object, an executable, a program, a function, a subroutine, a computer, or a combination of software and hardware. The term, “processor,” may refer to a hardware component, such as a processing unit of a computer system.

Furthermore, the claimed subject matter may be implemented as a method, apparatus, or article of manufacture using standard programming and engineering techniques to produce software, firmware, hardware, or any combination thereof to control a computing device to implement the disclosed subject matter. The term, “article of manufacture,” as used herein is intended to encompass a computer program accessible from any computer-readable storage device or media. Computer-readable storage media can include, but are not limited to, magnetic storage devices, e.g., hard disk, floppy disk, magnetic strips, optical disk, compact disk (CD), digital versatile disk (DVD), smart cards, flash memory devices, among others. In contrast, computer-readable media, i.e., not storage media, may additionally include communication media such as transmission media for wireless signals and the like.

The ability for machine learning models to understand and operate user interfaces (UIs) is an important task. A bot that can find a UI element for performing a task, such as an interactive icon or hypertext link, by an oral description can be very useful. A model that can predict the expected output of clicking a button or link can help with page navigation. To successfully operate a UI, models need to understand the user task and intent, and how to perform the task in the given UI.

User interface representation models have been used to allow systems to understand and operate user interfaces on a user's behalf. Training vision and language models requires high-quality paired visio-linguistic datasets. Common approaches to collect training datasets include (i) app crawling with human labelling and (ii) use of unlabeled data consisting of UI screens and associated metadata. While the former is hard to scale, in case of the latter. UI metadata, such as accessibility labels included in alternative text and structural representations of a screen's UI elements (referred to as DOM tree in webpages and View Hierarchy in Android devices) is often missing, partially defined, or not accessible for security reasons.

FIG.1is a simplified block diagram illustrating a system100for creating a machine learning model to understand user interfaces. System100includes an improved vision and language pre-trained model110that is designed to handle the unique features of user interfaces including their text richness and context sensitivity. In various examples, textual instructions relative to some user interface and sample screenshots of the user interface itself may be obtained by executing training data queries115against a source, such as cloud or Internet-based storage120. Storage120includes user interface images and associated instructions intended to inform users regarding how to use user interfaces. The images and associated instructions may be arranged as pairs, with each pair including an image and at least one instruction corresponding to the image.

The query returns images and text that are used as training data to train the model110to represent user interfaces. Instructions and images of user interfaces represent a new type of training data that can be obtained from instruction manuals, tutorials, and how-to guides that abound on the Internet (such as in technical support and how-to websites), exist for many different applications and platforms, and are easy to crawl.

In addition to textual instructions, for clarity, such training data usually includes visual information. The visual and textual descriptions are used to learn visually grounded textual representations of UI screens and their elements by model110.

A detector125performs functions of extracting information from the training data images. A two-stage object detection network may be used to generate region proposals from an image using standard computer vision techniques and utilize an image classifier to determine a class of an object in each region proposal. A ResNet image classifier fine-tuned on a dataset of mobile device user interface elements may be used. Alternatively, a machine learning object detection model trained large numbers of user interface images may be used. Text elements may also be detected and recognized. The text elements detected in the image data may be combined with text input, such as image captions or instructions associated with the image to form the input text. The input text may be tokenized.

Model110may include a vision component130and a language component135. The image data, including the detected and classified elements and extracted features, is provided to the vision component130. Similarly, the text data is provided to the language component135. The vision and language model110is trained in a self-supervised manner utilizing masking-based techniques145. Masking techniques basically mask portions of the input in each respective component and have the model make a prediction of the masked portion. Since the masked portion is known, referred to as a ground truth, the accuracy of the prediction is easy to determine and to provide feedback to improve the model.

Image text alignment150may be performed together with masking to train the model110to predict a score as output155. A binary cross-entropy loss may be applied to modify model weights to improve performance of the model. The model may be trained using the mask-based techniques, image text alignment, and optionally parts of speech techniques concurrently. Other pre-training tasks may be considered such as text-based image retrieval, image and text entailment, next instruction prediction, etc.

FIG.2is a block flow diagram illustrating training generally at200a combined vision and language (VL) transformer encoder model that includes a VL model210for understanding user interfaces. VL model210has a two-stream architecture, one for a vision component215and one for a text component220that each consist of a series of transformer blocks and co-attentional transformer layers for cross-model representation learning.

An example training data for the VL model210includes an image of a user interface screen222and a text caption223consisting of instructions on how to use the interface screen222, which were extracted from a user manual. The training data may be accessed from previously collected information or may be accessed via training data queries as described above. Text223is tokenized and provided to text component220as shown beginning at token225and extending to [SEP] token226. A [DET] token227indicates the beginning of the tokens for text223.

In one example, the [DET] token227indicates a concatenation of the text223with text derived from the image222by a user interface element detector230. Detector230provides several functions, including region proposal generation at233which detects coordinates of user interface elements present in image222, fusion (merging) and alignment of the detected user interface elements at231, classification of user interface elements at232, and text detection and recognition at235. Fusion, or merging and alignment of user interface elements may be performed by combining non-text elements and text element predictions through simple geometric rules. Adjacent elements that are horizontally or vertically aligned may be merged.

The recognized text from detector230is provided at234as tokens to the language component220. A [CLS] token235is used to indicate the beginning of text with a further [DET] token236used to indicate the concatenation or addition of text, such as text token237which represents the word “device” derived from a bounded user interface element238that includes the text “device settings.”

Image222is shown again at240with bounding boxes of the textual and visual elements detected in the image which include text indicated at238“Device Settings,”241“Leave,”242“Call health,”243“Hold,”244“Transfer,”245“Consult then transfer,”246“Park Call.” and247“Turn on live captions.” Each of these words may also be provided to the language component220as tokens as represented by “ . . . ” indications between the tokens shown as input to the language component220.

Four of the bounding boxes,251,252,253, and254contain only visual content, and may be icons that are selectable by a user. Image region features are provided as input to the vision component215with an [IMG] token255indicating the beginning of the image tokens. The same reference numbers are used for the bounding boxes in image240and for the image feature representations. Image240also includes other icons associated with the text in some of the bounding boxes and may also be input as image tokens. Each of the image tokens provided to vision component215also includes a classification label and a probability distribution generated by detector230.

VL model210is a transformer-based bidirectional language and vision model. VL model210can transfer learned features to multiple NLP and vision tasks. On a high level, VL model210takes in the embedding of word tokens and images and processes them through a multi-layer bidirectional transformer. Language component220is a transformer-based bidirectional language model.

Given a text input, word tokens w1, . . . , wTand an image I, image tokens, represented as a set of region features v1, . . . , vM, VL model210outputs final vector representations h0, . . . , hvMfor vision information and hw0, . . . , hwTfor text information beginning at256and257respectively. Text is encoded as a sequence of word tokens prepended with the [CLS] token. Sentence pairs are packed together into a single input sequence and distinguished using the [SEP] token. For a given token, the input representation is a sum of a token, segment, and position embedding. An image is encoded by extracting bounding boxes of its regions of interest and corresponding visual features using the classifier232such as an object detection model Faster R-CNN with ResNet-101 pre-trained on Visual Genome. Spatial location and fraction of the image area covered are also encoded.

The VL model210is then pre-trained with multiple concurrently performed pre-training tasks including masked multi-modal modeling and multi-modal alignment prediction. The vision component of the VL model210may be trained using masked region modeling with DL divergence (MRM-KL)260by masking one or more image tokens of the input to the vision model210as shown by the masking technique261. In one example, image regions are sampled, and visual features are masked with a probability of 15% in one example, or other selected probability which may be determined empirically. Zeros may be used to replace the actual values to perform the masking. The VL model210is tasked with reconstructing the masked regions given the remaining regions and the input words. However, the model is unlikely to be able to reconstruct exact image features. The model predicts a distribution over semantic classes for the corresponding image region and minimizes the KL divergence between the original class probabilities of the unmasked region and the predicted class probabilities. Cross-entropy loss may be used to adjust weights of the model for such minimization.

Similarly, language component220may also be trained using masked language modeling with detected text (MLM-DT)263by masking one or more text tokens of the input to the language component220as shown by the masking technique264. In one example, input words are randomly masked out with probability of 15% in one example. Other probabilities may be used in further examples. The masked input words are replaced with the special token [MASK]264which may be all zeros. A key difference from current VL models is that in VL model210any text detected in the visual input is appended to the text input, and its words may also be masked. This further teaches the VL model210to align the detected text with the visual content, especially the image regions containing the detected text. A standard cross-entropy loss may be used to learn the weights of the model.

The above masked-based training of the vision and language model210is performed separately for text and images. Masking only one modality at a time while keeping the other modality intact improves performance. Since the ground truth (the actual data that is masked) is known, training may be self-supervised, removing the need for human labeled data in the training of the models.

Image-text alignment (ITA)265may also be performed concurrently with the mask-based training as part of the pre-training of VL model210. ITA is useful for learning representations for downstream cross-modal tasks. It is a binary classification task. The dot product266of the hidden representations for the image stream and text stream are calculated. The corresponding cross-modal representation is fed into a feed-forward layer to predict a score267between 0 and 1. During training, a positive or negative pair is sampled at each step. In one example, the negative pair is created by sampling text samples that are within a distance of, for example, 0.5 in cosine-similarity with the ground-truth text using a sentence-vector similarity. Due to prevalence of common technical terms, the task captions tend to be semantically very similar. A binary cross-entropy loss is applied for training.

A challenge with text embeddings related to user interfaces is that text inputs inevitably exhibit a high recurrence of technical terms such as “click”, “tap”. “type”, “button”, etc. Many of these terms have high similarity using out-of-the-box language models such as BERT and RoBERTa, and some terms (e.g., “menu”, “bar”, etc.) have an entirely different meaning in a general domain. To help the VL model210learn the syntax of UI instructions, a weakly-supervised part-of-speech (POS) tagging may be performed as an additional pre-training task concurrently with the other pre-training tasks.

In POS Tagging (POS) the VL model210is trained to predict noun271and verb272spans present in the text. Learning to recognize parts of speech and how they relate to each other is useful for the VL model210to learn the “foreign” UI language and understand the meaning of highly recurrent technical terms. This is helpful for applications where text plays a predominant role in the prediction such as user interface entity recognition. The POS features are encoded in the learned representations during pre-training. In one example, SpaCy's POS tagger's predictions may be used as weak-supervision labels for learning the task. A BIO scheme may also be used. Each token representation may be classified as the beginning (B) or the inside (I) of a noun or verb entity, or as other (O) at273, i.e., a non-entity token. Cross-entropy loss may be used.

In summary, the total pre-training loss L with scalar coefficients α, β, γ, δ∈(0; 1) is given by:
=α·MLM-DT+β·MRM-KL+γ·ITA+δ·POS
wherexis the loss for the pre-training task x.

The collection of training data may be performed in one example by generating search queries using UI element names and sample instructions extracted from manuals, such as Microsoft®- and Android® manuals. An image search may be performed using one of many image-based search engines such as Google® image search for each query. Some top results, for example 50 or 100, are inspected to collect thousands of URLs of technical webpages. Each page may be crawled. General heuristics are used to extract images along with their alt-text and preceding texts.

By selecting the source webpages, this process ensures the collected images are indeed of UI elements and screenshots.

For each UI image some number of captions may be generated (e.g., 1 to 5 captions per image). The number of captions generated depends on the length and structure of the preceding text. The image's alt-text may be appended to the generated caption. The captions are referred to as task captions because rather than describing the appearance of an image such as is commonly done with alt-text, the captions describe functionality of UI elements. This textual context often translates into spatial context because the instructions to accomplish a task usually involve nearby UI elements.

FIG.3Ais an example of an instruction manual300with generated image-caption pairs310. The instruction “Select Video call or Audio call to start a call” is used as the captions of both the “Video call” and “Audio call” icons. The instruction defines a logical relationship between the “Video call” and “Audio call” functions, but also a spatial relationship, i.e., the two icons appear next to each other in the UI.

In one example 198 k UI image and caption pairs, split across 4 datasets: Mixed (154,196 pairs, containing UI data for many different apps, may be obtained using a generic crawler), MS (34,523 pairs, covering Microsoft apps), Teams® (8,778 pairs), and Zoom® (472 pairs). The crawled UI images may be used to train current UI understanding models.

The crawled training data may be extremely varied in terms of applications (Excel, Gmail, Chrome, Photos, etc.), platforms (mobile & desktop), number of UI components (from a single icon to a complete desktop), text density (Word documents, photos, or Excel spreadsheets), pixel density, and quality (e.g., screenshots may not be pristine or may have been annotated.)FIG.3Bis an example manual illustrating an annotated user interface at320.

The trained VL model210may be subsequently fine-tuned (trained with smaller but specialized sets of data) to perform specific tasks with higher accuracy. Fine tuning may involve small quantities of manually labeled data such as real user commands in natural language and corresponding sequences of UI actions. The process of fine-tuning may be specific to a target task or to a class of target tasks.

In one example of the fine-tuning task, the trained VL model210may be further trained to predict whether a user interface action can be recognized and performed. One example is a natural language instruction such as “Select video call or audio call to start a call”. This type of task has practical use in screen readers to locate UI components by their name/functionality in the absence of accessibility labels and alt-tests.

In a further example, the trained VL model210may be fine-tuned for task-based UI image retrieval. In one example, a goal may be to identify a UI image from a pool of 50 images given a description of an associated task.

In yet a further example, the trained VL model210may be fine-tuned to select a UI element referenced by an expression in natural language from a set of elements detected on a screen. This task is relevant to voice-guided navigation agents, where users can issue commands like “click home at the top” and the navigation system identifies the corresponding “home” icons and clicks on it.

Still a further example includes given a UI image and an instruction as input, the trained VL model210may be fine-tuned to extract spans of UI action verbs, UI action objects, and input parameters. The task is relevant to task completion systems to translate commands into executable procedures consisting of low-level UI interactions. A key challenge in this task is that not all instances of a word are of the same entity type. For example, the word “Alexa” may represent an input, but “Alexa app” may not represent an input. “Search” may often represent a UI action object, (e.g., a search box) but in other instances may be a UI action verb.

The pre-training of the VL model210does not rely on human-labeled data. Publicly available web data can be crawled at scale and for many different types of applications for training. This approach can alleviate the risk of training UI models that work only for certain types of applications or platforms.

Less popular applications may have less or lower-quality documentation online. One way to partly address lack of online documentation is to extend UI data collection beyond technical support websites and how-to guides to other documentation types (tutorials, bug reports, etc.) and formats (videos).

One application of the VL model includes screen readers for visually impaired users. As accessibility labels are often missing or incomplete, the use of the VL model could provide visually impaired users access to a much wider range of applications. In this regard, guaranteeing an unbiased coverage of applications and platforms is even more important.

Another potential use case is task automation, which has societal and security implications. What if an agent clicks the wrong button? Is there an “undo” option or other recourse? A significant frontier for intelligent task completion systems is architecting them such that they can increase human productivity yet remain amenable to human review.

FIG.4is a flowchart illustrating a computer implemented method400for learning representation models of user interfaces. Method400begins at operation410by accessing training data including images, alternative text associated with an image, and texts describing the functionality of a user interface image. Each image is paired at operation420with a text caption consisting of at least one proximately located text description and the image's associated alternative text, if any. The text captions and images may include position embedding. An image region detector and image classifier trained on user interface elements may be executed on the images to detect bounding boxes of UI elements and generate classification labels and region features for each one. In one example, a text caption is the set of textual instructions closest (proximate) to the respective image in an instruction manual or how to guide. In another example, the text caption is the alternative text associated with the image. In another example, the text caption is the concatenation of the textual instructions with the alternative text.

The vision and language model is pre-trained at operation430in a self-supervised manner using the above described pre-training tasks on respective image features and tokenized text captions. These tasks may utilize techniques include language masking, image region masking, image-text alignment, and POS tagging.

Image-text alignment of the VL model enables the VL model to map user interface elements to associated functions. Image-text alignment is performed to predict a score for the pairs of user interface images and corresponding descriptions. The image-text alignment of the vision and language model may be performed using samples of positive and negative pairs applying binary cross-entropy loss. The pre-trained VL model is a vision language transformer encoder.

Pre-training of the language model at operation430optionally includes using part of speech tagging. Part of speech tagging includes training the language component to predict noun and verb spans using a cross-entropy loss.

FIG.5is a computer implemented method500of including detecting and recognizing text from images to be included in text captions to train the VL model. Operation500begins by identifying text in the images at operation510. The text is recognized at operation520to generate recognized text. At operation530, the recognized text is added to the text captions that are used to train the VL model. The recognized text and the proximately located text captions provided to the VL model are tokenized with respective sets of text tokens separated by a [DET] token.

FIG.6is a flowchart illustrating a computer implemented method600of using a VL model to identify user interface elements corresponding to actions specified in natural language to be performed in a user interface. Method600begins by providing two inputs to the VL that was trained in method400. A first input, provided at operation610, relates to an action to be performed. In one example, the first input may be a user command in natural language that may be typed or spoken and converted to text. At operation615, a second input is provided and consists of an image of the user interface to be used. The second input may be a screenshot of the user interface of an application with which the user is currently interacting. At operation620, the model transforms, encodes, and classifies the inputs to generate an output identifying a candidate user interface element for performing the action.

FIG.7is a flowchart illustrating a further computer implemented method700for learning representation models of user interfaces. Method700begins at operation710by accessing training data including pairs of user interface images, image's alternative text, and text-based user interface descriptions. Pairs of user interface images and corresponding text captions descriptions derived from the training data are generated at operation720. At operation730, the text descriptions are encoded as language tokens and passed to the language component of the vision and language model. At operation740, the images are encoded as bounding boxes and associated image region features and passed to the vision component. Self-supervised pre-training of the language model is performed at operation750concurrently using the above-described pre-training tasks including using masked language modeling with detected text. Self-supervised training of the VL model is performed using masked region modeling with KL divergence. Image-text alignment is performed by applying a binary cross-entropy loss to enable the VL model to map user interface elements to associated functionality. Further concurrent pre-training of the VL model is performed using part of speech tagging. Part of speech tagging may include training the language component of the VL model to predict noun and verb spans using a cross-entropy loss.

FIG.8is a block schematic diagram of a computer system800to implement the VL model210for representing user interfaces and for performing methods and algorithms according to example embodiments. All components need not be used in various embodiments.

One example computing device in the form of a computer800may include a processing unit802, memory803, removable storage810, and non-removable storage812. Although the example computing device is illustrated and described as computer800, the computing device may be in different forms in different embodiments. For example, the computing device may instead be a smartphone, a tablet, smartwatch, smart storage device (SSD), or other computing device including the same or similar elements as illustrated and described with regard toFIG.8. Devices, such as smartphones, tablets, and smartwatches, are generally collectively referred to as mobile devices or user equipment.

Although the various data storage elements are illustrated as part of the computer800, the storage may also or alternatively include cloud-based storage accessible via a network, such as the Internet or server-based storage. Note also that an SSD may include a processor on which the parser may be run, allowing transfer of parsed, filtered data through I/O channels between the SSD and main memory.

Memory803may include volatile memory814and non-volatile memory808. Computer800may include—or have access to a computing environment that includes—a variety of computer-readable media, such as volatile memory814and non-volatile memory808, removable storage810and non-removable storage812. Computer storage includes random access memory (RAM), read only memory (ROM), erasable programmable read-only memory (EPROM) or electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, compact disc read-only memory (CD ROM), Digital Versatile Disks (DVD) or other optical disk storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other medium capable of storing computer-readable instructions.

Computer800may include or have access to a computing environment that includes input interface806, output interface804, and a communication interface816. Output interface804may include a display device, such as a touchscreen, that also may serve as an input device. The input interface806may include one or more of a touchscreen, touchpad, mouse, keyboard, camera, one or more device-specific buttons, one or more sensors integrated within or coupled via wired or wireless data connections to the computer800, and other input devices. The computer may operate in a networked environment using a communication connection to connect to one or more remote computers, such as database servers. The remote computer may include a personal computer (PC), server, router, network PC, a peer device or other common data flow network switch, or the like. The communication connection may include a Local Area Network (LAN), a Wide Area Network (WAN), cellular, Wi-Fi, Bluetooth. or other networks. According to one embodiment, the various components of computer800are connected with a system bus820.

Computer-readable instructions stored on a computer-readable medium are executable by the processing unit802of the computer800, such as a program818. The program818in some embodiments comprises software to implement one or more methods described herein. A hard drive, CD-ROM, and RAM are some examples of articles including a non-transitory computer-readable medium such as a storage device. The terms computer-readable medium, machine readable medium, and storage device do not include carrier waves or signals to the extent carrier waves and signals are deemed too transitory. Storage can also include networked storage, such as a storage area network (SAN). Computer program818along with the workspace manager822may be used to cause processing unit802to perform one or more methods or algorithms described herein.

Examples

1. A computer implemented method includes accessing training data that includes images, associated alternative text, and proximately located text providing instructions describing a user interface, pairing each image with text captions derived from the proximately located text and image's alternative text, training a vision and language model in a self-supervised manner using language masking, region masking, and image-text alignment techniques on respective image region features and tokenized text captions, and performing fine-tuning of the vision and language model to obtain a specialized model representing user interface elements and associated functions.2. The method of example 1 wherein the language masking, region masking, and image-text alignment techniques are performed concurrently.3. The method of any of examples 1-2 wherein training the vision and language model further includes concurrently performing part-of-speech tagging wherein part of speech tagging comprises training a language component of the vision and language model to predict noun and verb spans using a cross-entropy.4. The method of any of examples 1-3 and further including identifying text in the images, recognizing the text to generate recognized text, and adding the recognized text to the text input used by the language component to train the vision and language model.5. The method of example 4 wherein the recognized text and the text captions provided to the language component are tokenized with respective sets of text tokens separated by a [DET] token.6. The method of any of examples 1-5 wherein the text captions and images include position embedding.7. The method of any of examples 1-6 and further including executing an image region detector and image classifier trained on user interface elements on the images to generate bounding boxes, classifications for the image regions and image region features and including the bounding boxes, classification for the image regions and image region features used by the vision component to train the vision and language model.8. The method of any of examples 1-7 wherein the masking technique used in training the vision and language model includes using masked region modeling with KL divergence for a vision component of the vision and language model.9. The method of any of examples 1-8 wherein the masking technique used in training the vision and language model includes using masked language modeling with detected text for a language component of the vision and language model.10. The method of any of examples 1-9 wherein performing image-text alignment of the vision and language model is performed to predict a score for the pairs of user interface images and corresponding descriptions.11. The method of example 10 wherein performing image-text alignment of the vision and language model is performed using samples of positive and negative pairs defined by cosine-similarity and applying binary cross-entropy loss.12. The method of any of examples 1-11 wherein the vision and language model includes a vision language transformer encoder.13. The method of any of examples 1-12 and further including providing an input related to an action to be performed to the specialized model and a user interface image and receiving an output identifying a candidate user interface element for performing the action in the user interface image.14. A machine-readable storage device has instructions for execution by a processor of a machine to cause the processor to perform operations to perform any of the methods of examples 1-13.15. A device includes a processor and a memory device coupled to the processor and having a program stored thereon for execution by the processor to perform operations to perform any of the methods of examples 1-13.

Although a few embodiments have been described in detail above, other modifications are possible. For example, the logic flows depicted in the figures do not require the particular order shown, or sequential order, to achieve desirable results. Other steps may be provided, or steps may be eliminated, from the described flows, and other components may be added to, or removed from, the described systems. Other embodiments may be within the scope of the following claims.