Patent ID: 12249169

DETAILED DESCRIPTION

Disclosed are various approaches for processing multiple documents in an image. There are many circumstances in which a user may wish to scan multiple documents at once. For example, in the context of mobile banking applications, a user may wish to scan multiple checks at once to make a mobile deposit. In another example, an organization may wish to distinguish between the front and back of an identification card stored in a single image. Additionally, an enterprise may receive images of multiple bills, receipts, or invoices which require separation in order for a billing department to process the documents. However, many systems rely on the user to separate the documents before scanning, to ensure that the documents are flat and easily distinguishable, and that there is no overlap between documents.

In contrast to other approaches, the approaches herein do not rely on any assumptions about the documents in an image. For example, while many scanning and image processing applications assume that there will be four corners to a document or four vertices, the approaches herein make no such assumption. Additionally, the various embodiments of the present invention are domain agnostic, and can be applied to any document type, size, format, etc. Similarly, the approaches herein are language agnostic and do not rely on any assumptions about the type of language which may be in a document. Instead of relying on assumptions based on a high-quality, low-distortion image of a document, the approaches herein can identify and separate documents in complex and distorted images.

As such, various embodiments of the present disclosure are directed to providing systems and methods for processing images of multiple documents and separating the documents into separate files for further processing. A user can scan multiple documents at once, where the documents may be creased, partially folded, overlapping, and/or placed on low-contrast or busy backgrounds, as well as may have differing arrangements or orientations. A segmentation service can correct various aspects of the image before utilizing an object detection model to identify and separate the different documents into separate image files. Then, the segmentation service can send the separate image files to further processing systems.

In the following discussion, a general description of the system and its components is provided, followed by a discussion of the operation of the same. Although the following discussion provides illustrative examples of the operation of various components of the present disclosure, the use of the following illustrative examples does not exclude other implementations that are consistent with the principles disclosed by the following illustrative examples.

FIG.1depicts an example user interface100of a client device103according to various embodiments of the present disclosure. In the example ofFIG.1, the user interface100of the client device103displays an image106of two documents109aand109bon a colorful background113. Each document109has been identified within the image106and has its edges demarcated with a bounding box116. WhileFIG.1depicts space between each bounding box116and the edges of the documents109aand109b, in some embodiments, a bounding box116traces the edges of the document109without any space between. Additionally, each bounding box116can include a score119which can indicate an accuracy, confidence, or other metric about the bounding box116.

In various examples, an image106can contain multiple documents109. As shown inFIG.1, the image106includes two documents109aand109band a background113. While depicted inFIG.1as being captured with a camera application on a client device103, the image106can be obtained using any camera, scanner, or other imaging device. Additionally, the image106can be imported into any application on a client device103, such as a camera application, a scanning application, a social media application, banking application, or other application which can access and process images106.

The documents109aand109bshown in the example ofFIG.1are negotiable instruments (e.g., checks). However, in some examples, other documents109can be identified and separated from an image106. A document109can be representative of a check, a bill, a receipt, an invoice, an identification card, a hand-written note, a list, a letter, or any other type of document. While the documents109aand109bofFIG.1are depicted as having the same orientation and lying flat and separated on the background113, the documents109in an image106can also be partially folded, creased, torn, raised, overlapping, and in different orientations. Additionally, each document109can include text. The text in a document109can include any combination of alphanumerics or symbols from any language. In some embodiments, the documents109can include graphics, pictures, or other internal images in addition to text.

The background113can be representative of any surface or base upon which the documents109appear in the image106. In some embodiments, the background113can represent a reflective mat of a scanner lid, a surface on which the documents109were placed, or other form of background113for the documents109. In some examples, the background113can be white, providing no contrast with documents109and making it difficult to discern the edges of the documents109. In some embodiments, the background113can be colorful, having a one or more different colors. The background113can also be solid or patterned, include miscellaneous designs, text, images, or other noise which can make it difficult to determine the edges or vertices of a document109.

The bounding boxes116can represent the identified boundaries of a document109. In some embodiments, the bounding boxes116can be dashed lines as depicted inFIG.1. In some embodiments, the bounding boxes116can be represented by solid lines, dotted lines, dash-dot lines, transparent lines, or other mark which represents the detected edges of a document109. Additionally, a bounding box116can be formed from any color.

In some embodiments, the score119accompanying a bounding box116can be a percentage, ratio, letter, or other measure indicating an accuracy or confidence of a property on the bounding box116. WhileFIG.1depicts the score119as being located near the top-left corner of the bounding box116, the score119can appear anywhere within or adjacent to the bounding box116. In some embodiments, displaying the bounding boxes116and/or the scores119is an optional feature which can be turned on or off.

With reference toFIGS.2A and2B, shown are various examples of images106of documents109which have been identified. The following examples are intended to be illustrative and non-limiting regarding the types of documents109which can be identified from an image106in various embodiments of the present disclosure.

InFIG.2A, shown is an example of an image106of the front and back of an identification card. The front of the identification card represents a first document109a, while the back of the identification card represents a second document109b. These two documents109aand109bcan be identified by the system described herein and enclosed in bounding boxes116aand116b, where bounding box116acorresponds to document109aand bounding box116bcorresponds to document109b. As shown inFIG.2A, although the boundaries or edges of each document109can be indistinguishable from the background113, the documents109can be distinguished from one another and bounding boxes116applied. The scores119aand119bcan be included with the bounding boxes116aand116bas well.

InFIG.2B, two documents109aand109bare shown where the first document109ais an invoice and the second document109bis a receipt. The example ofFIG.2Bdemonstrates that documents109aand109bin a single image106can be different document types, different shapes, and sizes, have different patterns of text, and overlap at least partially, as shown in the example ofFIG.2B. Additionally, the documents109aand109bcan be folded or have otherwise abnormal edges. Each document109can be identified and enclosed in a respective bounding box116.

Moving now toFIG.3, shown is a network environment300according to various embodiments. The network environment300can include a computing environment303and a client device103, which can be in data communication with each other via a network306.

The network306can include wide area networks (WANs), local area networks (LANs), personal area networks (PANs), or a combination thereof. These networks can include wired or wireless components or a combination thereof. Wired networks can include Ethernet networks, cable networks, fiber optic networks, and telephone networks such as dial-up, digital subscriber line (DSL), and integrated services digital network (ISDN) networks. Wireless networks can include cellular networks, satellite networks, Institute of Electrical and Electronic Engineers (IEEE) 802.11 wireless networks (i.e., WI-FI®), BLUETOOTH® networks, microwave transmission networks, as well as other networks relying on radio broadcasts. The network306can also include a combination of two or more networks306. Examples of networks306can include the Internet, intranets, extranets, virtual private networks (VPNs), and similar networks.

The computing environment303can include one or more computing devices that include a processor, a memory, and/or a network interface. For example, the computing devices can be configured to perform computations on behalf of other computing devices or applications. As another example, such computing devices can host and/or provide content to other computing devices in response to requests for content.

Moreover, the computing environment303can employ a plurality of computing devices that can be arranged in one or more server banks or computer banks or other arrangements. Such computing devices can be located in a single installation or can be distributed among many different geographical locations. For example, the computing environment303can include a plurality of computing devices that together can include a hosted computing resource, a grid computing resource, or any other distributed computing arrangement. In some cases, the computing environment303can correspond to an elastic computing resource where the allotted capacity of processing, network, storage, or other computing-related resources can vary over time.

Various applications or other functionality can be executed in the computing environment303. The components executed on the computing environment303include a segmentation service309, an object detection model313, a processing service316, and other applications, services, processes, systems, engines, or functionality not discussed in detail herein.

The segmentation service309can be executed to determine an orientation of the image106and the one or more documents109within the image106. By identifying sections of text in each document109, the segmentation service309can determine the orientation of each document109with respect to other documents109in the image106as well as with respect to the image106itself. The segmentation service309can be executed to correct various aspects of the image106, such as the orientation of the documents109or the tilt, rotation, clarity, color, shading, or other aspect of the image106and the documents109within. The segmentation service309can use the object detection model313to identify the documents109from within the image106and use the output from the object detection model313to separate the documents109into their own respective separated image files323. In some embodiments, the segmentation service309can determine the document type of each document109and send the separated image files323to a processing service316based at least in part on the document type identified.

The object detection model313can represent a machine learning model which is trained to perform various computer vision tasks. In some embodiments, the object detection model313can be a region-based convolutional neural network (RCNN), a Fast-RCNN, a Faster RCNN, a Mask RCNN, or a Mesh RCNN. The object detection model313can be executed to receive an image106. In some embodiments, the object detection model313can identify regions of interest (ROIs) as candidates for potential documents109within an image106. The ROIs can be identified based at least in part on text, graphics, layout, corners, vertices, edges, or other features of a document109. The object detection model313can be further executed to output bounding boxes116encompassing the various ROIs identified in the image106. In some embodiments, the object detection model313can generate a score119for each bounding box116based at least in part on a confidence in the accuracy of the bounding box116. The object detection model313can further include a filter, such as a rule-based logic filter which can be configured to remove spurious bounding boxes116. Spurious bounding boxes116can be representative of erroneous results or false positives which do not properly identify a document109within an image106. Although depicted separately for clarity, the functionality provided by the object detection model313can be included in the segmentation service309in various embodiments of the present disclosure. In some embodiments, the object detection model313can be a part of the segmentation service309. In some embodiments, the object detection model313can be separate from the segmentation service309, and the segmentation service309can send the modified image106to the object detection model313to identify the documents109within an image106. The segmentation service309can be executed to receive the identified documents109in an image106from the object detection model313.

The processing service316can represent a system or service which can be executed to further process the separated image files323. According to various examples, the processing service316can extract data from the separated image files323and use that data in downstream applications. For example, the processing service316can be banking service which processes scanned checks to deposit into a banking system. In another example, the processing service316can be an optical character recognition (OCR) engine which extracts text from the document and analyzes it for various applications. In some embodiments, the processing service316is a billing service which extracts information from documents such as receipts, invoices, etc. to allocate payments and reimbursements as well as generate new bills. According to various examples, the processing service316is an identification verification service which can be executed to extract data from an identification document, cross check the data against one or more databases, and determine whether the identification document is verified.

Also, various data is stored in a data store319that is accessible to the computing environment303. The data store319can be representative of a plurality of data stores319, which can include relational databases or non-relational databases such as object-oriented databases, hierarchical databases, hash tables or similar key-value data stores, as well as other data storage applications or data structures. Moreover, combinations of these databases, data storage applications, and/or data structures may be used together to provide a single, logical, data store. The data stored in the data store319is associated with the operation of the various applications or functional entities described below. This data can include images106, separated image files323, and potentially other data.

The images106can represent files which contain graphic data, such as photographs or scans of documents109. The images106can be many different file types, such as a Join Photographic Experts Group (JPEG/JPG), bitmap (BMP), Portable Network Graphics (PNG), Tagged Image File Format (TIFF), Photoshop Document (PSD), Portable Document Format (PDF), a RAW format, or other file types. In some embodiments, the images106can be stored as black-and-white, grayscale, or color images106, such as cyan, yellow, magenta, and key (CYMK) or red, green, and blue (RGB) images106.

The separated image files323can represent files which contain graphic data, similar to the images106described above. As such, the separated image files323can be stored as many different file types, such as JPEG/JPG, BMP, PNG, TIFF, PSD, PDF, a RAW format, or other file types. In some embodiments, the separated image files323can be stored as black-and-white, grayscale, or color images, such as CYMK or RGB images. The separated image files323can represent new image files output by the segmentation service309. For example, the separated image files323can be new image files generated from the original image106which was input to the segmentation service309, where each of the new image files represents one of the documents109that was identified in the original image106.

The client device103is representative of a plurality of client devices that can be coupled to the network306. The client device103can include a processor-based system such as a computer system. Such a computer system can be embodied in the form of a personal computer (e.g., a desktop computer, a laptop computer, or similar device), a mobile computing device (e.g., personal digital assistants, cellular telephones, smartphones, web pads, tablet computer systems, music players, portable game consoles, electronic book readers, and similar devices), media playback devices (e.g., media streaming devices, BluRay® players, digital video disc (DVD) players, set-top boxes, and similar devices), a videogame console, or other devices with like capability. The client device103can include one or more displays326, such as liquid crystal displays (LCDs), gas plasma-based flat panel displays, organic light emitting diode (OLED) displays, electrophoretic ink (“E-ink”) displays, projectors, or other types of display devices. In some instances, the display326can be a component of the client device103or can be connected to the client device103through a wired or wireless connection.

The client device103can be configured to execute various applications such as a client application329or other applications. The client application329can be executed in a client device103to access network content served up by the computing environment303or other servers, thereby rendering a user interface100on the display326. To this end, the client application329can include a browser, a dedicated application, or other executable, and the user interface100can include a network page, an application screen, or other user mechanism for obtaining user input. The client device103can be configured to execute applications beyond the client application329such as email applications, social networking applications, word processors, spreadsheets, or other applications.

Next, a general description of the operation of the various components of the network environment300is provided. Although the following description provides an example of the interactions between the various components of the network environment300, other interactions or sequences of interactions are encompassed by the various embodiments of the present disclosure.

To begin, the segmentation service309can receive an image106for processing. The segmentation service309can identify text in the image106and use the identified text to determine an orientation of one or more documents109within the image106. In some embodiments, segmentation service309can modify the image106to enhance image quality. The segmentation service309can use an object detection model313to identify the documents109in the image106. In some embodiments, the object detection model313uses computer vision techniques to identify regions of interest within an image106and output bounding boxes116for each respective region of interest.

Once the object detection model313has returned the bounding boxes116, the segmentation service309can separate the documents109from the image106and store them as separated image files323, with one document109per separated image file323. According to various examples, the segmentation service309can determine a document type for the document109in each separated image file323. The segmentation service309can then send the separated image file323to a processing service316for further processing.

Moving on toFIG.4, shown is a flowchart that provides one example of the operation of a portion of the segmentation service309. The flowchart ofFIG.4provides merely an example of the many different types of functional arrangements that can be employed to implement the operation of the depicted portion of the segmentation service309. As an alternative, the flowchart ofFIG.4can be viewed as depicting an example of elements of a method implemented within the network environment300.

Beginning with block400, the segmentation service309can be executed to obtain or receive an image106. The image can be obtained in a variety of manners.

According to various examples, the segmentation service309can obtain the image106from a data store319. The segmentation service309can obtain the image106in response to receipt of an initiation request sent from a client application329, an interaction with a user interface100, or some other trigger received by the segmentation service309. In these embodiments, the initiation request could be received by the segmentation service309after the image106is stored in the data store319. In these instances, the initiation request could include an identifier of the image106to allow the segmentation service309to search for and obtain the image106from the data store319.

In other embodiments, the segmentation service309can receive the image106from another application, system, or service within the network environment300. For example, the client application329, after causing the client device103to capture the image106using a camera of the client device103, could send the image106to the segmentation service309.

Next, at block403, the segmentation service309can be executed to identify text within the image106. In some embodiments, the segmentation service309can use a computer vision technique to identify text within the image106. The segmentation service309can be language agnostic. For example, the segmentation service309can identify groups of alphanumerics or blocks of text by using various object identification means without regard to the language, context, or content of the text. In some implementations, optical character recognition (OCR) techniques can be used to identify individual characters, logographs, etc. in order to assist in identifying text blocks, including their orientation. In some embodiments, the segmentation service309can identify text from one or more documents109within the image106.

Next, at block409, the segmentation service309can calculate an average alignment angle for the text. In some embodiments, the segmentation service309can calculate an average alignment angle of the text identified at block403. In some embodiments, the segmentation service309can calculate an average alignment angle of the text identified at block403. The average alignment angle can be calculated by determining an alignment angle for each section of text or line of text and taking the average for all the text identified within the image106. In some embodiments, the segmentation service309can send the average alignment angle to a user interface100, a data store319, a client application329, or another application, system, or service in the network environment300.

At block413, the segmentation service309can be executed to determine an orientation of an image106. In some embodiments, the segmentation service309can determine the orientation of the image106based at least in part on the average alignment angle calculated at block409. The segmentation service309can be executed to determine the orientation of one or more documents109within the image106and/or the orientation of the image106itself. In some embodiments, the segmentation service309can determine the orientation of the one or more documents109in an image106based at least in part on the text identified at block403. The segmentation service309can use the text identified at block403or the average alignment angle calculated at block409to determine the orientation of the documents109and the image106itself.

At block416, the segmentation service309can be executed to modify the image106. The segmentation service309can adjust various features of the image106, such as tilt, orientation, shading, focus, scale, tone, contrast, or other aspects of the image106. In some embodiments, the segmentation service309can modify the image106in response to receiving a request from a client application329, an interaction with a user interface100, or another trigger from another application, system, or service within the network environment300. According to various examples, the segmentation service309can be executed to adjust at least a tilt and/or a rotation of the image106based at least in part on the orientation determined at block413. In some embodiments, the image106is modified based at least in part on the orientation determined at block413, the average alignment angle calculated at block409, or the text identified at block403. For example, the segmentation service309can compare the various alignment angles used to calculate the average alignment angle at block409to an angle of orientation determined at block413. From such a comparison, the segmentation service309can identify one or more lines of text which are out of alignment with the determined orientation and adjust the tilt of those identified lines of text to conform with the orientation or average alignment angle. In some embodiments, the segmentation service309can store the modified image106in a data store319.

Next, at block419, the segmentation service309can be executed to identify documents109within the image106. In some embodiments, the segmentation service309can identify documents109within the image106based at least in part on the modified image106from block416, the orientation determined at block413, the average alignment angle calculated at block409, or the text identified at block403. According to various examples, the segmentation service309can utilize an object detection model313to identify the documents109in an image106. The segmentation service309can use the object detection model313to produce bounding boxes which encapsulate the documents109within the image106. An example of the operation of the object detection model313to identify one or more documents109from within an image106is provided in the flowchart ofFIG.5.

Moving to block423, the segmentation service309can be executed to separate the documents109from the image106. In some embodiments, the segmentation service309can extract each document109identified at block419from the image106and generate separated image files323for each of the documents109identified. For example, the segmentation service309can separate the documents109from the image106by decompositing a first plurality of pixels from within a first bounding box identified in the image106at block419from a second plurality of pixels within a second bounding box identified in the image106. In another example, the segmentation service309can extract the pixels of one identified document109into a new separated image file323and crop the remaining image down to another identified document109, saving the cropped image as a new separated image file323. The segmentation service309can separate the documents109using image decomposition software (e.g., Adobe Photoshop, etc.), command line tools (e.g., Image Magick, etc.), or by manually editing the image106. In one example, the segmentation service309can perform a bit selection of all of the bits within a bounding box, extract those selected bits into a new file, and save the selected bits as a separated image file323. The segmentation service309can store the separated image files323in a data store319. In some embodiments, the segmentation service309can send the separated image files323to a client application329, a user interface100, or another application, system, or service in the network environment300.

At block426, the segmentation service309can be executed to extract data from one or more of the separated image files323. The segmentation service309can extract data from one or more separated image files323generated at block423for further processing of the documents109within. In some embodiments, the segmentation service309uses computer vision techniques to identify text, images, symbols, or other various features in a document109. The segmentation service309can use OCR to convert identified text to a machine-readable format for further data extraction. In such instances, the segmentation service309can perform text analysis such as key-word detection, word frequency, text classification, or other forms of text analysis. In various embodiments, the segmentation service309can extract other data from the document109, such as the shape, size, orientation, layout, etc. The data extracted from the separated image files323can be stored in a data store319, or sent to a client application329, a user interface100, or another application, system, or service in the network environment.

Next, at block429, the segmentation service309can be executed to identify a document type of a respective document109. In some embodiments, the segmentation service309can identify a document type of a document109in a separated image file323. In some embodiments, the segmentation service309can identify the document type of a document109based at least in part on the data extracted at block426, or other information. The segmentation service309can identify a document109as a check, a receipt, a bill, an invoice, a letter, or other document type. For example, the segmentation service309can use the data extracted at block426and conduct further processing to identify numeric values representative of an amount, an account number, a routing number, a check number, etc., within the document109. By comparing the position of the identified numeric values to expected positions of such values for a standard check, the segmentation service309can determine that the document109is a check. Next, the segmentation service309can determine to send the document109and extracted data from block426to a check processing service for deposit. In some embodiments, the segmentation service309stores the document type identified in a data store319or sends the document type identified to a client application329, a user interface100, or another application, system, or service within the network environment.

At block433, the segmentation service309can be executed to send separated image files323. The segmentation service309can use various data about a separated image file323to determine where to send the separated image file323. In some embodiments, the segmentation service309can use the document type identified at block429to determine that the separated image file323should be send to a processing service316. Based at least in part on the document type identified at block429, the segmentation service309can determine which of several processing services316the separated image file323should be sent to for further processing. In some embodiments, the segmentation service309can send the separated image files323to a processing service316, a data store319, a client application329, a user interface100, or another application, system, or service in the network environment300. After block433, the flowchart ofFIG.4comes to an end.

Moving on toFIG.5, shown is a flowchart that provides one example of the operation of a portion of the object detection model313. The flowchart ofFIG.5provides merely an example of the many different types of functional arrangements that can be employed to implement the operation of the depicted portion of the object detection model313. As an alternative, the flowchart ofFIG.5can be viewed as depicting an example of elements of a method implemented within the network environment300.

Beginning with block500, the object detection model313can be executed to identify regions of interest (ROIs). In some embodiments, the object detection model313can identify ROIs from an image106received from the segmentation service309. The object detection model313can identify ROIs in the image106which correspond to documents109or regions where a potential document109may be located. In some embodiments, the object detection model313can use the text identified at block403ofFIG.4to identify ROIs. In some embodiments, the object detection model313uses a feature map of the image106to determine ROIs. The object detection model313can utilize a Region Proposal Network to identify ROIs as well as, in some embodiments, a convolutional feature map. In some embodiments, the object detection model313uses an ROI Pool and a fully connected layer to identify one or more ROIs corresponding to documents109within an image106.

At block503, the object detection model313can output one or more bounding boxes116. According to various examples, the object detection model313can output a bounding box116for each ROI identified at block500. In some embodiments, the bounding boxes116represent a border surrounding an ROI. The bounding boxes116can correspond to a document109within the image106. In some embodiments, the object detection model313outputs more bounding boxes116than there are documents109in the image106. In some embodiments, the bounding boxes116are output based at least in part on the ROIs identified at block500and data associated with the ROIs, such as a total area, an overlap area, a height-width ratio, etc.

At block506, the object detection model313can generate a confidence score. In some embodiments, the object detection model313outputs a confidence score for each bounding box116. The confidence score can represent a measure of the determined accuracy of a bounding box116, or the likelihood that the bounding box116is correctly identifying a document109. The confidence scores can be stored as scores119along with the respective bounding box116and sent to a data store319, a client application329, a user interface100, or other application, system, or service in the network environment300. In some embodiments, the object detection model313can generate a confidence score based at least in part on an intersection over union (IoU) measure or an aspect ratio of an individual bounding box116, as well as potentially other factors, such as a word count within a bounding box116, etc.

Next, at block509, the object detection model313can remove spurious bounding boxes116. In some embodiments, the object detection model313can use a rule-based logic filter to detect and remove spurious bounding boxes116which were output at block503. The filter can detect spurious bounding boxes116based at least in part on the confidence score generated at block506, an overlap area between bounding boxes116, or various other indicators that a bounding box116may not be accurate. In some embodiments, the object detection model313filters out any bounding boxes116identified as spurious or non-confident and extracts them from the output of the object detection model313. The object detection model313can return the ROIs, confident bounding boxes116, and confidence scores to the segmentation service309, a client application329, a user interface100, or another application, system, or service in the network environment300. After block509, the flowchart ofFIG.5comes to an end.

A number of software components previously discussed are stored in the memory of the respective computing devices and are executable by the processor of the respective computing devices. In this respect, the term “executable” means a program file that is in a form that can ultimately be run by the processor. Examples of executable programs can be a compiled program that can be translated into machine code in a format that can be loaded into a random access portion of the memory and run by the processor, source code that can be expressed in proper format such as object code that is capable of being loaded into a random access portion of the memory and executed by the processor, or source code that can be interpreted by another executable program to generate instructions in a random access portion of the memory to be executed by the processor. An executable program can be stored in any portion or component of the memory, including random access memory (RAM), read-only memory (ROM), hard drive, solid-state drive, Universal Serial Bus (USB) flash drive, memory card, optical disc such as compact disc (CD) or digital versatile disc (DVD), floppy disk, magnetic tape, or other memory components.

The memory includes both volatile and nonvolatile memory and data storage components. Volatile components are those that do not retain data values upon loss of power. Nonvolatile components are those that retain data upon a loss of power. Thus, the memory can include random access memory (RAM), read-only memory (ROM), hard disk drives, solid-state drives, USB flash drives, memory cards accessed via a memory card reader, floppy disks accessed via an associated floppy disk drive, optical discs accessed via an optical disc drive, magnetic tapes accessed via an appropriate tape drive, or other memory components, or a combination of any two or more of these memory components. In addition, the RAM can include static random-access memory (SRAM), dynamic random access memory (DRAM), or magnetic random access memory (MRAM) and other such devices. The ROM can include a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or other like memory device.

Although the applications and systems described herein can be embodied in software or code executed by general purpose hardware as discussed above, as an alternative the same can also be embodied in dedicated hardware or a combination of software/general purpose hardware and dedicated hardware. If embodied in dedicated hardware, each can be implemented as a circuit or state machine that employs any one of or a combination of a number of technologies. These technologies can include, but are not limited to, discrete logic circuits having logic gates for implementing various logic functions upon an application of one or more data signals, application specific integrated circuits (ASICs) having appropriate logic gates, field-programmable gate arrays (FPGAs), or other components, etc. Such technologies are generally well known by those skilled in the art and, consequently, are not described in detail herein.

The flowcharts show the functionality and operation of an implementation of portions of the various embodiments of the present disclosure. If embodied in software, each block can represent a module, segment, or portion of code that includes program instructions to implement the specified logical function(s). The program instructions can be embodied in the form of source code that includes human-readable statements written in a programming language or machine code that includes numerical instructions recognizable by a suitable execution system such as a processor in a computer system. The machine code can be converted from the source code through various processes. For example, the machine code can be generated from the source code with a compiler prior to execution of the corresponding application. As another example, the machine code can be generated from the source code concurrently with execution with an interpreter. Other approaches can also be used. If embodied in hardware, each block can represent a circuit or a number of interconnected circuits to implement the specified logical function or functions.

Although the flowcharts show a specific order of execution, it is understood that the order of execution can differ from that which is depicted. For example, the order of execution of two or more blocks can be scrambled relative to the order shown. Also, two or more blocks shown in succession can be executed concurrently or with partial concurrence. Further, in some embodiments, one or more of the blocks shown in the flowcharts can be skipped or omitted. In addition, any number of counters, state variables, warning semaphores, or messages might be added to the logical flow described herein, for purposes of enhanced utility, accounting, performance measurement, or providing troubleshooting aids, etc. It is understood that all such variations are within the scope of the present disclosure.

Also, any logic or application described herein that includes software or code can be embodied in any non-transitory computer-readable medium for use by or in connection with an instruction execution system such as a processor in a computer system or other system. In this sense, the logic can include statements including instructions and declarations that can be fetched from the computer-readable medium and executed by the instruction execution system. In the context of the present disclosure, a “computer-readable medium” can be any medium that can contain, store, or maintain the logic or application described herein for use by or in connection with the instruction execution system. Moreover, a collection of distributed computer-readable media located across a plurality of computing devices (e.g., storage area networks or distributed or clustered filesystems or databases) may also be collectively considered as a single non-transitory computer-readable medium.

The computer-readable medium can include any one of many physical media such as magnetic, optical, or semiconductor media. More specific examples of a suitable computer-readable medium would include, but are not limited to, magnetic tapes, magnetic floppy diskettes, magnetic hard drives, memory cards, solid-state drives, USB flash drives, or optical discs. Also, the computer-readable medium can be a random access memory (RAM) including static random access memory (SRAM) and dynamic random access memory (DRAM), or magnetic random access memory (MRAM). In addition, the computer-readable medium can be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or other type of memory device.

Further, any logic or application described herein can be implemented and structured in a variety of ways. For example, one or more applications described can be implemented as modules or components of a single application. Further, one or more applications described herein can be executed in shared or separate computing devices or a combination thereof. For example, a plurality of the applications described herein can execute in the same computing device, or in multiple computing devices in the same computing environment303.

Disjunctive language such as the phrase “at least one of X, Y, or Z,” unless specifically stated otherwise, is otherwise understood with the context as used in general to present that an item, term, etc., can be either X, Y, or Z, or any combination thereof (e.g., X; Y; Z; X or Y; X or Z; Y or Z; X, Y, or Z; etc.). Thus, such disjunctive language is not generally intended to, and should not, imply that certain embodiments require at least one of X, at least one of Y, or at least one of Z to each be present.

It should be emphasized that the above-described embodiments of the present disclosure are merely possible examples of implementations set forth for a clear understanding of the principles of the disclosure. Many variations and modifications can be made to the above-described embodiments without departing substantially from the spirit and principles of the disclosure. All such modifications and variations are intended to be included herein within the scope of this disclosure and protected by the following claims.