Using machine learning to classify insurance card information

An insurance management system receives, from a client device, at least one image of an insurance card associated with the user operating the client device. The insurance management system extracts features from the at least one image and generates a feature vector associated with the at least one image. A machine learning model receives the feature vector as input and outputs a classification comprising an association with an insurance plan. The insurance management system receives historical medical information that is specific to the user and indicates a current medical condition of the user and evaluates the insurance plan of the user in view of their historical medical information. Based on the evaluation, the insurance management system provides a recommendation that comprises one or more alternative insurance plans or one or more alternative pharmacies.

BACKGROUND

This disclosure relates generally to medical insurance and more specifically to machine learning algorithms applied to insurance cards for identifying and recommending insurance plans.

In today's complex medical insurance market, there are a myriad of choices for medical insurance plans for consumers. The various types of medical insurance plans range from health maintenance organizations (HMOs), preferred provider organizations (PPOs), exclusive provider organizations (EPOs), point-of-service (POS) plans, to high-deductible health plans (HDHPs). Evaluating medical insurance plans as a consumer can be overwhelming when trying to balance the benefits of each plan against the costs of each plan. While insurance policies define intricate cost calculations for healthcare providers and healthcare consumers, they are often written with complicated language and terminology, making the policies challenging to decipher and making it more difficult for consumers to comparison shop. As a result, many consumers enroll in insurance plans that are not the most appropriate for their medical needs and financial budget.

SUMMARY

The present invention involves a method for applying machine-learning algorithms to identify an insurance plan associated with an insurance card of a user and providing a recommendation to the user based on the user's insurance plan. An insurance management system receives, from a client device, at least one image of an insurance card associated with the user operating the client device. The at least one image may include, e.g., a front side of the insurance card and a back side of the insurance card. The insurance management system extracts features from the at least one image, such as text information, object information, color information, and shape information. The insurance management system generates a feature vector associated with the at least one image and inputs the feature vector into a machine learning model that has been trained using classifications of insurance cards of other users. The machine learning model outputs a classification, which comprises an association with an insurance plan. The insurance management system stores the insurance plan of the classification as a current selection of an insurance plan of the user. In one embodiment, the insurance management system receives historical medical information that is specific to the user and indicates a current medical condition of the user. The historical medical information comprises one or more of a prescription history, a medical expense history, a medical claims history, a medical procedure history, and a co-payment history of the user. Based in part on the historical medical information, the insurance management system evaluates the current selection of the insurance plan of the user. Based on the evaluation, the insurance management system provides for display to the user in a user interface a recommendation associated with the current selection of the insurance plan of the user. In one embodiment, the recommendation comprises one or more alternative insurance plans for the user for the user to consider as alternatives to the insurance plan of the user associated with the insurance card. In one embodiment, the historical medical information indicates a pharmacy of the user, and the recommendation comprises one or more alternative pharmacies for the user for the user to consider as alternatives to the pharmacy of the user.

DETAILED DESCRIPTION

System Architecture

With dozens of types of health insurance plans, navigating the healthcare insurance landscape can be difficult. Health insurance plans may be offered through government exchanges, private exchanges, an employer, or an insurance provider directly. In addition, several options for types of health insurance plans may be offered (e.g., HMO, PPO, EPO, POS, etc.). Determining the most suitable insurance plan for an individual or a family can be challenging, and often, several individuals and/or families end up choosing the wrong insurance plan.

An insurance management system is configured to manage the insurance information of a user. The system may capture an image of an insurance card of a user via an application on a user's device. The system may scan the image of the insurance card to identify the insurance plan associated with the insurance card. Scanning the image allows the system to extract text, objects, logos, and any other relevant information from the insurance card, which can be input into an insurance data model that classifies the insurance card as being associated with an insurance plan. This thus avoids the extra time of the user having to enter information by hand or deliver details via phone about an insurance plan, and ensures the right information is captured and right plan is identified. The system applies machine learning techniques to the extracted information to improve its classifications of insurance cards and to better identify the insurance plan associated an insurance card. Since there are so many different insurance plans and different insurance card formats, collecting the appropriate information from the card and identifying the correct plan can be challenging. In some embodiments, the insurance management system may be integrated with a prescription management system, which allows the insurance management system to use historical medical information of the user (e.g., prescription history, medical claims history, etc.) to determine if the insurance plan identified from the insurance card information is the most suitable insurance plan for the user. The insurance management system may recommend one or more alternative insurance plans that are more suitable for the user based on the user's medical needs.

FIG. 1is a block diagram of a system environment100for an insurance management system130, in accordance with an embodiment. The system environment100shown byFIG. 1comprises a network110, one or more client devices120, the insurance management system130, and one or more third party systems140. In alternative configurations, different and/or additional components may be included in the system environment100. The embodiments described herein can be adapted to other suitable online systems that manage insurance information for users.

The client devices120are one or more computing devices capable of receiving user input as well as transmitting and/or receiving data via the network110. In one embodiment, a client device120is a conventional computer system, such as a desktop or a laptop computer. Alternatively, a client device120may be a device having computer functionality, such as a personal digital assistant (PDA), a mobile telephone, a smartphone or another suitable device. A client device120is configured to communicate via the network110. In one embodiment, a client device120executes an application allowing a user of the client device120to interact with the insurance management system130as a user of an application to access insurance information. For example, the user can use an application that allows the user to use a camera integrated into the client device120and allows the user to import images or videos into the application. For example, a client device120executes a browser application to enable interaction between the client device120and the insurance management system130via the network110. In another embodiment, a client device120interacts with the insurance management system130through an application programming interface (API) running on a native operating system of the client device120, such as IOS® or ANDROID™.

The insurance management system130manages the insurance information of a user. The user may interact with the insurance management system130through an application on the client device120of the user. In some embodiments, the insurance information may be entered by a user into the application, or the insurance information may be extracted from an image of the insurance card of the user that is imported into or taken through the application on the client device120. The insurance information may include one or more of the following: a patient name, an insurance carrier name, an insurance plan name, a patient identification number, an issuer identification number, a group number, a coverage effective date, an employer name, a bank identification number (BIN), co-payment information, a customer service number, and any other relevant insurance information. The insurance information may be located on a front side or a back side of the insurance card.

The insurance management system130may analyze the insurance information of the user. Using the information extracted from the insurance card, the insurance management system130may use machine learning techniques to classify the insurance card as being associated with a type of insurance plan provided to the user. The insurance management system130may store the classifications of insurance cards from several users to build a database of existing insurance plans. Applying machine learning techniques allows the insurance management system130to improve its classifications of insurance cards and to better identify the insurance plan associated with an insurance card. In some embodiments, the insurance management system130may be integrated with a prescription management system, as described in U.S. patent application Ser. No. 14/656,691 filed on Mar. 12, 2015, which is fully incorporated herein by reference. In these embodiments, the insurance management system130uses historical medical information of the user (e.g., prescription history, medical claims history, etc.) to determine if the insurance plan identified from the insurance card information is the most suitable insurance plan for the user. The insurance management system130may recommend one or more alternative insurance plans that are more suitable for the user based on the user's medical needs. While described as separate systems, the insurance management system130and the prescription management system may be part of a larger system in some embodiments.

One or more third party systems140may be coupled to the network110for communicating with the insurance management system130, which is further described below in conjunction withFIG. 2. In one embodiment, a third party system140is an application provider communicating information in databases or keys for accessing information in databases by a client device120. A third party system140may also communicate information in databases or keys for accessing information in databases to the insurance management system130. These databases may include information from a prescription management system, a publicly available database of insurance plans (e.g., Medicare), or personal accounts of the user to which access by the insurance management system130has been authorized.

FIG. 2is a block diagram of an architecture of the insurance management system130, in accordance with an embodiment. The insurance management system130shown inFIG. 2includes a quality measurement module205, a data extraction module210, a machine learning module215, an insurance data store220, a template store225, a third-party database store230, and a web server235. In other embodiments, the insurance management system130may include additional, fewer, or different components for various applications. Conventional components such as network interfaces, security functions, load balancers, failover servers, management and network operations consoles, and the like are not shown so as to not obscure the details of the system architecture. For example, as previously described, the insurance management system130may use historical medical information from other systems, such as a prescription management system. The information may include prescription information, medical claims history, or any other relevant medical information.

The quality measurement module205determines one or more quality measurements of an image where a quality measurement is associated with one or more adjustments in an image. An adjustment of an image can be a filter applied to an image or an adjustment of orientation, brightness, contrast, hue, saturation, color balance, one or more color vibrancies, levels, curves, exposure, thresholds, luminance, and any other suitable adjustment that can be made to an image. The quality measurement module205determines one or more quality measurements of an image and, based on the quality measurements, provides information describing the quality measurements to a user of the system130. The provided information can include suggestions, instructions, or guidelines for improving quality of the images based on the quality measurements, as described further below in conjunction withFIGS. 3, 4A, and 4B.

The data extraction module210identifies or extracts machine-readable information or text in an image using one or more text recognition algorithms, one or more object recognition algorithms, or any other suitable feature recognition algorithm. The data extraction module210accesses images of an insurance card where each image includes a partial or full view of a front side or a back side of the insurance card. Associations are determined between the identified text and one or more types of information where a type of information can be information about an insurance plan or information about the user. In one embodiment, the data extraction module210determines these associations based on a determination of a template that matches (or matches with a threshold level of confidence) the insurance card. A template is a geometrical representation of an insurance card known to the insurance management system130. For example, if the template is known to have a certain type of information in a first slot in an upper right corner of an insurance card, then the identified text in the upper right corner of the insurance card is associated with the certain type of information. The identified text in an insurance card and determined associations of the identified text with one or more types of information is stored in the insurance data store220. The function provided by the data extraction module210is further described below in conjunction withFIG. 6.

The machine learning module215applies machine learning techniques to generate an insurance data model that when applied to images of insurance cards outputs a classification associated with the information on the insurance card. In the embodiment ofFIG. 2, the classification indicates a probability that the insurance card is associated with a specific type of insurance plan. In other embodiments, the classification may indicate a confidence level that the insurance card matches a template, a probability that the insurance card is associated with a specific insurance company, or some combination thereof.

As part of the generation of the insurance data model, the machine learning module215forms a training set of images of insurance cards by identifying a positive training set of images of insurance cards that have been accurately classified as being associated with a specific type of insurance plan. In some embodiments, the machine learning module215forms a negative training set of images of insurance cards as not belonging to a specific insurance company or being associated with a specific type of insurance plan. The training set of images may be reviewed manually by reviewers or automatically by the system130.

The machine learning module215extracts feature values from the images of insurance cards of the training set, the features being variables deemed potentially relevant to classifying the insurance cards. Specifically, the feature values extracted by the machine learning module215include text, font size and/or style, logos or objects, color, shape, or any other relevant feature that may be extracted from an image. An ordered list of the features for an insurance card is herein referred to as the feature vector for the insurance card. In one embodiment, the machine learning module215applies dimensionality reduction (e.g., via linear discriminant analysis (LDA), principle component analysis (PCA), or the like) to reduce the amount of data in the feature vectors for insurance cards to a smaller, more representative set of data.

The machine learning module215uses supervised machine learning to train the insurance data model, with the feature vectors of the positive training set and the negative training set serving as the inputs. Different machine learning techniques—such as linear support vector machine (linear SVM), boosting for other algorithms (e.g., AdaBoost), neural networks, logistic regression, naïve Bayes, memory-based learning, random forests, bagged trees, decision trees, boosted trees, or boosted stumps—may be used in different embodiments. The insurance data model, when applied to the feature vector extracted from an insurance card, outputs a classification of the insurance card as being associated with a type of insurance plan, such as a Boolean yes/no estimate or a scalar value representing a probability.

In some embodiments, a validation set is formed of additional images of insurance cards, other than those in the training sets, which have already been classified as being associated or not associated with a type of insurance plan. The machine learning module215applies the trained validation insurance data model to the images of insurance cards of the validation set to quantify the accuracy of the insurance data model. Common metrics applied in accuracy measurement include: Precision=TP/(TP+FP) and Recall=TP/(TP+FN), where precision is how many the insurance data model correctly classified (TP or true positives) out of the total it classified (TP+FP or false positives), and recall is how many the insurance data model correctly classified (TP) out of the total number of insurance cards that were classified (TP+FN or false negatives). The F score (F-score=2*PR/(P+R)) unifies precision and recall into a single measure. In one embodiment, the machine learning module215iteratively re-trains the insurance data model until the occurrence of a stopping condition, such as the accuracy measurement indication that the model is sufficiently accurate, or a number of training rounds having taken place.

The insurance data store220stores insurance information of users of the insurance management system130. In one embodiment, the stored insurance information is stored in association with identifiers of users and, in another embodiment, the insurance data store220stores one or more user profiles associated with users of the insurance management system130and each user profile associated with a user includes insurance information associated with the user. The insurance data store220may also store the one or more images of the insurance card associated with the insurance information of the user and may store the classifications associated with the one or more images of the insurance card of the user.

The template store225stores one or more templates and the companies and/or insurance plans associated with the one or more templates. A template is a geometrical representation of an insurance card known to the insurance management system130. For example, if insurance company A is known to use a first type of insurance card for a first insurance plan and a second type of insurance card for a second insurance plan, the template store225stores a first template for the first insurance plan, a second template for the second insurance plan, and an association with insurance company A with the first and second types of templates. Thus, if Anthem® is known to use a certain template for an insurance card for a family insurance plan, then the template store225can store the template in association with Anthem® and the type of insurance plan. Each template in the template store225includes one or more slots and, for each slot in the template, stores a type of information associated with the slot. For example, if a first slot in a template is known to be associated with a first type of information (e.g., user name), then the template store225stores the first slot in the template in association with the first type of information. In one embodiment, a location of the first slot in the template is associated with the first type of information.

The third-party database store230stores information from one or more databases associated with third party systems140or keys accessing information in one or more databases associated with third party systems140. For example, if a known insurance company is company A, then the insurance management system130may have access to company A's databases and company A may have access to the insurance data store220of the insurance management system130. In one embodiment, information associated with a user of the system130included in a database in a third party system130is only accessible by the system130if the user gives permission for the information to be transferred.

The web server235links the insurance management system130via the network110to the one or more client devices120, as well as to the one or more third party systems140. The web server235serves web pages, as well as other content, such as JAVA®, FLASH®, XML, and so forth. The web server235may receive and route messages between the insurance management system130and the client device120, for example, information describing quality measurements, insurance cards, or other suitable messages. Additionally, the web server235may provide application programming interface (API) functionality to send data directly to native client device operating systems, such as IOS®, ANDROID™, WEBOS® or BlackberryOS.

Preprocessing an Image Using Quality Measurements

FIG. 3is a flowchart of a method for preprocessing an image using quality measurements, in accordance with an embodiment. In various embodiments, the steps described in conjunction withFIG. 3may be performed in different orders. Additionally, in some embodiments, the method may include different and/or additional steps than those shown inFIG. 3. The functionality described in conjunction with the insurance management system130inFIG. 3may be provided by the quality measurement module205, in one embodiment, or may be provided by any other suitable component, or components, in other embodiments. Additionally, the client device120may execute one or more instructions associated with the insurance management system130, such as an application associated with the insurance management system130, to provide the functionality described in conjunction withFIG. 3.

The system130receives305an image of an insurance card associated with a user from a client device operated by the user. Each image is of a front side or a back side of the insurance card. The insurance card may include insurance information regarding one or more of the following: a patient name, an insurance carrier name, an insurance plan name, a patient identification number, an issuer identification number, a group number, a coverage effective date, an employer name, a bank identification number (BIN), co-payment information, a customer service number, and any other relevant insurance information. The insurance information may be located on a front side or a back side of the insurance card. In some embodiments, the system130receives305a first image of a first side of the insurance card and then receives305a second image of a second side of the insurance card. In some embodiments, the front image and the back image may be combined as a single image by the system130.

In alternative embodiments, the images can be a video feed and can be automatically captured by an application executing on the client device operated by the user that is associated with the system130. For example, a video feed or images can be captured automatically based on camera motion (based on motion data from a motion sensor integrated in the client device120), values associated with pixels in the image, any suitable real-time analysis of the images, information describing an environment where the image or video feed is being captured, or any combination thereof. In these embodiments, the system130may be configured to detect objects that are shaped similar to a card (e.g., square, rectangular, etc.). The system130may detect the corners and/or the edges of the object and apply a transformation (e.g., reverse perspective, homography, etc.) on the object to capture an image of a certain quality. For example, the system130may determine the four edges of the card in the image and apply a homography to normalize the card to a substantially flat rectangle (i.e., transforming a card that is diagonal or at an angle, such as illustrated inFIG. 4B, to a rectangle having a standard card aspect/ratio). In this configuration, the system130captures an image of, e.g., a flat rectangular card even if a camera of the client device is not oriented head-on or parallel to the card. The image can be captured automatically once a frame of the video feed has a quality measurement above a certain threshold.

One or more quality measurements of the image are determined310. Each quality measurement is associated with one or more adjustments in the image. An adjustment of an image includes adjusting brightness, contrast, hue, saturation, color balance, one or more color vibrancies, levels, curves, exposure, thresholds, luminance, and any other suitable adjustment that can be made to an image. The adjustment of an image can also be a filter applied to the image. A quality measurement of an image is a measurement of an adjustment of the image. For example, if the adjustment of the image is contrast, then the quality measurement of the image can be a measurement of how much contrast has been adjusted for the image. Responsive to determining310one or more quality measurements of the image, the system130can present information to the user describing the quality measurements, as further described below in conjunction withFIGS. 4A and 4B. Similarly, the system130can present to the user, via an application executing on a client device120operated by the user, instructions for rotating the insurance card in a certain orientation, as further described below in conjunction withFIGS. 4A and 4B.

In response to a quality measurement (e.g., in the one or more quality measurements determined as described above) not exceeding one or more threshold measurements associated with one or more adjustments of the quality measurement, the system130requests315a replacement image for the image. In one embodiment, the one or more quality measurements are for lighting, camera motion, and focus. Thus, an application associated with the system130executing on the client device120checks for lighting, camera motion, and focus through luminance values in an image, motion data associated with the client device120, and edge density analysis, respectively.

The application can check for sufficient lighting through the luminance values of the image. The application extracts the luminance values of the image and, if the captured image is in YUV format, the luminance values are simply the Y components of the pixel values. If the image is in another format, a color space transform can be applied to the image to convert the image into YUV format. The application or the server130can determine a histogram of the luminance values and determine brightness and a contrast of the image based on a minimum value associated with the histogram, a maximum value associated with the histogram, a difference between the minimum value and the maximum value, or any combination thereof. Responsive to brightness, contrast, or both brightness and contrast of the image not exceeding corresponding threshold measurements associated with brightness, contrast, or both brightness and contrast, respectively, the application can display information to the user describing the brightness and contrast of the image. For example, the information displayed can include a message encouraging the user to increase the ambient lighting, a message informing the user that more light is needed for accuracy, or any other suitable message assisting the user in capturing an image with better quality.

The application can check for camera motion via an accelerometer or any other suitable motion sensor integrated into the client device120operated by the user executing the application. For an accelerometer, if a magnitude of the accelerometer readings is above a threshold measurement associated with the accelerometer readings, the application can display information to the user describing the motion of the camera and whether the camera needs to be steadier during image capture.

The application or the system130can also check for focus by applying one or more filters to an image. For example, the application can apply a Gaussian blur filter to the image to eliminate spurious high-frequency artifacts, a Laplacian filter to detect edges, or any combination thereof. An edge density value associated with the image after the image is filtered can be determined. The edge density value can be an average edge density value for a plurality of sample points in the image, a maximum edge density value associated with the image, or any other suitable combination of edge density values in the image. If the edge density value does not exceed a threshold measurement associated with edge densities, the application can display information to the user describing edge density in the image. The threshold measurement for edge densities can be based on empirical testing on various device models and configurations.

The system130can determine one or more threshold measurements corresponding to one or more quality measurements measuring adjustment levels of an image. Thus, a quality measurement of an image exceeding a corresponding threshold measurement represents the image having an adjustment of the image that does not lessen the quality of the image. For example, if the quality measurement of an image is a measurement of contrast, then the quality measurement exceeding a corresponding threshold measurement associated with contrast represents the image having enough contrast that details in the image are recognizable, such as text in the image. In another embodiment, a quality measurement of an image not exceeding a corresponding threshold measurement represents the image having an adjustment of the image that does not lessen the quality of the image. For example, if the quality measurement of an image is a measurement of exposure, then the quality measurement not exceeding a corresponding threshold measurement associated with exposure represents the image not having too much exposure so that details in the image such as text are recognizable. Rather than or in addition to threshold measurements, the system130can determine ranges of measurements associated with adjustment levels of an image. Thus, a quality measurement of an image that is within the range of measurements represents the image having an adjustment of the image that is within a range that does not lessen the quality of the image. The system130can also present information to the user describing whether one or more of the quality measurements exceed corresponding threshold measurements, do not exceed corresponding threshold measurements, are within a range of measurements determined by the system130, or any combination thereof, as further described below in conjunction withFIGS. 4A and 4B.

In response to the one or more quality measurements exceeding corresponding threshold measurements associated with adjustments for the quality measurements, the system130determines320one or more additional quality measurements of another image (e.g., of the second side of the insurance card). If each of the quality measurements exceed corresponding threshold measurements, do not exceed corresponding threshold measurements, are within a range of measurements determined by the system130, or any combination thereof, then the quality of the image is suitable for identifying details and features in the image. The system130can also determine whether the images capture the front side and the back side of the insurance card. Where each image has quality measurements exceeding corresponding threshold measurements, the images can be stored in the insurance data store220.

User Interface

FIGS. 4A, 4B, 4C, and 4Dillustrate a user interface400for capturing an image of an insurance card, in accordance with an embodiment. The system130may display the user interface400to the user during initialization of the application, or the user may skip this step during initialization and may later initiate the user interface400through the settings of the application. If skipped, the system130may send a reminder to the user to complete this insurance card step. As shown inFIG. 4A, the system130can provide guidelines405to a client device120operated by a user for presentation to the user in the user interface400on an application executing on the client device120. In addition to the guidelines405, the system130can also present instructions410to the user. For example,FIG. 4Apresents “Place the front of your insurance card on a flat surface. We'll autocapture the image.” The application may automatically capture images responsive to detecting the insurance card aligned within the guidelines405. As shown inFIG. 4A, the user interface includes a toggle button415to turn on and off the autocapture. Additional examples of instructions include “Place the back of your insurance card on a flat surface,” “frame and focus the insurance card,” and “capture all of the insurance card,” which can also be presented to the user via a client device120operated by the user prior to capturing images.

As shown inFIG. 4B, the system130can also present information420in the user interface400describing quality measurements associated with a captured image or a real-time video feed. The information can describe whether a quality measurement of an adjustment in a captured image or real-time video feed is above or below a threshold measurement associated with the adjustment. Thus, if the lighting is poor based on luminance values in an image not exceeding threshold luminance values, the information420can state “LOW LIGHT—Please use better lighting.” If the focus is poor based on motion data of the client device120or edge density values associated with an image exceeding threshold measurements associated with motion data or threshold measurements associated with edge density values, respectively, the information420can state “OUT OF FOCUS—Please move the card farther away from the camera.” As shown inFIG. 4B, an insurance card425does not adequately fill the guidelines405; thus, the information420states “Move closer” and depicts arrows to instruct the user to fill the guidelines405with the insurance card425.

As shown inFIG. 4C, the system130presents information420in the user interface400that indicates an image of the insurance card425is captured. In the embodiment ofFIG. 4C, the autocapture toggle button415is in the “ON” position. In an autocapture mode, the system130may capture an image of the insurance card425responsive to detecting that all quality measurements are above or below a respective threshold measurement.FIG. 4Cillustrates that the insurance card425adequately fills the guidelines405, and the lighting, contrast, and focus of the insurance card425are sufficient such that the text on the insurance card425is legible. If the autocapture mode is off, a user may provide a user input via the user interface400to capture an image of the insurance card425. In some embodiments, the information420presented inFIG. 4Cmay indicate that the insurance card425is appropriately positioned within the guidelines405and is ready to be captured.

As shown inFIG. 4D, the system130presents a captured image430of the insurance card425in the user interface400. The user interface400allows a user to review the captured image430to ensure that the quality of the captured image430is sufficient. The user interface400offers the user an “Accept” option435if the captured image430is acceptable and can be stored in association with the user and a “Retake” option440if the captured image430should be retaken. If a user chooses the “Accept” option435, the system130may store the captured image430in association with the user. If a user chooses the “Retake” option440, the system130may delete the captured image430and return the user to the user interface shown inFIG. 4AorFIG. 4B.

Using Information from the Image of the Insurance Card

FIG. 5is a schematic block diagram of a training phase of the machine learning module215, in accordance with an embodiment. During the training phase, the machine learning module215implements a learning algorithm to learn how to classify an insurance card as being associated with an insurance plan. The machine learning module215inputs into its model a large number of examples of insurance cards of a training set. As described with regards toFIG. 2, the training set may be a positive training set or a negative training set, where the example insurance cards are classified as associated or not associated, respectively, with an insurance plan. The machine learning module215learns using these examples to determine a set of coefficients for the insurance data model, enabling the insurance data model to output classifications of other insurance cards as being associated with an insurance plan. In some embodiments, the machine learning module215identifies or creates one or more templates based on the learned examples and stores the templates in the template data store225such that the templates may be later used to determine a match between other insurance cards and the templates.

In the embodiment ofFIG. 5, the machine learning module215inputs data from several examples of insurance cards that have been previously classified as associated or not associated with an insurance plan. As illustrated inFIG. 5, an example includes a set of features comprising one or more captured images505, text information510, object information515, shape information520, and color information525. The text information510, object information515, shape information520, and color information525may be information extracted from the one or more captured images505. In alternate embodiments, the set of features for an example may include different types of combinations of information. For example, the examples may be associated with a template such that the machine learning module215may also learn how to determine a match between the example insurance card and the template. The captured images505may include multiple images of the insurance card, e.g., an image of the front side and an image of the back side. The text information510includes text on the insurance card that may be identified using text recognition algorithms, where the text includes one or more of the following: a patient name, an insurance carrier name, an insurance plan name, a patient identification number, an issuer identification number, a group number, a coverage effective date, an employer name, a bank identification number (BIN), co-payment information, and a customer service number. The object information515may include one or more logos, images, or some combination thereof on the insurance card. The shape information520includes a shape of the insurance card, e.g., rectangular or square. The color information525may include color of text on the insurance card, a background color of the insurance card, color of one or more logos or images, or some combination thereof.

The machine learning module215receives as input, for each example insurance card, the captured images505, the text information510, the object information515, the shape information520, and the color information525, and then, for each example, outputs a classification of an insurance plan associated with the example insurance card. The machine learning module215performs error detection530and compares the output classification to the actual classification of the insurance card. Based on a calculated difference535, the machine learning module215may adjust the coefficients of its insurance data model to reduce the detected error. The machine learning module215may perform the training process multiple times for one or more example insurance cards that it receives.

In some embodiments, the machine learning module215trains the insurance data model using the positive training set or the negative training set and then validates the insurance data model using the validation training set. In these embodiments, the machine learning module215determines the coefficients of the insurance data model using the positive training set or the negative training set and then inputs the example insurance cards of the validation training set to the insurance data model to perform error detection530. The machine learning module compares the output classifications to the actual classifications of the example insurance cards of the validation training set. Performing error detection530may include evaluating the insurance data model using one or more of the metrics described with regard toFIG. 2, e.g., precision, recall, the F score, or some combination thereof. The machine learning module215may iteratively train the insurance data model until the occurrence of a stopping condition, such as the difference535is below a specified threshold, the accuracy measurement indication that the model is sufficiently accurate, or a number of training rounds having taken place.

FIG. 6is a flowchart of a method600for applying machine learning techniques to an image of an insurance card, in accordance with an embodiment. In various embodiments, the steps described in conjunction withFIG. 6may be performed in different orders. Additionally, in some embodiments, the method may include different and/or additional steps than those shown inFIG. 6. The functionality described in conjunction with the insurance management system130inFIG. 6may be provided by the quality measurement module205, the data extraction module210, and the machine learning module215, or may be provided by any other suitable component, or components, in other embodiments. Additionally, the client device120may execute one or more instructions associated with the insurance management system130, such as an application associated with the insurance management system130, to provide the functionality described in conjunction withFIG. 6.

The system130accesses605an image of an insurance card of a user. Each image is associated with a front side or a back side of the insurance card. In some embodiments, the system130accesses605multiple images of the insurance card, e.g., an image of the front side and an image of the back side. The accessed image(s) may be the image(s) received305from the method described in conjunction withFIG. 3. As previously described in conjunction withFIG. 3, one or more quality measurements associated with each image can be determined310and compared to threshold measurements to determine whether a new image is needed.

The system130extracts610information from the insurance card. The extracted information may include text, font size and/or style, logos or objects, color, shape, or any other relevant feature that may be extracted from an image. In addition, the system130can extract610other features on the insurance card such as a barcode, a QR code, or any other suitable machine-readable representation of data. Text on the insurance card may be identified using text recognition algorithms. For example, the system130may use optical character recognition, optical word recognition, intelligent character recognition, intelligent word recognition, or any other suitable text recognition algorithm. Based on patterns of characters in the extracted text, the system130can associate various patterns of the characters in the extracted text with types of information. For example, if the characters include five numbers, then the system130can associate the five numbers with a zip code. If the characters include ten numbers, the system130can associate the ten numbers as a phone number.

The system130creates615a feature vector using the extracted information from the insurance card. The feature vector is an ordered list of the features for the insurance card. In one embodiment, the system130applies dimensionality reduction (e.g., via linear discriminant analysis (LDA), principle component analysis (PCA), or the like) to reduce the amount of data in the feature vectors for insurance cards to a smaller, more representative set of data.

The system130inputs620the feature vector into an insurance data model. The system130uses the insurance data model to determine if the insurance card associated with the feature vector is associated with a type of insurance plan. In some embodiments, the insurance data model allows the system130to determine if the insurance card matches a template. As previously described, each template is associated with one or more slots of the template associated with a type of information. For example, a template associated with a company and, based on the association with the company, various slots in the template are associated with a type of information provided on the insurance card. For example, a template can include a slot that is a designated sub portion of the insurance card for a type of information such as information about an insurance plan or information about the user. If the feature vector associated with the insurance card includes a name of a company, then the system130may determine that the insurance card may use a template that the company is known to use. As an example, if Aetna® is known to use a certain type of template and the feature vector includes text having the sequence of characters “Aetna,” then the system130determines that the insurance card may use the certain type of template. In some instances, the system130may determine that the template used by the company is not the template of the insurance card based on the feature vector associated with the insurance card.

The system130outputs625a classification of the insurance card using the insurance data model, which allows the system130to identify the insurance plan on the insurance card. The classification indicates if the insurance card is associated with a specific type of insurance plan known to the system130in the form of, e.g., a Boolean yes/no estimate or a scalar value representing a probability. In other embodiments, the classification may indicate a confidence level that the insurance card matches a stored template of an insurance card, a probability that the insurance card is associated with a specific insurance company, or some combination thereof. The classification of the insurance card is stored in association with the user of the insurance card.

Once the insurance plan of the user is identified, the system130may analyze630the historical medical information of the user. The historical medical information may include prescription history, medical procedure history, medical claims, co-payment history, or any other relevant medical history of the user. The system130analyzes630the historical medical information of the user in the context of the identified insurance plan of the user. This analysis allows the system130to determine if the identified insurance plan is the most suitable insurance plan for the user. The insurance management system130may recommend635one or more alternative insurance plans to the user that are more suitable for the user based on the user's medical needs.

FIG. 7illustrates a user interface700for presenting information extracted from the captured image430of the insurance card425, in accordance with an embodiment. The user interface700may be a user interface of an application executed on a client device120that allows a user to interact with the system130and access insurance information. In the embodiment ofFIG. 7, the user interface700presents the captured image430, which includes the front side of the insurance card425. The user interface includes an option705that, when selected, allows the user to view a captured image of the back side of the insurance card425. A notification710states “Card scan successful!,” indicating that the insurance card425in the captured image430has been successfully scanned. A successful scan indicates that the captured image(s) of the insurance card425is of sufficient quality and that the system130is able to extract the information on the insurance card425from the captured image430. The extracted information includes text, objects, and/or logos on the insurance card425.

Once the information is extracted from the captured image430, the system130presents the extracted information to the user in the user interface700. As shown inFIG. 7, the user interface700includes a “Details” tab715and a “Benefits” tab720. The “Details” tab715displays the extracted information by populating respective data fields in the user interface700. For example, the extracted claim number populates data field715, the extracted coverage effective dates populate data fields720and725, and the extracted customer service number populates data field730. The “Benefits” tab720may display the benefits information associated with the insurance plan identified in the insurance card425. Benefits information may include co-payment information for in-network and out-of-network providers and/or procedures, deductible amount, out-of-pocket expense limits, or any other information relevant to an insurance plan. The system130may determine the benefits of the insurance plan using an internal database or a third party database or system. The data fields in the user interface700may be editable and selectable options so that, if the user identifies an error in the extracted information, the user can edit the extracted text and/or association of the text to a data field. Thus, if the user sees that a portion of a phone number has been associated as a date, the user can select the date and an option associating the date as part of the phone number via options presented to the user responsive to the user selecting the date.

Features Available to a User of the Insurance Management System

A user of the insurance management system130has access to various features associated with stored insurance information. The features may be accessed through a client device120of the user, and many of the features be automatically set up by the system130without requiring additional user input (or requiring minimal user input or simply user confirmation of a feature) after images of the insurance card of the user are captured. Example features include network updates, insurance benefits updates, payment history and management options, and insurance customer service access. A network update may be a feature specific to providers or procedures available in a user's network. An insurance benefit update may be a feature specific to benefits of the user's insurance plan. Payment history and management options include other features that may allow a user to track their medical claims, explanation of benefits (EOBs), health spending account (HSA) activity and/or balance, etc. Insurance customer service access may offer a method for conveniently contacting customer service at the insurance company. Additionally, in some embodiments, the system130may be integrated with a prescription management system or may have access to personal accounts of the user that provide historical medical information of the user to the system130. In these embodiments, the system130may analyze the user's medical needs to determine if the insurance plan identified on the insurance card is the most suitable insurance plan for the user. The system130may recommend one or more alternative insurance plans that are more suitable for the user based on the user's medical needs or that may result in cost-savings for the user. The various features will be described in conjunction withFIGS. 7 and 8.

FIG. 8is an example of an application feature800for comparing insurance plans, in accordance with an embodiment. As illustrated inFIG. 8, the application feature800presents a financial overview805of the user's current insurance plan. The financial overview805includes the current annual cost of $647, the monthly premium of $28.90, the health deductible amount of $0, and the drug deductible amount of $300. Additional information may be presented to the user that highlights the costs associated with the user's insurance plan (e.g., average out-of-pocket expense, percentage of deductible used, etc.).

The application feature800additionally presents a first insurance plan, Plan Name A, and a second insurance plan, Plan Name B. The system130may select the first insurance plan and the second insurance plan from a database of stored insurance plans. The insurance plans may be selected based on an analysis of the user's historical medical information (e.g., prescription history, medical claims history, etc.), the benefits offered by each insurance plan, and the estimated cost savings compared to the user's current insurance plan. The system130may perform the analysis without requiring user input into the user interface of the application and may provide a notification to the user through the user interface that informs the user that a recommendation for alternative insurance plans is available. For example, the system130may detect that a user has several recurring prescriptions based on payment history of the user. If the user's current insurance plan does not cover or only partially covers the cost for the prescriptions, or has a high drug deductible, the user may have high out-of-pocket expenses. An insurance plan that has a lower drug deductible or that covers a larger portion of the costs of the prescriptions may be a better suited insurance plan for the user based on the user's medical needs. As illustrated inFIG. 8, the application feature800presents a financial overview810,815for each selected plan, Plan Name A and Plan Name B, respectively, that shows the estimated annual costs of the plan and compares the annual savings of the plan compared to the user's current insurance plan. The financial overviews810,815may include a link to view the full benefits of the plan. The application feature800may be further configured to connect the user with a representative at the insurance company of the alternative plan and/or schedule a phone call for the user during open enrollment dates. WhileFIG. 8presents two alternative insurance plans, the number of alternative insurance plans presented may vary in other embodiments.

FIG. 9is an example of an application feature900for comparing medication co-payments, in accordance with an embodiment. As illustrated inFIG. 9, the application feature900presents a financial overview905that compares a user's prescription co-payment costs at the user's current pharmacy and possible annual savings for the user if the user switches to a second pharmacy. The application feature900presents a financial breakdown910of the user's prescription costs at the user's current pharmacy and a financial breakdown915of the user's prescription costs at an alternative pharmacy. The financial breakdowns910,915list each prescription of the user and the respective insurance plan contribution and individual contribution for each prescription. The system130may select the alternative pharmacy based on an analysis of the user's prescription payment history, insurance plan benefits, pharmacies in the network of the user's insurance plan, pharmacy proximity to the user, and any other relevant factor, or some combination thereof. The application feature900may offer an option920that allows a user to change their current pharmacy.

In some embodiments, the application associated with the insurance management system130that is executed on the client device120may have additional features that allow a user to manage their insurance and/or medical information. For example, the application may allow the user to import medical claims, EOBs, and insurance payments. The user interface on the application may display this information to the user in an easily navigable manner. The application may be further configured to highlight duplicates, errors, and/or outstanding payments, reimbursements, or claims.

In some embodiments, the application may have additional features that allow a user to manage their insurance and/or medical information for future planning. For example, the system130may analyze patterns of the user based on procedure history, claims history, payment history, etc. The system130may run a user-specific simulation that can inform the user of their predicted health spending account usage, deductible tracking, etc. and make recommendations to the user based on the simulation. For example, the system130may be aware that the user has been wanting to receive corrective eye surgery. Based on analysis of the user's patterns, the system may recommend an appropriate timing for the user to receive the corrective eye surgery (e.g., “You're on track to meet your deductible by June; this might be a good time to schedule the corrective eye surgery.”).

Any one or more of the features described above can be automatically set up for the user based merely on the user having taken an image to capture insurance plan information. Thus, the user does not have to manually input information as this information can be automatically done by the system. In one embodiment, the system130automatically sets up these features for the user based on the information extracted from the captured image of the insurance card such that the user automatically starts receiving recommendations or can access the recommendations after capturing the image. In another embodiment, the system130automatically provides a prompt to the user to confirm to have these features set up or otherwise collects information/input needed from the user for setting up these features. In a further embodiment, the user provides preferences upon initially setting up an account (e.g., I would like recommendations for cost-saving insurance plans), and these preferences are applied in automatically setting up these features as images are taken of an insurance card.

Additional Considerations