Patent Description:
Skin pigmentation, also referred to herein as skin tone or skin color, is highly variable across and within populations. Skin color is important from a clinical perspective because the incidence, morbidity, clinical presentation, and treatment of skin diseases may vary based on skin type. However, biases against non-white skin types in dermatology practice and research today are having a significant impact on quality of care. Presentation of skin diseases may vary by skin tone, with significant impacts on survival rate. As an example, skin cancers such as melanoma, the deadliest form of skin cancer, are diagnosed at later, less-treatable stages, in people of color (i.e., non-white), due in part to their presentation in non-UV exposed areas of the body, such as the hands, nails, feet, and mucous membranes, which are not commonly examined. Therefore, while there is a higher incidence of melanoma in Caucasians, there is a higher mortality rate among skin of color.

One of the most well-known classification systems is the Fitzpatrick scale, which categorizes skin color into six skin types (I-VI) based on self-reported questionnaires. Skin Types I-IV were originally created in <NUM> in France to categorize Caucasian skin based on a clinical response to ultraviolet (UV) radiation, but Type V (brown Asian and Latin American) and Type VI (dark African skin) were only later added to capture non-Caucasian skin tones based on constitutive pigmentation or ethnic origin. The Fitzpatrick scale is commonly used in skin cancer studies; however, critiques of the tool note errors associated with self-reporting and the limited applicability to all skin types given the basis of ethnic origin, which is changing, resulting in difficulties with the clustering of certain ethnic/racial group into one skin type.

<CIT> describes a method for detecting coronary artery calcification in an individual, wherein the method comprises providing a spectroscopic apparatus configured to measure the skin fluorescence of the individual. Further, the method comprises detecting the intrinsic skin fluorescence of the individual with the spectroscopic apparatus and determining a measure of coronary artery calcification from the intrinsic fluorescence.

Use of systems like the Fitzpatrick scale are inadequate in terms of their ability to accurately define skin color. This in turn results in imprecise or missed diagnoses of skin conditions through existing processes.

The invention is defined by the enclosed independent claims. Embodiments result from the dependent claims and the description below.

The present disclosure relates to an automated system and method for objectively analyzing and classifying skin tones using machine learning. Such analysis and classification may be used to augment the analysis, diagnosis, and treatment of skin. In an embodiment according to the claimed invention, the system comprises a classifier selection module for selecting a set of machine learning diagnostic models from a plurality of sets of candidate machine learning models, each of the sets of candidate machine learning diagnostic models trained to receive a concern image of a portion of the patient's skin and output a diagnosis of a condition of the patient. According to the invention, the system further comprises an image capture module for receiving a base skin tone image of a patient and for generating a calibrated base skin tone image by calibrating the base skin tone image using a reference calibration profile. Further according to the invention, the system comprises a base skin tone determination module for determining a base skin tone of the patient based on the calibrated base skin tone image and a concern image processing module for receiving the concern image, wherein the classifier selection module selects the set of machine learning models based on the attribute of the patient, wherein the attribute is the base skin tone. The system receives a base skin tone image of a patient. The system generates a calibrated base skin tone image by calibrating the base skin tone image using a reference calibration profile. The system determines a base skin tone of the patient based on the calibrated base skin tone image. The system then receives a concern image of a portion of the patient's skin, and selects a set of machine learning diagnostic models from a plurality of sets of candidate machine learning diagnostic models based on the skin tone measurement of the patient, where each of the sets of candidate machine learning diagnostic models is trained to receive the concern image and output a diagnosis of a condition of the patient.

The figures depict various embodiments of the present invention for purposes of illustration only. One skilled in the art will readily recognize from the following discussion that alternative embodiments of the structures and methods illustrated herein may be employed without departing from the principles of the invention described herein.

<FIG> is an exemplary block diagram of system components in an environment for utilizing a skin image calibration tool, in accordance with one embodiment. Environment <NUM> includes imaging device <NUM>, network <NUM>, skin image calibration tool <NUM>, care selection tool <NUM>, and disease diagnosis tool <NUM>. Imaging device <NUM> may be any device configured to capture an image of an affected area of a patient. An example that pervades this disclosure is skin image calibration and skin disease diagnosis; however, the imaging, calibration, and diagnosis aspects disclosed herein may apply to any other organ of the human body (e.g., the retina). Imaging device <NUM> may be specially configured to capture images of a patient's skin. Alternatively, imaging device <NUM> may be a generic client device, such as a smartphone, laptop, tablet, or other camera-equipped computing device, with software (e.g., an application) installed thereon for processing the image in the manners disclosed herein. In an embodiment, imaging device <NUM> may be a generic client device that captures images and transmits the images to a remote server (e.g., skin image calibration tool <NUM>), where processing and calibration of the image is performed by the remote server.

Network <NUM> may be any communications network, such as a local area network, a wide area network, the Internet, and the like. Alternatively, or additionally, network <NUM> may represent on-device activity (such as in a scenario where some or all functionality of skin image calibration tool <NUM>, care selection tool <NUM>, and/or disease diagnosis tool <NUM>, were installed on-board imaging device <NUM>).

Skin image calibration tool <NUM> receives a base skin tone image of a patient, and determines a base skin tone of the patient from the base skin tone image. The term base skin tone image, as used here, in may refer to an image captured that satisfies certain criteria, or was captured based on instructions that the image should satisfy certain criteria. For example, a base skin tone image, in an embodiment, is an image that should show healthy, untanned skin. The term base skin tone, as used herein, may refer to a value representative of a pigmentation (or tone, as interchangeably used herein) of a patient's skin. Further details on the mechanics of how skin image calibration tool <NUM> determines the base skin tone are described in further detail below with respect to <FIG>. While depicted as being opposite network <NUM> in <FIG>, skin image calibration tool <NUM> may, in part or in whole, be installed on imaging device <NUM>, and/or in a device local to imaging device <NUM>.

Care selection tool <NUM> receives input (e.g., a determined base skin tone), and outputs a care decision. For example, based on a base skin tone determined by skin image calibration tool <NUM>, care selection tool <NUM> may select a machine learning model that is best optimized for diagnosing a concern image. Disease diagnosis tool <NUM> performs a diagnosis based on its inputs. For example, disease diagnosis tool <NUM> may take a concern image as input, and may output a diagnosis. Further details of how care selection tool <NUM> and disease diagnosis tool <NUM> perform these and other operations is disclosed in further detail below with respect to <FIG>. Wherever a classifier or machine learning diagnostic model is mentioned herein, the performance of inputting data into such a model and obtaining output from the model may be performed by disease diagnosis tool <NUM>. Like skin image calibration tool <NUM>, while depicted as being opposite network <NUM> in <FIG> from each other element depicted in <FIG>, care selection tool <NUM> and/or disease diagnosis tool <NUM> may be, in part or in whole, installed on imaging device <NUM> and/or in a device local to imaging device <NUM>. Functionality of skin image calibration tool <NUM>, care selection tool <NUM>, and disease diagnosis tool <NUM>, may be consolidated on a single server or group of servers, and/or may be distributed across many servers.

<FIG> is an exemplary block diagram of modules and components of a skin image calibration tool, in accordance with one embodiment. As depicted in <FIG>, skin image calibration tool <NUM> includes various modules, such as image capture module <NUM>, base skin tone determination module <NUM>, and adapted imaging profile generation module <NUM>. Skin image calibration tool <NUM> is also depicted as including various databases, such as reference calibration profile <NUM>, skin tone images <NUM>, and base skin tone classifier <NUM>. The modules and databases depicted are merely exemplary, and more or fewer modules and/or databases may be used to achieve the functionality disclosed herein.

Image capture module <NUM> captures a base skin tone image of a patient's skin for further processing. In order to capture the image, image capture module <NUM> first acquires image data from a camera of imaging device <NUM>. The acquired image data may be completely uncalibrated and thus not yet be synthesized into an image perceptible by a human being. This RAW image data often effectively represents photon intensity received per pixel in the bayer pattern requires further post processing, such as debayering and color correction in order to be converted into a calibrated image. Image capture module <NUM> may apply a reference calibration profile (e.g., as retrieved from reference calibration profile <NUM>, or from memory of imaging device <NUM>) to the acquired image data in order to capture the base skin tone image. Imaging device <NUM> may be calibrated to be within set tolerances, including but not limited to, colorimetric accuracy (conventionally described in terms of Delta E, measuring change in visual perception of two given colors) and optical properties, such as relative lens positions. This calibration may be implemented in any number of ways, including, for example, camera settings (such as exposure or channel gains), color transformation profiles (such as ICC profiles) and/or transformations based on spatially relevant points (such as which pixels are located within the camera's effective field of view). These calibrations may be performed by the factory when creating imaging device <NUM>, and/or may be instructed by imaging device <NUM>. The resulting calibration of imaging device <NUM> forms the reference calibration profile.

In an embodiment, image capture module <NUM>, prior to acquiring image data, may cause an output of instructions to be provided (e.g., displayed and/or audio instructions) to an operator of imaging device <NUM>. The operator may be any human being, such as a medical assistant, a minimally-trained operator, or the patient. The instructions may include instructions to take a photograph of skin having one or more pre-defined characteristics, such as the skin being on the inner-wrist, the skin being healthy, the skin being untanned, the skin being a consistent tone without any artifacts, such as a mole, being present, and so on. Image capture module <NUM> may perform processing on the base skin tone image to verify whether the skin photographed complies with the pre-defined characteristics and whether the image itself is of sufficient quality. For example, pattern recognition may be performed, or the base skin tone image may be input into a classifier that is trained to output a label of whether the base skin tone image is compliant with the instructions.

Image capture module <NUM> may capture a plurality of base skin tone images. Image capture module <NUM> may prompt the operator to capture several base skin tone images. For example, a patient may not have uniform skin tone across the patient's body, and therefore, multiple skin tone images may be used to obtain a more robust data set for determining the base skin tone of the patient. Image capture module <NUM> may store base skin tone images in skin tone images database <NUM>. Skin tone image database <NUM> may store base skin tone images in association with an electronic health record of a patient that includes other information about the patient, including biographic information, demographic information, and any other information describing the patient.

Base skin tone determination module <NUM> determines a base skin tone for a patient based on one or more base skin tone images. Base skin tone determination module <NUM> may determine the base skin tone in any of a variety of manners. In an embodiment, base skin tone determination module <NUM> may determine a numerical representation of each pixel of the calibrated base skin tone image. For example, each color of each pixel may correspond to a number on a scale. Base skin tone determination module <NUM> may generate an aggregate representation by performing a statistical operation based on each numerical representation. For example, base skin tone determination module <NUM> may average the numerical representation. Where multiple base skin tone images are used, base skin tone determination module <NUM> may repeat this activity for each base skin tone image, and then perform a statistical operation on each of the aggregate representations, resulting in an aggregate representation of the entire set of base skin tone images. Base skin tone determination module <NUM> may then map, using the scale, the aggregate representation to a point in a color space, and may assign this point as the base skin tone. Alternatively, rather than immediately assigning this point as the base skin tone, base skin tone determination module <NUM> may map the point in color space into a discrete value in a classification system, such as the Fitzpatrick system, or any other classification system. This embodiment of base skin tone determination may be sufficient in many cases, but it may not account for pigmentation inconsistencies within the reference area, such as hypopigmented or hyperpigmented skin patches, so other measurement means may be performed.

In an embodiment, base skin tone determination module <NUM> may input the base skin tone image(s) into base skin tone classifier <NUM>. Base skin tone classifier <NUM> may be a machine learning model that is trained to generate a discrete value from a set of images. For example, in order to train base skin tone classifier <NUM>, expert graders may assign a value to each image in a training set. The labels may indicate a precise skin tone of a patient. In an embodiment, the labels may additionally, or alternatively, indicate an attribute of the skin tone (e.g., that the skin is hypopigmented or hyperpigmented). Thus, base skin tone classifier <NUM> may output the base skin tone of the patient.

Alternatively, or additionally, base skin tone classifier <NUM> may output a type of skin indicated (e.g., hypopigmented or hyperpigmented skin, or normal skin). Base skin tone determination module <NUM> may determine, based on the output of base skin tone classifier <NUM>, that the aforementioned aggregate representation method is sufficient responsive to receiving output from the classifier that skin is normal. Responsive to receiving output from the classifier that the skin is of a certain type (e.g., hypopigmented or hyperpigmented), base skin tone determination module <NUM> may determine to use a measurement output by base skin tone classifier <NUM> as the base skin tone. Example classifier algorithms for base skin tone classifier <NUM> may include convolutional neural networks, support vector machines on extracted features, and XGBoost on extracted features. As another example, a segmentation algorithm could be trained to separate healthy, untanned skin from discolored skin, and then either the basic approaches described above, or a different classifier could be used solely on the healthy patches of skin to determine the base skin tone. Example segmentation algorithms include fully convolutional neural networks, Otsu method, and thresholding the color variance.

Adapted imaging profile generation module <NUM> generates an adapted imaging profile. The term adapted imaging profile, as used herein, may refer to imaging parameters that are optimized to acquire a skin concern image that is most useful for diagnosis, relative to a skin concern image that would be acquired using the reference calibration profile alone. Adapted imaging profile generation module <NUM> may generate the adapted imaging profile in a number of ways. In an embodiment, adapted imaging profile generation module <NUM> may determine, based on a data structure such as a mapping table, a set of predefined imaging parameters stored in reference calibration profile database <NUM> as mapping to the determined base skin tone. In another embodiment, instead of basing the imaging parameters on a data structure such as a mapping table, the adapted imaging profile generation module <NUM> would use the base skin tone to adjust the imaging parameters as a difference from base values using the numerical representation of skin tone. An example of this could include increasing the exposure proportional to the darkness of the patients skin to maximize detail. A more complex example could involve interpolating between the values of known profiles to choose intermediate values for skin tones that fall between the range of two explicitly listed profiles. Adapted imaging profile generation module <NUM> may store the adapted imaging profile in reference calibration profile database <NUM>.

<FIG> is an exemplary block diagram of modules and components of a care selection tool, in accordance with one embodiment. As depicted in <FIG>, care selection tool <NUM> includes various modules, such as concern image processing module <NUM>, suitability module <NUM> and classifier selection module <NUM>. Care selection tool <NUM> also includes various databases, such as diagnosis model inventory <NUM> and patient information <NUM>. The modules and databases depicted with respect to <FIG> are merely exemplary; fewer or more modules and/or databases may be used to achieve the functionality of care selection tool <NUM> described herein.

Concern image processing module <NUM> receives a skin concern image and may calibrate the skin concern image using the adapted imaging profile. Calibration of the skin concern image occurs in the same manner as calibration of the base skin tone image, except that the adapted imaging profile is used instead of the reference calibration profile.

Suitability module <NUM> determines whether the skin concern image is suitable for diagnosis. Suitability may be based on any predefined criteria. For example, a regulatory agency may indicate that some classifications (e.g., Fitzpatrick system classifications <NUM>-<NUM>) are suitable for diagnosis using the systems and methods disclosed herein, but other classifications are not suitable for diagnosis using the systems and methods disclosed herein (e.g., darker skin tones may experience a higher error rate in diagnosing whether a mole is benign). As another example, the skin concern image may fail a quality parameter that must be met for processing (e.g. the skin concern image is over-exposed, under-exposed, does not show enough of the affected skin area, or any other predefined parameter). Additional example validity criteria include no color channel saturation across the image (a channel being at maximum value and thus not being able to provide an accurate measurement would indicate failure), or passing a machine learning based quality checker. Image quality issues can be screened using a convolutional neural network classifier trained on examples of images with sufficient quality and images with insufficient quality. Additionally, image quality issues can be screened for using features extracted from the image, these features may include measures of the high frequency content of the images, and a histogram of the colors.

Responsive to determining that the skin concern image is not suitable for diagnosis, suitability module <NUM> may cause a prompt to be output to the operator indicating that the skin concern image is not suitable for diagnosis. In an embodiment where the skin concern image is not suitable for diagnosis due to a quality issue, suitability module <NUM> may prompt the operator to capture another skin concern image satisfying the quality parameter. Where the recaptured image is taken, in an embodiment, suitability module <NUM> may instruct that the reference calibration profile be used to generate the recaptured skin concern image. In such an embodiment, where the recaptured image still does not satisfy a quality parameter, suitability module <NUM> may prompt the operator to manually check the images for errors, to manually check for signs of a camera malfunction, to yet again re-take the skin concern image, and/or to re-take the base skin tone image (which may trigger re-generation of the adapted imaging profile using the retaken base skin tone image, where the re-generated adapted imaging profile may be used to calibrate further skin concern images for the patient). In an embodiment where the skin concern image is not suitable due to a characteristic of the patient, suitability module <NUM> may prompt the operator to instruct the patient that a diagnosis cannot be completed. Additional information may be prompted to the operator as well (e.g., instructions for the patient to obtain diagnosis by a medical doctor).

Classifier selection module <NUM> selects one or more classifiers, such as a set of machine learning diagnostic models, based on the base skin tone of the patient, the selected classifier to be used to establish care of the patient. The set may include one or more machine learning models and may additionally include heuristics that, together, form a workflow for care of the patient. A plurality of classifiers may be stored in diagnosis model inventory <NUM>. Each of these classifiers may be associated with criteria that, if satisfied, indicate to classifier selection module <NUM> that the classifier is to be used for diagnosis of the patient. Classifier selection module <NUM> may select a classifier based on information additional to the base skin tone, the additional information being retrieved from patient information database <NUM>. The additional information may include any information of the patient's electronic patient record, such as biographical information, demographic information, and/or any other information descriptive of the patient.

In an embodiment, classifier selection module <NUM> may perform activity beyond selecting one or more particular classifiers, or may perform activity in connection with obtaining further inputs for particular classifiers. For example, classifier selection module <NUM> may determine that the base skin tone is a high-risk skin tone (e.g., as is indicated in a database that maps base skin tones to risk categories). For example, the base skin tone may correspond to a very light skinned person who has higher incidences of skin cancer. Responsively, classifier selection module <NUM> may prompt inputs of answers to questions that are relevant to informing care (e.g., the patient may be prompted to answer questions about the patient's sun exposure behavior). As another example, classifier selection module <NUM> may prompt that further skin concern images be captured (e.g., palms of the hands and soles of the feet where the patient has darker skin tones that cause those patients to be more prone to dangerous growths in those locations).

In an embodiment, classifier selection module <NUM> may have a number of classifiers available in diagnosis model inventory <NUM>, each of the available classifiers corresponding to a different range of skin tones. Classifier selection module <NUM> may select a classifier based on the base skin tone of the patient. The selected classifier may be trained to be optimized for its corresponding range of skin tones, and may be less effective or ineffective where a patient has a base skin tone outside of that range. The selected classifier may take the concern image as input, with or without other information such as the base skin tone and/or additional patient information, and may output a diagnosis.

In an embodiment, classifier selection module <NUM> is not used as a gatekeeper to select a classifier. Instead, care selection tool <NUM> inputs the base skin tone and the concern image into the classifier (possibly with additional patient information), and outputs a diagnosis. As another example embodiment in this vein, a classifier may be used that takes the skin concern image as input, and outputs probabilities of a diagnosis. Care selection tool <NUM> then weights the probabilities based on the base skin tone of the patient.

Where a classifier outputs a diagnosis, the diagnosis may be a direct diagnosis (e.g., the concern image indicates that no disease is present, or a particular identification of a disease that is present). The diagnosis may, alternatively, be a plurality of probabilities, each probability corresponding to a different candidate diagnosis, the probability indicating the likelihood that the disease corresponds to each candidate diagnosis. Classifier selection module <NUM> may output the probabilities directly to the operator. In an embodiment, classifier selection module <NUM> may cause the candidate diagnosis with the highest probability to be displayed to the operator. Classifier selection module <NUM> may compare each probability to a threshold, and may limit candidate diagnoses displayed to the operator to those candidate diagnoses whose corresponding probabilities exceed the threshold. Each different candidate diagnosis may have a different corresponding threshold that must be crossed for the candidate diagnosis to be displayed to the operator.

Care selection tool <NUM> may determine, based on two or more candidate diagnoses having a probability that exceeds a threshold being output by a classifier, that at least two diseases cannot be ruled out as being depicted by the skin concern image. Accordingly, care selection tool <NUM> may prompt the operator to instruct the patient to obtain an expert opinion (e.g., from a medical doctor). Care selection tool <NUM> may withhold the two or more candidate diagnoses, or may cause the two or more candidate diagnoses to be indicated.

While skin is a primary example, care selection tool <NUM> may apply care selection based on any patient attributes. In an embodiment, a model may be trained to classify a patient based on any condition (e.g., skin tone, weight, gender, jaundice, diabetes, and/or any other condition of the patient). Care selection tool <NUM> may determine, based on output of the model, further models to be used to determine care for the patient (e.g., a precise model trained to determine whether a skin condition associated with diabetes is present in the patient).

In an embodiment, care selection tool <NUM> may receive biometric data corresponding to a patient. The biometric data may include a visual image such as the images discussed herein and any other type of image (whether using a camera lens or a spectrum that captures internal organs, such as a sonogram), and may be an image in non-visible wavelengths (e.g., infrared, ultraviolet, x-ray, and so on). The biometric data may be any other data about the patient, or a combination of different types of data. Exemplary biometric data includes measurements derived from bloodwork, cardiac activity (e.g., as measured by an electrocardiogram), pulse data, and any other data describing biometric activity of the patient. Care selection tool <NUM> may input the biometric data into a classifier trained to obtain an attribute of the patient. For example, bloodwork data may be input into the classifier, and the classifier may output whether the bloodwork indicates the patient has one or more conditions, such as a disease (e.g., diabetes).

Care selection tool <NUM> may select a set of machine learning models from a plurality of sets of candidate machine learning diagnostic models based on the attribute of the patient, where each of the sets of candidate machine learning diagnostic models are trained to receive the attribute and output a diagnosis of a condition of the patient. For example, a set of machine learning models corresponding to a particular disease, such as diabetes, might include machine learning models that take, in addition to the disease itself, other inputs (e.g., retinal images, images of toes) to obtain outputs relating to the disease (e.g., determination of whether the patient has diabetic retinopathy based on a retinal image, determination of whether the patient has a toe condition associated with diabetes, and so on). The operator might be prompted to obtain additional information about the patient, or to instruct the patient to obtain the other inputs, by visiting other operators (e.g., medical doctors) to determine additional information that might inform the selected set of machine learning models.

<FIG> is a block diagram illustrating components of an example machine able to read instructions from a machine-readable medium and execute them in a processor (or controller). Specifically, <FIG> shows a diagrammatic representation of a machine in the example form of a computer system <NUM> within which program code (e.g., software) for causing the machine to perform any one or more of the methodologies discussed herein may be executed. The program code may be comprised of instructions <NUM> executable by one or more processors <NUM>. In alternative embodiments, the machine operates as a standalone device or may be connected (e.g., networked) to other machines. In a networked deployment, the machine may operate in the capacity of a server machine or a client machine in a server-client network environment, or as a peer machine in a peer-to-peer (or distributed) network environment.

The machine may be a server computer, a client computer, a personal computer (PC), a tablet PC, a set-top box (STB), a personal digital assistant (PDA), a cellular telephone, a smartphone, a web appliance, a network router, switch or bridge, or any machine capable of executing instructions <NUM> (sequential or otherwise) that specify actions to be taken by that machine. Further, while only a single machine is illustrated, the term "machine" shall also be taken to include any collection of machines that individually or jointly execute instructions <NUM> to perform any one or more of the methodologies discussed herein.

The example computer system <NUM> includes a processor <NUM> (e.g., a central processing unit (CPU), a graphics processing unit (GPU), a digital signal processor (DSP), one or more application specific integrated circuits (ASICs), one or more radio-frequency integrated circuits (RFICs), or any combination of these), a main memory <NUM>, and a static memory <NUM>, which are configured to communicate with each other via a bus <NUM>. The computer system <NUM> may further include visual display interface <NUM>. The visual interface may include a software driver that enables displaying user interfaces on a screen (or display). The visual interface may display user interfaces directly (e.g., on the screen) or indirectly on a surface, window, or the like (e.g., via a visual projection unit). For ease of discussion the visual interface may be described as a screen. The visual interface <NUM> may include or may interface with a touch enabled screen. The computer system <NUM> may also include alphanumeric input device <NUM> (e.g., a keyboard or touch screen keyboard), a cursor control device <NUM> (e.g., a mouse, a trackball, a joystick, a motion sensor, or other pointing instrument), a storage unit <NUM>, a signal generation device <NUM> (e.g., a speaker), and a network interface device <NUM>, which also are configured to communicate via the bus <NUM>.

The storage unit <NUM> includes a machine-readable medium <NUM> on which is stored instructions <NUM> (e.g., software) embodying any one or more of the methodologies or functions described herein. The instructions <NUM> (e.g., software) may also reside, completely or at least partially, within the main memory <NUM> or within the processor <NUM> (e.g., within a processor's cache memory) during execution thereof by the computer system <NUM>, the main memory <NUM> and the processor <NUM> also constituting machine-readable media. The instructions <NUM> (e.g., software) may be transmitted or received over a network <NUM> via the network interface device <NUM>.

While machine-readable medium <NUM> is shown in an example embodiment to be a single medium, the term "machine-readable medium" should be taken to include a single medium or multiple media (e.g., a centralized or distributed database, or associated caches and servers) able to store instructions (e.g., instructions <NUM>). The term "machine-readable medium" shall also be taken to include any medium that is capable of storing instructions (e.g., instructions <NUM>) for execution by the machine and that cause the machine to perform any one or more of the methodologies disclosed herein. The term "machine-readable medium" includes, but not be limited to, data repositories in the form of solid-state memories, optical media, and magnetic media.

<FIG> depicts exemplary images showing a base skin tone image and a skin concern image, in accordance with one embodiment. Base skin tone image <NUM> is shown being captured by an operator, who as depicted, is not the patient. The imaging device used is shown to be a smartphone camera. The imaging device is shown to display the reference skin tone of the patient.

Skin concern image <NUM> is shown being captured by the operator. Skin concern image <NUM> includes an image of a mole. Skin concern image <NUM> is calibrated using an adapted reference profile, which results in a skin concern image that, when input into a classifier for diagnosis, optimizes the accuracy of the results of the diagnosis.

<FIG> depicts an exemplary flow chart for determining a diagnosis based on a base skin tone of a patient, in accordance with one embodiment. Process <NUM> may begin with skin image calibration tool <NUM> receiving <NUM> a base skin tone image of a patient. The base skin tone image may be received by image capture module <NUM>, which may be executed by processor <NUM>. The base skin tone image may be base skin tone image <NUM>. Skin image calibration tool <NUM> generates <NUM> a calibrated skin tone image (e.g., using image capture module <NUM>) by calibrating the base skin tone image using a reference calibration profile (e.g., as retrieved from reference calibration profile database <NUM>).

Skin image calibration tool <NUM> determines <NUM> a base skin tone of the patient based on the calibrated base skin tone image (e.g., using base skin tone determination module <NUM>). Care selection tool <NUM> receives <NUM> a concern image of a portion of the patient's skin (e.g., as depicted in concern image <NUM>). Care selection tool <NUM> selects <NUM> (e.g., using classifier selection module <NUM>) a set of machine learning diagnostic models from a plurality of sets of candidate machine learning diagnostic modules based on the base skin tone of the patient, each of the sets of candidate machine learning models trained to receive the concern image and output a diagnosis of a condition of the patient.

The foregoing description of the embodiments of the invention has been presented for the purpose of illustration; it is not intended to be exhaustive or to limit the invention to the precise forms disclosed. Persons skilled in the relevant art can appreciate that many modifications and variations are possible in light of the above disclosure.

Some portions of this description describe the embodiments of the invention in terms of algorithms and symbolic representations of operations on information. These algorithmic descriptions and representations are commonly used by those skilled in the data processing arts to convey the substance of their work effectively to others skilled in the art. These operations, while described functionally, computationally, or logically, are understood to be implemented by computer programs or equivalent electrical circuits, microcode, or the like. Furthermore, it has also proven convenient at times, to refer to these arrangements of operations as modules, without loss of generality. The described operations and their associated modules may be embodied in software, firmware, hardware, or any combinations thereof.

Embodiments of the invention may also relate to an apparatus for performing the operations herein. This apparatus may be specially constructed for the required purposes, and/or it may comprise a general-purpose computing device selectively activated or reconfigured by a computer program stored in the computer. Such a computer program may be stored in a non-transitory, tangible computer readable storage medium, or any type of media suitable for storing electronic instructions, which may be coupled to a computer system bus. Furthermore, any computing systems referred to in the specification may include a single processor or may be architectures employing multiple processor designs for increased computing capability.

Embodiments of the invention may also relate to a product that is produced by a computing process described herein. Such a product may comprise information resulting from a computing process, where the information is stored on a non-transitory, tangible computer readable storage medium and may include any embodiment of a computer program product or other data combination described herein.

Finally, the language used in the specification has been principally selected for readability and instructional purposes, and it may not have been selected to delineate or circumscribe the inventive subject matter. It is therefore intended that the scope of the invention be limited not by this detailed description, but rather by any claims that issue on an application based hereon. Accordingly, the disclosure of the embodiments of the invention is intended to be illustrative, but not limiting, of the scope of the invention, which is set forth in the following claims. processing arts to convey the substance of their work effectively to others skilled in the art. These operations, while described functionally, computationally, or logically, are understood to be implemented by computer programs or equivalent electrical circuits, microcode, or the like. Furthermore, it has also proven convenient at times, to refer to these arrangements of operations as modules, without loss of generality. The described operations and their associated modules may be embodied in software, firmware, hardware, or any combinations thereof.

Claim 1:
A system for determining a diagnosis of a patient using machine learning, the system configured to:
receive biometric data corresponding to a patient;
input the biometric data into a classifier trained to obtain an attribute of the patient;
receive, as output from the classifier, the attribute of the patient; and
select a set of machine learning diagnostic models from a plurality of sets of candidate machine learning diagnostic models, each of the sets of candidate machine learning diagnostic models trained to receive the attribute and output a diagnosis of a condition of the patient;
wherein the system is characterised in that
the system comprises
an image capture module (<NUM>) for
receiving (<NUM>) a base skin tone image (<NUM>) of a patient, and for
generating (<NUM>) a calibrated base skin tone image by calibrating the base skin tone image using a reference calibration profile (<NUM>);
a base skin tone determination module (<NUM>) for determining (<NUM>) a base skin tone of the patient based on the calibrated base skin tone image;
a concern image processing module (<NUM>) for receiving (<NUM>) a concern image (<NUM>, <NUM>) of a portion of the patient's skin; and
a classifier selection module (<NUM>) for selecting (<NUM>) a set of machine learning diagnostic models from a plurality of sets of candidate machine learning diagnostic models, each of the sets of candidate machine learning diagnostic models trained to receive the concern image (<NUM>) and output a diagnosis of a condition of the patient; wherein
the classifier selection module selects the set of machine learning models based on the attribute of the patient, wherein the attribute is the base skin tone.