Patent ID: 12191037

DETAILED DESCRIPTION

Terms used in the present specification will be described in brief and the present invention will be described in detail.

Terms used in the present invention adopt general terms which are currently widely used as possible by considering functions in the present invention, but the terms may vary depending on an intention of those skilled in the art, a precedent, emergence of new technology, etc. Accordingly, the terms used in the present invention should be defined based on not just a name of the term but a meaning of the term and contents throughout the present invention.

Throughout the specification, when any part “comprises” any component, the part may further include other components instead of excluding other components unless specifically stated otherwise.

An embodiment of the present invention will be described more fully hereinafter with reference to the accompanying drawings so as to be easily implemented by those skilled in the art. However, the present invention may be embodied in many different forms and is not limited to embodiments described herein.

Specific matters including problems to be solved for the present invention, solutions of the problems, and the effects of the invention for the present invention are included in embodiments and drawings to be described below. Advantages and features of the present invention, and methods for accomplishing the same will be more clearly understood from embodiments described in detail below with reference to the accompanying drawings.

Hereinafter, the present invention will be described in more detail with reference to the accompanying drawings.

As illustrated inFIG.1, a medical equipment learning system of the present invention is largely configured by including a data extraction module100, a visualization module200, a pre-processing module300, a learning module400, a prediction module500, and a storage module600.

First, the data extraction module100serves to extract necessary information from medical data110to generate text-type data120. A subject to be extracted may be a clinical document received via USB or e-mail, as well as a hospital, or electronic medical record system materials stored in the hospital. These materials may be stored in document forms or stored in a database as materials. When a patient visits as an outpatient or is hospitalized several times, the medical materials may be prepared according to the patient, and thus the data extraction module100serves to extract the data in a required form. Further, the data extraction module100may extract the data from materials received from other hospitals or even from materials stored in a server outside the hospital, materials stored in a personal device, or materials received from various medical devices.

The text-type data120means text-type data included in the medical data110, such as disease names or diagnosis names, symptoms, blood test results, reading papers, surgical names, nursing records, and nursing measures, as data acquired from the medical data110represented as clinical records, electronic medical records, progress recodes, discharge summaries, medical terminologies, or other many text types or number types.

The text-type data120is not limited to a diagnosis name, and the text-type data120may include data defined in anatomical sites, procedure names, measured blood pressure values, and the activity of a patient of a massage medical person or a medical assistant, or various text-type materials indicating patient's conditions such as “serious”, “light”, “large”, and “small”. For example, the text-type data120may be Korean or English characters such as “fatty liver”, “ankle pain”, and “heart failure”, or standardized data or medical term codes such as “K76.0”, “61515”, “N05”, and “M51.0”, which are numbers or combinations of characters and numbers. The standardized medical term code refers to a range in which medical concepts are presented in SNOMED-CT, ICD-9, ICD-10, ICD-11, LOINC, CPT, ATC, RxNorm, ICNP, NMDS, and the like. In addition, a test result of a hemoglobin level of 10.3 gram/deciliter may be data expressed by numbers.

FIG.16is an embodiment illustrating that data required for extracting the text-type data120, such as a medical record document, are stored in the electronic medical record system as separate items. In this case, the text-type data120may be extracted by accessing the database and reading only required items. In addition, in the data stored in the form of a document, as illustrated inFIG.17, data in the form of JSON or XML may be classified for each diagnosis name or each symptom, and at this time, the required items may be read and taken.

In addition, when the text-type data120is free-text data that is not organized into separate items, unstandardized character strings, or data in the form of binary large object (BLOB) data in a database, as illustrated inFIG.18, a list of the text-type data120may be specified for the required items, and required values may be extracted. The data extraction module100is configured to collect information from data scattered in various hospitals and distributed to external servers.

Next, the visualization module200generates the text-type data120collected by the data extraction module100as visualization data. More specifically, the acquired text-type data120is converted into image-type data210. At this time, the image-type data210is a predetermined 3D model, and may be generated by combining one or more medical information models212with a basic model211which is a 3D model.

As illustrated inFIGS.2and3, the information representing the medical data110is expressed as the image-type data210, which is an image including the medical data110. The image-type data210includes the medical information model212representing the text-type data120acquired from the medical data110and an image (basic model211) which is an image expressing the human body. The medical information model212may be expressed in more detail by dots, lines, areas, volumes, or various shapes or combinations thereof, and may be expressed as a 2D model as well as a 3D model.

As illustrated inFIG.4, the basic model211may include the plurality of medical information models212to provide the image-type data210. The plurality of medical information models212that have been previously prepared and stored are referred to the text-type data120extracted from the medical data110and used as a medical information model212of a certain patient.

The basic model211may also be an empty space in which no picture is drawn, and at this time, the visualization module200may express the medical information model212in the empty space. Equipment learning may be performed even if the basic model is the empty space. In addition, the image-type data210may be a whole body and may represent only some systems of the human body, such as a digestive system or a respiratory system, or may represent only a certain area of the body, such as a left leg or a head.

The medical information model212may be a model of entire organ or part of an organ. For example, in the case of a tumor occurring from the liver, the entire liver organ may be expressed as the medical information model212or a portion where the tumor occurs, that is, a segment (e.g., posterior lateral segment) that is a part of the liver. Alternatively, the medical information model212may also be expressed as a shape itself in which the tumor occurs.

In addition, the medical information model212may be added or drawn directly to be imaged by a user without being extracted from the medical data110. In the medical information model212, a patient or a medical person may directly draw a painful spot, or express a site with a spot, an itchy site, a site where a blood pressure is checked, a site where a nail is cut, and a site to be injected. For example, when the patient has a fatty liver, the medical person may directly draw the basic model211in the background without recording the medical data110. Alternatively, as illustrated inFIG.4, the medical person or the patient may select one of the medical information models212prepared and stored in advance to express a disease condition of a patient.

The visualization module200varies the color, brightness, or transparency of the image according to the name of the disease, the severity of the disease, the chronicity, and the degree of malignancy. The visualization module200may determine the color of the image by selecting or combining any one or more of red, green, and blue colors. In one embodiment, the patient may have a paralysis of the tibialis anterior muscle. If the muscle strength of the corresponding muscle is 20% of the normal, an R value representing the red color among RGB channels which are colors expressing the muscle strength may represent the muscle strength with 20% of the maximum value, and if the maximum value of the red channel is 255, the R value may be expressed as 255*0.2=50.5. On the other hand, the function of the kidney may be confirmed by an estimated glomerular filtration rate (eGFR) as one of the blood tests, and may be represented as 255 if the eGFR is 100 and 255/2 if the eGFR is 50 in conjunction with a G value representing the green value to the eGFR value. If the eGFR is 0, the G value may be represented as 0. That is, the patient's condition may be represented by changing and expressing the attributes of the image by a function of using a result value of the blood test as a factor. As such, the color may be defined as a value determined by a function of using clinical data as a factor.

As illustrated inFIG.6, the image-type data210may be completed in the medical information model212by adding a texture220to the basic model211in addition to general image attributes such as color, brightness, and transparency.

In addition, the visualization module200may express the texture220in the medical information model212according to a name of the disease, a medical term code, the chronicity, severity, and the degree of malignancy. For example, as illustrated inFIG.6, the stenosis may be expressed by a round pattern211, the squamous cell carcinoma may be expressed by a thin diagonal pattern222, the hemangioma may be expressed by a thick diagonal pattern223, and the paralysis may be expressed by a dotted diagonal pattern224. The patterns presented herein represent a few of examples of the texture220, and the texture220is not limited thereto and may be prepared by using a man-made figure, an icon representing a disease, or the like. In addition, the image-type data210may be expressed by applying various types of clinical data as well as the shape of the pattern.

In addition, an image extracted from a medical image, a photograph or image showing an anatomical pathology finding, an image to be photographed or extracted such as a skin disease photograph, and the like may be provided so as to be directly converted into the texture220. That is, all medical images may be applied to the medical information model212. For example, inFIG.7A, microscopic tissue findings may be used as the texture220, and typical pathological findings that may well express the patient's condition or a photograph of the corresponding patient may be taken directly.FIG.7Bis an image of photographing a skin lesion, and the image may be used as the texture220in a corresponding region and may also be a photograph of directly photographing a patient's skin.FIG.7Cis a part of an image photographed by an MRI. In addition,FIG.8illustrates a case in which a part of the CT image is taken to make the texture220of a patient with brain hemorrhage. As such, more various clinical data may be imaged, which may be used as equipment learning data.

InFIG.9, the visualization module200is characterized to use one or more layers according to the characteristics of the medical data110to further expand expression of information. When there are many concurrent diseases in the kidney, various disease conditions may be expressed in multiple layers which represent disease categories. As illustrated inFIG.9, diabetic kidney disease, infectious kidney disease, and neoplastic kidney disease may be simultaneously expressed and represented.

The visualization module200may further include the medical information model212that further expresses patient's diseases or symptoms which are not able to be anatomically expressed inside and outside the body shown in the basic model211. Based on the basic model211expressing the human body, there may be more medical information that cannot be expressed by the medical information model212. For example, the medical information is high blood pressure and diabetes. Of course, in the case of diabetes, the malfunction of the pancreas may be the cause, but when a relationship with the pancreas cannot be confirmed, it may be difficult to express information due to this pancrease condition.FIG.13illustrates an embodiment of the medical information model212that additionally shows a model of diabetes and hypertension outside the body to compensate for this.

FIG.19illustrates a form of visualizing data on a plurality of basic models211by the visualization module200, and illustrates an example of visualizing information classified by disease, symptom, and procedure in each of the basic models211as the medical information model212. In a manner in which data is separately recorded according to the classification of the text-type data120in the plurality of basic models211, in this case, there are advantages of managing the medical data110by type as needed and of being able to describe more accurately the medical data110.

Next, the pre-processing module300generates an input data set310for executing equipment learning based on the visualization data. The pre-processing module300generates the input data set310by processing the visualization data in a required form. The pre-processing module300may generate the input data set310by normalizing various types of image-type data210having various formats.

In the pre-processing of the data, when the image generated from the visualization data is a 2D image, data defined as one image in RGBA channels at a resolution of 50×50 may be defined as one input data set310. Alternatively, in the case of 3D data, an image created by applying the RGBA channels to each voxel at a resolution of 50×50×50 may be defined as one input data set310.

Meanwhile, the pre-processing module300may create a data set reflecting a change in data according to a change in time and provide the created data set as a learning material. That is, multiple image-type data210generated by the visualization module200may be generated as needed. For example, first data was generated from information on hospitals visited at the age of 20, second data was generated from information on hospitals visited at the age of 30, and third data was generated from information on hospitals visited at the age of 40, so that a total of three data were generated. The data made as temporal data or videos using the three data to reflect the passage of time may also be provided as the input data set310. That is, the pre-processing module300may learn a trend of the data according to a change in time point by combining materials prepared at various time points.

As illustrated inFIGS.10to12, the pre-processing module300may generate the input data set310by normalizing various types of image-type data210having various formats. The normalized input data set310is provided to the learning module400.

FIG.10is an embodiment of generating the input data set310by processing only a vascular system medical information model by the pre-processing module300among the image-type data210generated by the visualization module200. The pre-processing module300may convert the input data set310by selecting only a portion of the image-type data210.

FIG.11illustrates an embodiment of generating the input data set310with the 2D-converted data among the 3D-expressed image-type data210by the pre-processing module300.

FIG.12illustrates an embodiment of generating the input data set310with only a left shoulder and an upper arm while being converted into the 2D image among the 3D-expressed image-type data210by the pre-processing module300.

Next, the learning module400executes equipment learning on the input data set310generated by the pre-processing module300. The learning module400may include an algorithm that may be classified as equipment learning, such as a support vector equipment, a convolutional neural network, and a generative adversarial neural network.

Next, the prediction module500predicts data when the new image-type data210is input based on the results learned in the learning module400.

FIG.14illustrates an embodiment of learning life extension by using the learning module500. InFIG.14, a training image represents the input data set310input from the pre-processing module300. The first image is a condition in which a person cannot be moved well with a lesion in the left brain, diseases in the liver and the kidney, and a disease in the leg. The second image is an image with diseases of the liver and the kidney, and the third image is an image with diseases in only the legs. For each image, life expectancies of 3, 15, and 30 years were given as target values.

FIG.15is an embodiment of presenting a prediction value by receiving new image-type data210after learning by the learning module400. In the prediction module500, it is possible to present a prediction value for a life expectancy, such as 3.32 years.

By the technical solution, according to the present invention, it is possible to provide a medical AI learning system capable of avoiding disadvantages of existing text-type data and improving performance of AI by converting medical information expressed in texts into images and using the converted images for AI learning.

Further, according to the present invention, it is possible to normalize text-type materials which have different lengths while having ambiguous meanings.

Further, it is possible to provide medical information having a richer meaning by converting text-type data into image-type data.

Further, it is possible to implement a medical AI system capable of predicting precise and accurate diseases.

As described above, it will be understand to those skilled in the art that a technical configuration of the present invention can be easily executed in other detailed forms without changing the technical spirit or an essential feature thereof.

Therefore, the embodiments described as above are exemplary in all aspects and should be understood as not being restrictive and the scope of the present disclosure is represented by claims to be described below rather than the detailed description, and it is to be interpreted that the meaning and scope of the claims and all the changes or modified forms derived from the equivalents thereof come within the scope of the present invention.

EXPLANATION OF REFERENCE NUMERALS AND SYMBOLS

1. Medical equipment learning system100. Data extraction module110. Medical data120. Text-type data200. Visualization module210. Image-type data211. Basic model212. Medical information model220. Texture221. Round pattern222. Thin diagonal pattern223. Thick diagonal pattern224. Dotted diagonal pattern300. Pre-processing module310. Input data set400. Learning module500. Prediction module600. Storage module