Patent ID: 12216695

DESCRIPTION OF THE PREFERRED EMBODIMENTS

Hereinafter, preferred embodiments of the present invention will be described with reference to accompanying drawings.

Configuration Example of Medical Information System

An example of an information processing apparatus10that analyzes a text of an interpretation report, which is a kind of a medical text, will be described.FIG.1is a block diagram showing a configuration example of a medical information system200including the information processing apparatus10according to an embodiment. The medical information system200is formed as a computer network constructed in a medical institution such as a hospital. The medical information system200includes an electronic medical record system202, an examination order system203, a CT apparatus204, an MRI apparatus206, an image saving server210, a report server220, the information processing apparatus10, and a terminal apparatus230, and these elements are connected via the communication line240.

A server-type computer may be applied to each of the electronic medical record system202and the examination order system203. A form in which a plurality of computers cooperate with each other may be applied to the server-type computer. The communication line240may be a private communication line in a medical institution. Further, a part of the communication line240may include a wide-area communication line. Some of the elements of the medical information system200may be configured by cloud computing.

InFIG.1, the CT apparatus204and the MRI apparatus206are illustrated as examples of modalities. However, the apparatus that captures a medical image is not limited to the CT apparatus204and the MRI apparatus206, and may be various examination apparatuses such as an ultrasonic diagnostic apparatus, a positron emission tomography (PET) apparatus, a mammography apparatus, an X-ray diagnostic apparatus, an X-ray fluoroscopy diagnostic apparatus, and an endoscopic apparatus (not shown). There may be various combinations of types and the number of modalities connected to the communication line240for each medical institution.

The electronic medical record system202manages an electronic medical record for each patient. The electronic medical record system202includes an electronic medical record storage apparatus that stores the electronic medical record. The electronic medical record system202may store patient identification information and the electronic medical record in association with each other and search for the electronic medical record for each patient with the patient identification information as a parameter. The electronic medical record system202searches for the electronic medical record in response to a readout request transmitted from the terminal apparatus230, the information processing apparatus10, or the like, and transmits various types of information included in the electronic medical record corresponding to the readout request to the terminal apparatus230or the like, which is the request source.

The examination order system203manages an examination order issued based on an examination order request issued by a doctor. The examination order includes various types of information related to the examination, such as the patient identification information such as a patient identification (ID), identification information of a doctor in charge of check-up such as a doctor in charge ID, and a type of examination. The examination order system203comprises an examination order storage apparatus that stores the examination order.

The image saving server210may be, for example, a digital imaging and communications in medicine (DICOM) server that operates according to a specification of DICOM. The image saving server210is a computer that saves and manages various types of data including images captured by using various modalities such as the CT apparatus204and the MRI apparatus206, and comprises a large-capacity external storage apparatus and a program for database management. The image saving server210performs communication with another apparatus via the communication line240, and transmits and receives various types of data including image data. The image saving server210receives various types of data including the images generated by the modality such as the CT apparatus204via the communication line240, and saves and manages the received data in a recording medium such as the large-capacity external storage apparatus. A storage format of the image data and the communication between the apparatuses via the communication line240are based on a protocol of DICOM.

The report server220is a computer that saves and manages medical texts such as various reports including the interpretation report. The medical text includes a report on an image diagnosis result represented by the interpretation report, a text on the patient's medical record, and the like. In the present embodiment, the interpretation report is mainly described as an example, but a target text is not limited to the interpretation report. The description of the interpretation report can be read and understood as a text for various other purposes.

The report server220stores the electronic medical record, the medical image, and the interpretation report in association with each other. The report server220may comprise a program that supports the creation of the interpretation report. The report server220communicates with another apparatus via the communication line240, and transmits and receives various types of data such as the interpretation report.

The information processing apparatus10can acquire data from the report server220and the like via the communication line240. The information processing apparatus10processes text data described in the report, and performs text analysis such as classification of description information and extraction of terms. Details of processing functions of the information processing apparatus10will be described below. The information processing apparatus10can be formed by using hardware and software of a computer. The form of the information processing apparatus10is not particularly limited, and may be a server computer, a workstation, a personal computer, a tablet terminal, or the like. In the present embodiment, an example in which the information processing apparatus10and the report server220are separate apparatuses is described. However, a part or all of the processing functions of the information processing apparatus10may be incorporated into another computer such as the report server220.

The information processing apparatus10may comprise an input apparatus22and a display apparatus24. The input apparatus22may be, for example, a keyboard, a mouse, a multi-touch panel, another pointing device, a voice input apparatus, or an appropriate combination thereof. The display apparatus24may be, for example, a liquid crystal display, an organic electro-luminescence (OEL) display, a projector, or an appropriate combination thereof. The input apparatus22and the display apparatus24may be integrally configured as in the touch panel. The input apparatus22and the display apparatus24may be included in the information processing apparatus10, or the information processing apparatus10, the input apparatus22, and the display apparatus24may be integrally configured.

The information processing apparatus10can transmit a processing result of the text analysis to other apparatuses such as the report server220and the terminal apparatus230.

The terminal apparatus230may be a viewer terminal for image browsing, which is referred to as a picture archiving and communication systems (PACS) viewer or a DICOM viewer. Although one terminal apparatus230is illustrated inFIG.1, a plurality of terminal apparatuses230may be connected to the communication line240. A form of the terminal apparatus230is not particularly limited and may be a personal computer, a workstation, a tablet terminal, or the like. The terminal apparatus230comprises an input apparatus232and a display apparatus234. The input apparatus232and the display apparatus234may have the same configuration as the input apparatus22and the display apparatus24of the information processing apparatus10.

Various pieces of data saved in an image database of the image saving server210and various pieces of information including the processing result generated by the information processing apparatus10can be displayed on the display apparatus234of the terminal apparatus230.

The medical information system200may include an image processing apparatus (not shown). The image processing apparatus comprises an image processing program that performs image analysis on the medical image captured by the modality. For example, the image processing apparatus may be configured to perform analysis processing of various computer aided diagnoses (computer aided diagnosis, computer aided detection: CAD) or the like, such as processing of recognizing a lesion region or the like from an input image, processing of specifying a classification such as a disease name, or segmentation processing of recognizing a region of an organ, or may perform processing of supporting the creation of the interpretation report using an image processing result. The processing function of the image processing apparatus may be incorporated into the information processing apparatus10.

Outline of Processing Function of Information Processing Apparatus10

FIG.2is a functional block diagram showing an outline of the processing function of the information processing apparatus10. The information processing apparatus10comprises a text acquisition unit12, a description information classification unit14, a text analysis unit16, an analysis result output unit18, and a data saving unit20. The text acquisition unit12acquires a text to be processed. The text acquisition unit12may receive an input of the interpretation report saved in the report server220or the like, or may acquire a text input by the doctor using the input apparatus22. Further, the text acquisition unit12may receive a text from another apparatus such as the terminal apparatus230.

The text acquisition unit12may be configured to include a communication interface for receiving the interpretation report from an external apparatus such as the report server220, or may be configured to include a media interface for reading the interpretation report from a removable medium such as a memory card. Further, the text acquisition unit12may read out the text to be processed from the data saving unit20in the information processing apparatus10. The text acquisition unit12may be configured to include a text acquisition program for automatically acquiring the text from the data saving unit20or the external apparatus. The text acquired via the text acquisition unit12is transmitted to the description information classification unit14. Data such as the interpretation report input from the external apparatus such as the report server220is saved in the data saving unit20.

The description information classification unit14performs processing of classifying attributes of information described (hereinafter referred to as text description information) in a fixed unit for the input text. The fixed unit may be any unit of a sentence unit, a phrase unit, a word unit, or a character unit. The text of the interpretation report often includes a plurality of sentences, and each sentence is often a relatively short sentence. The fixed unit is preferably the sentence unit. The term “attribute” includes a concept of type. The attribute of the text description information includes, for example, one or more of a type of a human body part, a type of an organ, a type of a disease or a lesion, a type of a medical process, and the presence or absence of the disease or the lesion. The type of the medical process may include, for example, findings, diagnosis, past comparison, and message. In a case of handling the interpretation report, it is desirable that the description information classification unit14performs processing of classifying the type of the medical process and processing of classifying the type of the organ. A classification result by the description information classification unit14is saved in a classification result saving unit50of the data saving unit20.

The text analysis unit16analyzes the text for each of the same classifications based on the classification result by the description information classification unit14. The text analysis unit16includes a term extraction unit30and a relationship acquisition unit32and structures the text. The term extraction unit30acquires a term expression from the input text and determines a type (attribute) of each term. The relationship acquisition unit32acquires a relationship between the terms extracted by the term extraction unit30. The relationship acquisition unit32determines whether or not there is a relationship between a subject and an object from information of a subject term, an object term, and a span between these terms in the text. Each of the term extraction unit30and the relationship acquisition unit32may be configured to use a prediction model subjected to machine learning in advance to perform the term extraction processing or the relationship acquisition processing.

The analysis result output unit18outputs an analysis result by the text analysis unit16. The analysis result includes structured information indicating a structuring result. The analysis result output unit18converts the analysis result by the text analysis unit16into data in a format suitable for an output mode and outputs the data. The output mode may include display, transmission, saving, and the like. The analysis result output unit18may include a display control unit40and a communication unit42. The display control unit40controls the display of the display apparatus24. The display control unit40generates data for display applied to the display apparatus24and outputs the data for display to the display apparatus24. Accordingly, the structured information of the analysis result is displayed on the display apparatus24. Further, the display apparatus24can display the input text (unstructured text), information related to the classification result by the description information classification unit14, and the like.

The communication unit42generates data for communication using the communication line240and transmits the data to the external apparatus such as the report server220. Accordingly, the analysis result can be saved in the report server220, or the analysis result can be displayed on the display apparatus234of the terminal apparatus230or the like.

Hardware Configuration Example of Information Processing Apparatus10

FIG.3is a block diagram schematically showing a hardware configuration example of the information processing apparatus10. The information processing apparatus10can be formed by a computer system configured by using one or a plurality of computers. Here, an example will be described in which one computer executes the program to realize various functions of the information processing apparatus10.

The information processing apparatus10includes a processor102, a computer-readable medium104that is a non-transitory tangible object, a communication interface106, an input/output interface108, and a bus110.

The processor102includes a central processing unit (CPU). The processor102may include a graphics processing unit (GPU). The processor102is connected to the computer-readable medium104, the communication interface106, and the input/output interface108via the bus110. The processor102reads out various programs, data, and the like stored in the computer-readable medium104to execute various types of processing. The term program includes a concept of a program module and includes a command according to the program.

The computer-readable medium104is, for example, a storage apparatus including a memory122which is a main memory and a storage124which is an auxiliary memory. The storage124is configured by using, for example, a hard disk drive (HDD) apparatus, a solid state drive (SSD) apparatus, an optical disk, a magneto-optic disk, or a semiconductor memory, or an appropriate combination thereof. The storage124stores various programs, data, and the like. The storage124includes storage areas of the classification result saving unit50and an analysis result saving unit52, and can function as the data saving unit20(refer toFIG.2).

The memory122is used as a work area of the processor102, and is used as a storage unit that temporarily stores the program and various types of data read out from the storage124. The program stored in the storage124is loaded into the memory122, the processor102executes the command of the program, and thus the processor102functions as a unit that performs various types of processing defined by the program. The memory122stores programs such as a description information classification program140, a text analysis program160, a display control program180, and a communication control program182, which are executed by the processor102, various types of data, and the like.

The description information classification program140causes the processor102to realize the processing function as the description information classification unit14. The text analysis program160causes the processor102to realize the processing function as the text analysis unit16. The text analysis program160includes a term extraction program162and a relationship acquisition program164. The term extraction program162and the relationship acquisition program164cause the processor102to realize the processing functions as the term extraction unit30and the relationship acquisition unit32.

The display control program180causes the processor102to realize the processing function as the display control unit40. The communication control program182causes the processor102to realize a processing function of performing the communication with the external apparatus via the communication interface106.

The communication interface106performs the communication processing with the external apparatus in a wired manner or a wireless manner to exchange the information with the external apparatus. The information processing apparatus10is connected to the communication line240via the communication interface106(refer toFIG.1), and can exchange the data with the apparatuses such as the image saving server210, the report server220, and the terminal apparatus230. The communication interface106can play a role of the text acquisition unit12(FIG.2) that receives the input of the text data such as the interpretation report and the communication unit42that outputs the analysis result.

The input apparatus22and the display apparatus24are connected to the bus110via the input/output interface108.

Example of Operation of Information Processing Apparatus10

An example of an operation of the information processing apparatus10will be described with reference toFIGS.4to7.FIG.4is a flowchart showing an example of an information processing method executed by the information processing apparatus10. In step ST1, the processor102receives the input of the interpretation report and acquires the interpretation report. The processor102can read out the interpretation report, which is an unstructured text (free description text) saved in the report server220or the like, and display the readout interpretation report on the display apparatus24.FIG.5shows an example of a text described in the acquired interpretation report. The text shown inFIG.5is an example of a medical text.

Next, in step ST2ofFIG.4, the processor102determines the type of the medical process for each sentence included in the acquired text. Here, an example will be shown in which the type of the medical process is classified into three types of “findings”, “diagnosis”, and “other”. The “other” is a category in which those that do not correspond to any one of the findings or the diagnosis are classified. For example, the past comparison, the message, or the like corresponds to “other”. The processor102determines the type of the medical process in a sentence unit and may assign a label indicating the type of the medical process to each sentence. With such labeling, each sentence is classified in terms of the type of the medical process. The determination of the type of the medical process in a sentence unit is an example of classifying the attributes of the text description information in a fixed unit. The processing of step ST2is performed by the description information classification unit14.

FIG.6shows an example of a determination result in which the type of the medical process is determined in a sentence unit for the text of the interpretation report shown inFIG.5. As shown inFIG.6, each sentence of a text portion of “Compare with 10/15. CT<chest and abdomen simultaneously>” is a description of “other” and is classified as “other”. Each sentence of a text portion of “Irregular tubercle with diameter of 1.5 cm is found in left lung S6. With internal calcification. Bulla in right lung S3.” is a description of “findings” and is classified as “findings”. A sentence of “Tumor with diameter of 6 cm is found in liver S3, enhancement effect and washout are exhibited, and HCC is suspected.” includes a description of “findings” and a description of “diagnosis”. Thus, both a “findings” label and a “diagnosis” label may be assigned, or any one of the labels may be assigned. A sentence of “Enhanced tumor with diameter of 5 mm is also found in liver S4 and is persistent.” is a description of “findings” and is classified as “findings”. Each sentence of a text portion of “Hemangioma is suspected. Calcification myoma in uterus.” is a description of “diagnosis” and is classified as “diagnosis”. A sentence of “Please follow.” is a description of the message and is classified as “other”. The bulla is one of cystic lung diseases. The HCC is an abbreviation for Hepatocellular Carcinoma and means a liver cell cancer.

In a case of assuming a secondary use of the interpretation report, important description items in the text included in the interpretation report are the contents of the findings and the diagnosis, and it is desired to structure the text related to the findings and the diagnosis. Therefore, in a case where the type of the medical process is determined in a sentence unit, the processor102may collectively label the type of the medical process as “findings or diagnosis” without distinguishing between the “findings” and the “diagnosis”. Further, the processor102may exclude the text classified as “other” in the texts included in the interpretation report from a target of the analysis processing by the text analysis unit16based on the determination (classification) result of the type of the medical process, may set the text related to the finding or the diagnosis as the target of the analysis processing by the text analysis unit16.

Furthermore, in a case where the interpretation report, which is an unstructured text, is displayed on the display apparatus24or the like, the processor102may display the determination result (classification result) of the type of the medical process in an identifiable manner. In the example shown inFIG.6, a display color of characters is different such as displaying characters of the text portion classified as “other” in gray and displaying characters of the text portion classified as “findings” in black. In order to relatively reduce visibility of the text portion excluded from the target of the analysis processing, various display modes may be employed such as a mode in which each group of the same classification is surrounded by a frame line, a mode in which a line feed is formed for each group of the same classification, a mode in which an annotation indicating the classification result is displayed, or an appropriate combination thereof, in addition to the mode in which the display color of the characters is different.

Next, in step ST3ofFIG.4, the processor102determines the type of the organ for each sentence included in the acquired text. However, the processor102may exclude the sentence corresponding to “other” in the determination of the type of the medical process from the processing target of step ST3. The processor102determines the type of the organ in a sentence unit and classifies each sentence by the type of the organ according to the determination result. The determination of the type of the organ in a sentence unit is an example of classifying the attributes of the text description information in a fixed unit. The processing of step ST3is performed by the description information classification unit14.

An upper part ofFIG.7shows an example of the result of determining the type of the organ in a sentence unit. The three sentences of “Irregular tubercle with diameter of 1.5 cm is found in left lung S6. With internal calcification. Bulla in right lung S3.” are descriptions of the lung, and the type of the organ in each of these sentences is determined to be “lung”. The three sentences of “Tumor with diameter of 6 cm is found in liver S3, enhancement effect and washout are exhibited, and HCC is suspected. Enhanced tumor with diameter of 5 mm is also found in liver S4 and is persistent. Hemangioma is suspected.” are descriptions of the liver, and the type of the organ in each of these sentences is determined to be “liver”. The sentence of “Calcification myoma in uterus.” is a description of the uterus, and the type of the organ in this sentence is determined to be “uterus”.

Further, in the case where the interpretation report, which is an unstructured text, is displayed on the display apparatus24or the like, the processor102displays the determination result (classification result) of the type of the organ in an identifiable manner. In the display mode in which the classification result is identifiable, various display modes may be employed such as a mode in which the display color of the characters is different for each classification, a mode in which each group of the same classification is surrounded by a frame line, a mode in which a line feed is formed for each group of the same classification, a mode in which character information indicating the classification result is displayed, or an appropriate combination thereof.

In the example shown in the upper part ofFIG.7, the display color of the characters is different such as displaying the characters of the sentence classified as “lung” in, for example, orange, displaying the characters of the sentence classified as “liver” in, for example, blue, and displaying the characters of the sentence classified as “uterus” in, for example, purple.

Next, in step ST4, the processor102analyzes the text for each of the same classifications based on the determination of the type of the organ to perform the term extraction. The processing of step ST4is performed by the term extraction unit30of the text analysis unit16.

In step ST5, the processor102acquires the relationship between the extracted terms. The processing of step ST5is performed by the relationship acquisition unit32of the text analysis unit16. The input of the type of the medical process, the type of the organ, the extracted term expression as the term expression, and the type thereof is received as the text description information, and the presence or absence of a relationship between the terms is determined to acquire the relationship. The processor102excludes those in which the type of the medical process is “other” and then performs the processing of acquiring the relationship in a set of sentences having the same organ type (sentence sets of the same classification).

A middle part ofFIG.7shows an example in a case where the term extraction and the relationship acquisition are performed for the sentence classified as “liver”. In the middle part ofFIG.7, a term underlined in the text is a term as the object or the subject and represents that terms connected by a broken line are related to each other. Here, it is grasped that terms of each type of a location, a quantity, a property, and a disease name related to the term “tumor”, which is a term representing a lesion, are described in the text. In a case where the text is displayed, the processor102may display the terms extracted by the term extraction in different colors for each type.

Although not shown inFIG.7, the pieces of processing of steps ST4and ST5are similarly performed for each of the sentences classified into the “lung” and the “uterus”.

Next, in step ST6, the processor102displays the processing result of the term extraction and the relationship acquisition as a structuring result. A lower part ofFIG.7shows a display example of the structuring result for the sentence classified as “liver”.

Configuration Example of Description Information Classification Unit14

The description information classification unit14can be configured by using, for example, a natural language processing model, which is referred to as bidirectional encoder representations from transformers (BERT) described in Jacob Devlin, Ming-Wei Chang, Kenton Lee, Kristina Toutanova, “BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding”<https://arxiv.org/pdf/1810.04805>. The present invention is not limited to BERT, and another machine learning model such as a recurrent neural network (RNN) or a support vector machine (SVM) may be applied.

The description information classification unit14determines what is written about the input text in a fixed unit, for example, in a sentence unit, by using a text classification technique. In a case where a text includes a plurality of sentences, the description information classification unit14determines which classification each sentence belongs to from connection of each sentence in the text.

In a case where the text is structured for each organ, the description information classification unit14determines the type of the organ of each sentence in the report by using the text classification technique. In a case where a text is input to the description information classification unit14, a sentence surrounding a target sentence may also be input to determine the type of the organ. The surrounding sentence may be an N sentence before the target sentence and an M sentence after the target sentence. N represents an integer of 1 or more and M represents an integer of 0 or more. For example, in a case where the target sentence is the text of “With internal calcification” in the text of “Irregular tubercle with diameter of 1.5 cm is found in left lung S6. With internal calcification.”, the two sentences of “Irregular tubercle with diameter of 1.5 cm is found in left lung S6. With internal calcification.” are input including the previous sentence as an input to the BERT model. With the input of the two sentences, it is possible to determine that the target sentence is a sentence related to “lung”.

Configuration Example of Term Extraction Unit30

The term extraction unit30uses a named entity recognition (NER) technique to acquire the term expression and determine the type thereof. A task of the term extraction unit30includes a classification task of receiving an input of a token series and predicting a label for each token being input, and a task of distinguishing between a start and an end of a named entity (NE) by a BIO method. In the BIO method, the start and the end of the named entity are grasped by using tags of “Begin” indicating the start of the named entity, “Inside” indicating a continuation (inside) of the named entity, and “other” indicating something that is not the named entity (other than named entity).

The term extraction unit30can be configured using, for example, an encoder-decoder model.FIG.8shows an example of a term extraction model300applied to the term extraction unit30. The term extraction model300includes an encoder302and a decoder304. The encoder302is configured by using, for example, a neural network such as BERT or long short term memory (LSTM). The encoder302obtains a distributed expression from an input text. The decoder304predicts a probability distribution for the label from the distributed expression of each token output from the encoder302by a fully-connected layer. In the NER, an NER label in which a BIO label is combined with an NE label is handled.

For example, a label “B area” shown inFIG.8is an NER label in which “Begin” of the BIO label is combined with “Area” of the NE label. A label “I lesion” is an NER label in which “Inside” of the BIO label is combined with “Lesion” of the NE label.

FIG.9shows an example of the probability distribution of the NER label obtained by the decoder304. The term extraction model300is trained to minimize cross-entropy of a softmax function output of the decoder304.

Configuration Example of Relationship Acquisition Unit32

FIG.10is a block diagram showing a configuration example of the relationship acquisition unit32. The relationship acquisition model320applied to the relationship acquisition unit32has, for example, a structure in which a pooling layer324, a concatenating layer326, and a linear layer328are added to an output of a BERT model322.

The relationship acquisition unit32performs binary classification of the presence or absence of the relationship for the input text from the subject term, the object term, and the span between these terms.

In the input text illustrated inFIG.10, the term “liver S3” is the subject term, the term “tumor” is the object term, and a character string of “with diameter of 7 mm” between these terms is the span. In a case where a text is input to the BERT model322, output values t1, t2, t3, . . . , and the like for each token are obtained from the BERT model322. InFIG.10, “000” surrounded by a square with rounded corners represents an output value corresponding to each token. For example, the output value t1 inFIG.10represents an output value corresponding to the token of “liver S3” in an input token string. The output value t2 represents an output value corresponding to the token of the span, and the output value t3 represents an output value corresponding to the “tumor”.

The output of the BERT model322is input to the pooling layer324and is subjected to pooling processing by the pooling layer324. An output of the pooling layer324is input to the concatenating layer326and is subjected to concatenating processing by the concatenating layer326. An output of the concatenating layer326is subjected to linear conversion processing by the linear layer328. Binarization indicating “relevant” or “unrelated” is performed based on a value output from the linear layer328.

Various Application Examples

The configuration has been described in which the above information processing apparatus10performs the text analysis for each of the same classifications to perform the term extraction and the relationship acquisition. However, a configuration may be employed in which only the term extraction is performed as the processing content of the text analysis performed for each of the same classifications. Further, the classification is not limited to the sentence unit and may be the phrase unit or the like. Hereinafter, specific variations of application examples will be described as Examples 1 to 7.

Example 1

Example 1 is an example in which an organ recognition result in a sentence unit is used to perform the term extraction on the sentence for each organ for the structuring. An upper part ofFIG.11shows an example of structuring the medical text of “Irregular tubercle with diameter of 1.5 cm is found in left lung S6. With internal calcification. Bulla in right lung S3. Tumor with diameter of 6 cm is found in liver S3, enhancement effect and washout are exhibited, and HCC is suspected”.

In this case, the processor102performs organ recognition for each sentence from the first sentence to the fourth sentence to obtain a result of “lung, lung, lung, liver”. A notation of the organ recognition result of “lung, lung, lung, liver” means a classification result that each of the first sentence to the third sentence is a sentence related to the lung and the fourth sentence is a sentence related to the liver. In a lower part ofFIG.11, the result of the organ recognition for each sentence is shown by a notation such as “/[lung]”.

The processor102performs the term extraction on the sentence for each organ based on the classification result of each sentence and thus can obtain structured information of “organ: lung, location: left lung S6 and right lung S3, quantity: diameter of 1.5 cm, lesion (+): tubercle and bulla, property (+): irregular, calcification” and “organ: liver, location: liver S3, quantity: diameter of 6 cm, lesion (+): tumor, property (+): enhancement effect and washout, disease name: HCC” as shown inFIG.12. The type of the lesion and the disease name, such as tubercle, bulla, or HCC, are examples of the type of the disease.

Example 2

Example 2 is an example in which an organ recognition result in a phrase unit is used to perform the term extraction on the phrase for each organ for the structuring. An upper part ofFIG.13shows an example of structuring a medical text of “Cyst is found in liver S3 and no ascites is found”. The processor102performs the organ recognition on each phrase of this sentence to obtain a classification result that “Cyst is found in liver S3” is the liver and “no ascites is found” is the abdominal cavity. In a middle part ofFIG.13, the result of organ recognition for each phrase is shown by a notation such as “/[liver]”.

The processor102performs the term extraction on the phrase for each organ and thus can obtain structured information of “organ: liver, location: liver S3, lesion (+): cyst” and “organ: abdominal cavity, lesion (−): ascites” as shown in a lower part ofFIG.13.

Example 3

Example 3 is an example in which the organ recognition result in a sentence unit is used to perform the term extraction and the relationship acquisition on the sentence for each organ for the structuring. As in Example 1, an example will be described in which the medical text of “Irregular tubercle with diameter of 1.5 cm is found in left lung S6. With internal calcification. Bulla in right lung S3. Tumor with diameter of 6 cm is found in liver S3, enhancement effect and washout are exhibited, and HCC is suspected.” is structured.

The processor102performs the organ recognition for each sentence to obtain a result of “lung, lung, lung, liver”. The processor102performs the term extraction and the relationship acquisition on the sentence for each organ and thus can obtain structured information of “organ: lung, location: left lung S6, quantity: diameter of 1.5 cm, lesion (+): tubercle, property (+): irregular and calcification”, “organ: lung, location: right lung S3, lesion (+): bulla”, and “organ: liver, location: liver S3, quantity: diameter of 6 cm, lesion (+): tumor, property (+): enhancement effect and washout, disease name: HCC” as shown inFIG.14. As another form, in a case where a model is used in which the sentence for each organ and the type of the organ can be considered, a configuration may be employed in which the type of the organ is input to the model in addition to the sentence for each organ to perform the term extraction or the relationship acquisition.

Example 4

Example 4 is an example in which the organ recognition result in a sentence unit is used to perform the term extraction and the relationship acquisition, using a different model for each organ, on the sentence for each organ for the structuring. An upper part ofFIG.15shows an example of structuring a medical text of “Right coronary artery: calcification (+). Left coronary artery: stenosis (−). Irregular tubercle with diameter of 1.5 cm is found in left lung S6”.

The processor102performs the organ recognition on each sentence to obtain a classification result of “heart, heart, lung”. In a lower part ofFIG.15, the result of organ recognition for each sentence is shown by a notation such as “/[heart]”.

Based on the result of such organ recognition, the processor102uses a machine learning model trained for the heart for the sentence described for the heart and uses a machine learning model trained for the lung for the sentence described for the lung to perform the term extraction and the relationship acquisition.

FIG.16is a conceptual diagram of the text analysis unit16applied to Example 4. The text analysis unit16shown inFIG.16includes a machine learning model410trained for the heart, a machine learning model420trained for the lung, and a model selection unit430that selects a model to be used. The machine learning model410trained for the heart is a learned model trained using a text describing the heart as learning data, and includes a term extraction model412trained to perform a term extraction task and a relationship acquisition model414trained to perform a relationship acquisition task.

The machine learning model420trained for the lung is a learned model trained using a text described about the lung as learning data, and includes a term extraction model422trained to perform the term extraction task and a relationship acquisition model424trained to perform the relationship acquisition task. Although not shown inFIG.16, the text analysis unit16may include a model trained for another organ, such as a machine learning model trained for the liver.

The model selection unit430performs processing of selectively switching the machine learning model to be used based on the classification information obtained as a result of the organ recognition. For example, in a case where the classification information of the input text indicates a label of “heart”, the model selection unit430selects the machine learning model410trained for the heart as the model used for the text analysis. The term extraction and the relationship acquisition are performed on the text related to the heart using the machine learning model410trained for the heart to obtain the structured information of the text related to the heart. Further, in a case where the classification information of the input text indicates a label of “lung”, the model selection unit430selects the machine learning model420trained for the lung as the model used for the text analysis. The term extraction and the relationship acquisition are performed on the text related to the lung using the machine learning model420trained for the lung to obtain the structured information of the text related to the lung.

FIG.17is an example of structured information obtained by the text analysis using the machine learning model410trained for the heart.FIG.18is an example of structured information obtained by the text analysis using the machine learning model420trained for the lung.

As described with reference toFIG.16, with the text analysis using an appropriate model in accordance with the type of the organ, it is possible to perform more accurate analysis. With such a configuration, it is possible to perform the structuring with high accuracy even in a case where the writing is different depending on the organ and to obtain structured information of “organ: heart, location: right coronary artery, lesion (+): calcification”, “organ: heart, location: left coronary artery, lesion (−): stenosis”, and “organ: lung, location: left lung S6, quantity: diameter of 1.5 cm, lesion (+): tubercle, property (+): irregular” as shown inFIGS.17and18.

The present invention is not limited to the configuration in which the machine learning model is individually prepared for each organ. For example, a mode may be employed in which the machine learning model410trained for the heart is prepared for an organ with a special text expression, for example, the heart and a machine learning model trained for an organ other than the heart is employed for the organ other than the heart.

Example 5

Example 5 is an example referred to as another form of Example 3 and is an example in which a model in which the organ recognition result is input as auxiliary information is used in a case where the organ recognition result in a sentence unit is used to perform the term extraction and the relationship acquisition on the sentence for each organ.

FIG.19is a conceptual diagram of the text analysis unit16applied to Example 5. The text analysis unit16shown inFIG.19includes a machine learning model450that processes the text related to various organs. The machine learning model450includes a term extraction model452that performs the term extraction task and a relationship acquisition model454that performs the relationship acquisition task. The text for each organ and type information (classification information) of the organ indicating the organ recognition result of the text are input to the machine learning model450. With the input of the classification information of the organ as the auxiliary information, each of the term extraction model452and the relationship acquisition model454can perform the term extraction and the relationship acquisition with high accuracy.

In a case where the machine learning model450commonly used for the text related to the various organs is trained, data including the text related to the various organs and the type information of the organ indicating the classification of the text is used as the learning data.

FIG.20is an explanatory diagram showing an example of the processing by the machine learning model450. As shown inFIG.20, for example, the text of “Irregular tubercle with diameter of 1.5 cm is found in left lung S6. With internal calcification. Bulla in right lung S3.” and the auxiliary information of “lung” indicating the type of the organ is input to the machine learning model450, and an output of a processing result is obtained from the machine learning model450. From the processing result of the machine learning model450, structured information of “organ: lung, location: left lung S6, quantity: diameter of 1.5 cm, lesion (+): tubercle, property (+): irregular, calcification” and “organ: lung, location: right lung S3, lesion (+): bulla” is obtained.

Example 6

The information processing apparatus10that handles the interpretation report has been described so far. However, the technique of the present disclosure is not limited to an image diagnosis report represented by the interpretation report, but is applicable to a system that handles various medical texts such as a text related to the patient's medical record. The text related to the patient's medical record includes, for example, an intermediate summary or a discharge summary. Further, the technique of the present disclosure is not limited to medical text, but is applicable to processing that handles texts in various fields, such as a factory maintenance report, an examination result report of an industrial product, an examination result report of a building or the like, or various types of appraisal reports, regardless of a type of an object or use.

In Example 6, an example of handling the maintenance report of the factory will be described.FIG.21is an example of processing of structuring by classifying the maintenance report of the factory into the types of contents in a sentence unit and performing the term extraction for each of the same classifications. An upper part ofFIG.21shows an example of structuring a maintenance report of “Check error display with air flow rate meter. Current value is 3.8 mA when checked. Malfunction of substrate or diagram is considered after confirming with manufacturer. Air flow rate meter is at failure from investigation result. Thus, update to successor model is performed”.

The information processing apparatus that processes such a text may be configured to be the same as the configuration of the information processing apparatus10described with reference toFIG.2or is configured to perform determination as to whether each sentence is “symptom”, “cause”, or “countermeasure”, instead of the determination of the type of the organ. In this case, a result of “symptom, symptom, cause, cause, countermeasure” is obtained as shown in a middle part ofFIG.21as a classification result of each sentence by the description information classification unit14.

The text analysis unit16performs the term extraction on the sentences for each of the classified contents. Accordingly, it is possible to obtain structured information of “content: symptom, part: air flow rate meter, phenomenon: error display, measurement item: current value, measured value: 3.8 mA”, “content: cause, part: substrate, diagram, and air flow rate meter, phenomenon: malfunction and failure”, and “content: countermeasure, part: successor model, phenomenon: update” as shown in a lower part ofFIG.21.

Program for Operating Computer

A program causing a computer to realize a part or all of the processing functions in the above information processing apparatus10can be recorded on a computer-readable medium which is a non-transitory tangible information storage medium such as an optical disk, a magnetic disk, or a semiconductor memory, and the program can be provided through this information storage medium.

Further, instead of the mode in which the program is provided by being stored in such a non-transitory tangible computer-readable medium, a program signal may be provided as a download service using a telecommunication line such as the Internet.

Furthermore, a part or all of the processing functions in the information processing apparatus10may be realized by cloud computing, or may be provided as a software as a service (SasS).

Hardware Configuration of Each Processing Unit

A hardware structure of the processing units executing various types of processing such as the text acquisition unit12, the description information classification unit14, the text analysis unit16, the analysis result output unit18, the term extraction unit30, the relationship acquisition unit32, and the display control unit40in the information processing apparatus10is, for example, various processors as shown below.

The various processors include a CPU which is a general-purpose processor that functions as various processing units by executing a program, a GPU which is a processor specialized for image processing, a programmable logic device (PLD) such as a field programmable gate array (FPGA) which is a processor capable of changing a circuit configuration after manufacture, a dedicated electric circuit such as an application specific integrated circuit (ASIC) which is a processor having a circuit configuration specifically designed to execute specific processing, and the like.

One processing unit may be configured by one of these various processors or may be configured by two or more processors having the same type or different types. For example, one processing unit may be configured by a plurality of FPGAs, a combination of a CPU and an FPGA, or a combination of a CPU and a GPU. The plurality of processing units may be configured of one processor. As an example in which the plurality of processing units are configured by one processor, firstly, as represented by a computer such as a client and a server, a form may be employed in which one processor is configured by a combination of one or more CPUs and software and the processor functions as the plurality of processing units. Secondly, as represented by a system on chip (SoC) or the like, a form may be employed in which a processor that realizes the function of the entire system including the plurality of processing units by one integrated circuit (IC) chip is used. As described above, the various processing units are configured by using one or more various processors as a hardware structure.

Further, as the hardware structure of the various processors, more specifically, an electric circuit (circuitry) in which circuit elements such as semiconductor elements are combined may be used.

Advantages According to Embodiment

With the information processing apparatus10according to the present embodiment, the analysis processing such as the term extraction is performed for each text belonging to the same classification. Therefore, it is possible to analyze the text with high accuracy and to acquire a correct relationship between the terms in the text. Accordingly, it is possible to obtain the structured information with high accuracy. The technique of the present disclosure is particularly effective in a case of analyzing a text including a plurality of sentences describing a plurality of matters having different types (attributes).

Other

The present disclosure is not limited to the above embodiment, and various modifications can be made without departing from the spirit of the technical idea of the present disclosure.

EXPLANATION OF REFERENCES

10: information processing apparatus12: text acquisition unit14: description information classification unit16: text analysis unit18: analysis result output unit20: data saving unit22: input apparatus24: display apparatus30: term extraction unit32: relationship acquisition unit40: display control unit42: communication unit50: classification result saving unit52: analysis result saving unit102: processor104: computer-readable medium106: communication interface108: input/output interface110: bus122: memory124: storage140: description information classification program160: text analysis program162: term extraction program164: relationship acquisition program180: display control program182: communication control program200: medical information system202: electronic medical record system203: examination order system204: CT apparatus206: MRI apparatus210: image saving server220: report server230: terminal apparatus232: input apparatus234: display apparatus240: communication line300: term extraction model302: encoder304: decoder320: relationship acquisition model322: BERT model324: pooling layer326: concatenating layer328: linear layer410: machine learning model412: term extraction model414: relationship acquisition model420: machine learning model422: term extraction model424: relationship acquisition model430: model selection unit450: machine learning model452: term extraction model454: relationship acquisition modelST1˜ST6: step of information processing method