Patent ID: 12211600

DETAILED DESCRIPTION

Hereinafter, each exemplary embodiment of the present disclosure will be described with reference to the drawings.

First, a configuration of a medical information system1to which an information processing apparatus of the present disclosure is applied will be described.

FIG.1is a diagram showing a schematic configuration of the medical information system1. The medical information system1shown inFIG.1is, based on an examination order from a doctor in a medical department using a known ordering system, a system for imaging an examination target part of a subject, storing a medical image acquired by the imaging, interpreting the medical image by a radiologist and creating an interpretation report, and viewing the interpretation report and observing the medical image to be interpreted in detail by the doctor in the medical department that is a request source.

As shown inFIG.1, the medical information system1is configured to include a plurality of imaging apparatuses2, a plurality of interpretation work stations (WS)3that are interpretation terminals, a medical care WS4, an image server5, an image database (DB)6, a report server7, and a report DB8, which are connected via a wired or wireless network10so as to be able to communicate with each other.

Each apparatus is a computer on which an application program for causing each apparatus to function as a component of the medical information system1is installed. The application program is recorded on a recording medium, such as a digital versatile disc (DVD) or a compact disc read only memory (CD-ROM), and distributed, and is installed on the computer from the recording medium. Alternatively, the application program is stored in a storage apparatus of a server computer connected to the network10or in a network storage in a state in which it can be accessed from the outside, and is downloaded and installed on the computer in response to a request.

The imaging apparatus2is an apparatus (modality) that generates a medical image showing a diagnosis target part of the subject by imaging the diagnosis target part. Specifically, examples of the imaging apparatus include a simple X-ray imaging apparatus, a CT apparatus, an MRI apparatus, a positron emission tomography (PET) apparatus, and the like. The medical image generated by the imaging apparatus2is transmitted to the image server5and is saved in the image DB6.

The interpretation WS3is a computer used by, for example, a radiologist of the radiology department to interpret a medical image and to create an interpretation report, and encompasses an information processing apparatus20(which will be described in detail later) according to the present exemplary embodiment. In the interpretation WS3, a viewing request for a medical image to the image server5, various image processing for the medical image received from the image server5, display of the medical image, and input reception of comments on findings regarding the medical image are performed. In the interpretation WS3, an analysis process for medical images, support for creating an interpretation report based on the analysis result, a registration request and a viewing request for the interpretation report to the report server7, and display of the interpretation report received from the report server7are performed. The above processes are performed by the interpretation WS3executing software programs for respective processes.

The medical care WS4is a computer used by, for example, a doctor in a medical department to observe an image in detail, view an interpretation report, create an electronic medical record, and the like, and is configured to include a processing apparatus, a display apparatus such as a display, and an input apparatus such as a keyboard and a mouse. In the medical care WS4, a viewing request for the image to the image server5, display of the image received from the image server5, a viewing request for the interpretation report to the report server7, and display of the interpretation report received from the report server7are performed. The above processes are performed by the medical care WS4executing software programs for respective processes.

The image server5is a general-purpose computer on which a software program that provides a function of a database management system (DBMS) is installed. The image server5comprises a storage in which the image DB6is configured. This storage may be a hard disk apparatus connected to the image server5by a data bus, or may be a disk apparatus connected to a storage area network (SAN) or a network attached storage (NAS) connected to the network10. In a case where the image server5receives a request to register a medical image from the imaging apparatus2, the image server5prepares the medical image in a format for a database and registers the medical image in the image DB6.

Image data of the medical image acquired by the imaging apparatus2and accessory information are registered in the image DB6. The accessory information includes, for example, an image identification (ID) for identifying each medical image, a patient ID for identifying a subject, an examination ID for identifying an examination, a unique ID (unique identification (UID)) allocated for each medical image, examination date and examination time at which a medical image is generated, the type of imaging apparatus used in an examination for acquiring a medical image, patient information such as the name, age, and gender of a patient, an examination part (an imaging part), imaging information (an imaging protocol, an imaging sequence, an imaging method, imaging conditions, the use of a contrast medium, and the like), and information such as a series number or a collection number in a case where a plurality of medical images are acquired in one examination.

In addition, in a case where the viewing request from the interpretation WS3and the medical care WS4is received through the network10, the image server5searches for a medical image registered in the image DB6and transmits the searched for medical image to the interpretation WS3and to the medical care WS4that are request sources.

The report server7incorporates a software program for providing a function of a database management system to a general-purpose computer. In a case where the report server7receives a request to register the interpretation report from the interpretation WS3, the report server7prepares the interpretation report in a format for a database and registers the interpretation report in the report DB8.

In the report DB8, an interpretation report including at least the comments on findings created by the radiologist using the interpretation WS3is registered. The interpretation report may include, for example, information such as a medical image to be interpreted, an image ID for identifying the medical image, a radiologist ID for identifying the radiologist who performed the interpretation, a lesion name, lesion position information, and property information (which will be described in detail later).

Further, in a case where the report server7receives the viewing request for the interpretation report from the interpretation WS3and the medical care WS4through the network10, the report server7searches for the interpretation report registered in the report DB8, and transmits the searched for interpretation report to the interpretation WS3and to the medical care WS4that are request sources.

The network10is a wired or wireless local area network that connects various apparatuses in a hospital to each other. In a case where the interpretation WS3is installed in another hospital or clinic, the network10may be configured to connect local area networks of respective hospitals through the Internet or a dedicated line.

Next, the information processing apparatus20according to the present exemplary embodiment will be described.

First, with reference toFIG.2, a hardware configuration of the information processing apparatus20according to the present exemplary embodiment will be described. As shown inFIG.2, the information processing apparatus20includes a central processing unit (CPU)11, a non-volatile storage unit13, and a memory16as a temporary storage area. Further, the information processing apparatus20includes a display14such as a liquid crystal display and an organic electro luminescence (EL) display, an input unit15such as a keyboard and a mouse, and a network interface (I/F)17connected to the network10. The CPU11, the storage unit13, the display14, the input unit15, the memory16, and the network I/F17are connected to a bus18. The CPU11is an example of a processor in the present disclosure.

The storage unit13is realized by a storage apparatus such as a hard disk drive (HDD), a solid state drive (SSD), and a flash memory. An information processing program12is stored in the storage unit13as the storage medium. The CPU11reads out the information processing program12from the storage unit13, loads the read-out program into the memory16, and executes the loaded information processing program12.

Next, with reference toFIGS.3to5, a functional configuration of the information processing apparatus20according to the present exemplary embodiment will be described. As shown inFIG.3, the information processing apparatus20includes an acquisition unit21, a derivation unit22, a reception unit23, a generation unit24, and a display control unit25. The CPU11executing the information processing program12functions as the acquisition unit21, the derivation unit22, the reception unit23, the generation unit24, and the display control unit25.

FIG.5is a diagram showing an example of the interpretation report creation screen30displayed on the display14. The creation screen30includes an image area32on which a medical image is displayed, a reception area40on which a property item44and a reception means46(details will be described later) are displayed, a comment-on-findings area50in which candidates of comments on findings are displayed, and a cancel and OK button36. Hereinafter, the functions of the acquisition unit21, the derivation unit22, the reception unit23, the generation unit24, and the display control unit25will be described with reference toFIG.5.

The acquisition unit21acquires a medical image G0as an example of the image from the image server5via the network I/F17.FIG.4is a diagram schematically showing the medical image G0. In the present exemplary embodiment, as an example, a CT image of a lung is used as the medical image G0. The medical image G0includes a nodular shadow N as an example of a structure of interest such as a lesion.

The derivation unit22derives property information indicating a property of the property item44for each predetermined property item44from at least one medical image G0. Specifically, the derivation unit22analyzes the medical image G0via CAD or the like, and specifies the position, type, and size of the lesion included in the medical image G0. Then, a property score indicating the prominence of a property of a predetermined property item44regarding a specified lesion is derived, and the property information indicating the property of the property item44is derived based on the property score. The property information is information showing the result of determining whether the property is positive or negative, for example, depending on whether the property score is equal to or higher than a predetermined threshold value. The property item44is an item that is predetermined and stored in the storage unit13according to at least one of the position, type, or size of the lesion.

As shown inFIG.5, the display control unit25performs control such that the medical image G0acquired by the acquisition unit21is displayed in the image area32. Further, the display control unit25controls the display in the reception area40such that the user can check the property information for each property item44based on the property information derived by the derivation unit22.

FIG.5shows, as an example of the property item44related to the nodular shadow N, the presence or absence of solid type, spicula, pleural invagination, air bronchogram, cavity, and calcification. An example is shown in which these property items44are color-coded (see Legend34) and displayed as positive or negative based on the property information. The method of displaying the property information by the display control unit25is not limited thereto, and for example, characters and marks indicating positive or positive may be displayed for each property item. In addition, for example, a property score may be used as the property information and displayed in a gradation in which the color becomes darker as the property becomes more prominent.

Incidentally, in the case of supporting the creation of an interpretation report, it is desirable to generate a plurality of comment-on-findings candidates whose description contents are changed according to the intention and preference of the radiologist who creates the interpretation report. For example, some radiologists prefer a concise interpretation report in which only important findings are described, while others prefer a complete interpretation report in which all positive findings are described.

Therefore, the reception unit23receives, for each property item44, a description regarding the property item44to be selectable as to whether the description is necessary, unnecessary, or optional in a document. As shown inFIG.5, the display control unit25controls the reception unit23such that the reception means46for receiving the selection of the description being necessary, unnecessary, or optional for each property item44is displayed in the reception area40. For example, the reception means46may be a button that transitions from the initial value “optional” to “necessary”, “unnecessary, “optional”, . . . by left-clicking the mouse as an example of the input unit15, and transitions in the reverse order by right-clicking. Further, for example, the reception means46may be a pull-down list for selecting any one of “necessary”, “unnecessary”, and “optional”, a radio button, and the like.

The initial value of the reception means46is not limited to “optional”, and for example, a property item that requires description and a property item having a high priority of description may be set in advance for each property item44, such as “necessary”. Further, by learning the setting history of the reception means46by the user, a value with a high frequency of setting may be set as an initial value.

Further, as shown inFIG.5, the display control unit25may display, for each property item44, an icon42(see Legend34) indicating the description being necessary, unnecessary, or optional that has been currently selected so that the user can easily identify it. Further, as shown inFIG.5, the display control unit25may change the color and frame of the property item44selected as the description being unnecessary by the user so that the user can easily identify it.

The user selects “necessary”, “unnecessary”, or “optional” for each property item44by operating the reception means46via the input unit15while checking the property information displayed in the reception area40(the color of the property item44in the example ofFIG.5). That is, in addition to selecting a property item that requires description and a property item that omits description, the user can make a selection that entrusts the information processing apparatus20to determine whether or not the description is necessary. Then, in a case where the user finishes the operation of the reception means46, the user presses a comment-on-findings creation button48.

In a case where the comment-on-findings creation button48is pressed, the generation unit24generates a plurality of comment-on-findings candidates52as an example of character strings including the description based on the property information regarding the property items44that have been selected as the descriptions being necessary and optional by changing a combination of the property items44selected from among the property items44that have been selected as the description being optional. As a method of generating the comment-on-findings candidate52by using the generation unit24, for example, a learning model in which machine learning is performed, such as the recurrent neural network described in JP2019-153250A can be applied. In the case of applying such a learning model, the comment-on-findings candidate52may be generated by the generation unit24analyzing the medical image G0via CAD or the like separately from the analysis performed by the derivation unit22, and inputting the result of the analysis performed by the generation unit24into the learning model. Further, the comment-on-findings candidate52may be generated by inputting the result of the analysis performed by the derivation unit22into the learning model.

The display control unit25performs control such that the plurality of comment-on-findings candidates52generated by the generation unit24are displayed in the comment-on-findings area50. InFIG.5, an example is shown in which three comment-on-findings candidates52generated by changing the combination of whether or not to describe “solid type” and “calcification”, which are the property items44that have been selected as the description being optional, are displayed. The comments on findings of the first candidate include descriptions regarding “solid type” and “calcification”. The comments on findings of the second candidate include a description regarding “solid type” but not a description regarding “calcification”. The comments on findings of the third candidate do not include descriptions regarding “solid type” and “calcification”.

In addition, it is preferable that the generation unit24constructs a combination of the property items44selected from among the property items44that have been selected as the description being optional such that the comments on findings satisfying a predetermined rule are included as any of the comment-on-findings candidates52. That is, the rule defines a criterion for determining which property item44is included in the comment-on-findings candidates52among the property items44that have been selected as the description being optional. It is sufficient that any one of the comment-on-findings candidates52generated by the generation unit24is generated so as to satisfy the rule, and all the comment-on-findings candidates52do not necessarily satisfy the rule.

As an example of the rule, there is a rule regarding the number of the property items44included in the comment-on-findings candidate52. In this case, the generation unit24selects the property item to be included in the comment-on-findings candidate52from among the property items44that have been selected as the description being optional such that the number of the property items44included in the comment-on-findings candidate52satisfies the rule. By providing a rule regarding the number of the property items44, for example, the comment-on-findings candidate52can be generated so as not to be too long and not too short.

Further, as an example of the rule, there is a rule regarding the priority set for each property item44. For example, rules may be set such as lowering the priority of the description because whether or not it is “solid type” can be determined by looking at the medical image G0, and increasing the priority of the description because malignancy is suspected in the case where “spicula” is found. In this case, the generation unit24may select the property item44having a relatively high priority from among the property items44that have been selected as the description being optional as the property item to be included in the comment-on-findings candidate52. Thereby, the comment-on-findings candidate52including the description regarding the property item44having a high priority can be generated.

Further, as an example of the rule, there is a rule regarding the property score. As described above, the property score is a score indicating the prominence of the property of the property item44. For example, a rule may be defined in which the description regarding the property item44whose property score is equal to or higher than the threshold value is preferentially included in the comment-on-findings candidate52. In this case, the generation unit24may select the property item44whose property score is equal to or higher than the threshold value from among the property items44that have been selected as the description being optional as the property item to be included in the comment-on-findings candidate52. Thereby, the comment-on-findings candidate52with high reliability of the content can be generated.

In addition, as an example of the rule, there is a rule regarding the relationship between the property items44. For example, in a case where the nodular shadow N is a frosted glass-like nodule, calcification is usually not found. Therefore, for example, in a case where it is determined from the property information that the nodular shadow N is likely to be frosted glass-like, a rule may be set so as not to make description regarding calcification. In this case, in the case where it is determined that the nodular shadow N is likely to be frosted glass-like, the generation unit24may exclude “calcification” that has been selected as the description being optional from the property items to be included in the comment-on-findings candidate52. Thereby, the comment-on-findings candidate52in which the description regarding calcification is omitted can be generated.

Note that each of the above-mentioned rules is an example and is not limited to these, and these may be selectively used in combination. In addition, the rule may be a predetermined rule or may be a predetermined rule selected by the user. For example, prior to the generation of the comment-on-findings candidate52performed by the generation unit24, a screen provided with a check box for selecting any rule from a plurality of predetermined rules may be displayed on the display14and the selection by the user may be received.

Further, desirably, the display control unit25performs control such that the comment-on-findings candidates52are displayed in the comment-on-findings area50by arranging them in the order of the rule compliance rate. Further, the display control unit25may perform control such that the comment-on-findings candidate52having the highest rule compliance rate among the plurality of comment-on-findings candidates52is emphasized and displayed on the display. Further, as shown inFIG.5, the display control unit25may display a list54of the property items44including the description in each comment-on-findings candidate52in the comment-on-findings area50.

The user selects a desired comment-on-findings candidate52with a radio button56while referring to the comment-on-findings candidate52and the list54, and presses the OK button36. Thereby, an interpretation report including the medical image G0and the comment-on-findings candidate52selected by the user is created.

Next, with reference toFIG.6, operations of the information processing apparatus20according to the present exemplary embodiment will be described. The CPU11executes the information processing program12, and thus, the information processing shown inFIG.6is executed. The information processing shown inFIG.6is executed, for example, in a case where an instruction to start creating an interpretation report for the medical image G0is input via the input unit15.

In Step S10ofFIG.6, the acquisition unit21acquires the medical image G0from the image server5. In Step S12, the derivation unit22specifies the position, type, and size of the lesion included in the medical image G0acquired in Step S10, and derives a property score indicating the prominence of the properties of predetermined property items related to the specified lesion. In Step S14, the derivation unit22derives the property information indicating the property of the property item from the property score derived in Step S12.

In Step S16, the reception unit23receives, for each property item, selection of a description regarding the property item as to whether the description is necessary, unnecessary, or optional in the document. In Step S18, the generation unit24generates a plurality of comment-on-findings candidates52including the description based on the property information derived in Step S14regarding the property items that have been selected as the descriptions being necessary and optional in Step S16by changing a combination of the property items selected from among the property items that have been selected as the description being optional. In Step S20, the display control unit25performs control such that the comment-on-findings candidate52generated in Step S18is displayed on the display14, and ends the process.

As described above, the information processing apparatus20according to the exemplary embodiment of the present disclosure derives property information indicating the property of the property item for each predetermined property item from at least one medical image G0. In addition, for each of the property items, a description regarding the property item is received to be selectable as to whether the description is necessary, unnecessary, or optional in a document. In addition, a plurality of character string candidates including the description based on the property information regarding the property items that have been selected as the descriptions being necessary and optional are generated by changing a combination of the property items selected from among the property items that have been selected as the description being optional. According to such a form, since a plurality of comment-on-findings candidates having contents desired by the radiologist can be presented, it is possible to support the creation of a document such as an interpretation report.

In the above exemplary embodiment, in the case where the user finishes the operation of the reception means46and presses the comment-on-findings creation button48, the generation unit24starts generation of the comment-on-findings candidate52, but the present disclosure is not limited thereto. For example, the comment-on-findings candidate52may be generated in real time each time the user operates the reception means46.

Further, although the derivation unit22specifies the position of the lesion included in the medical image G0in the above exemplary embodiments, the present disclosure is not limited thereto. For example, the user may select a region of interest in the medical image G0via the input unit15, and the derivation unit22may determine the properties of the property items of the lesion included in the selected region. According to such a form, for example, even in a case where one medical image G0includes a plurality of lesions, it is possible to create comment-on-findings candidates for the lesion desired by the user.

Further, in each of the above exemplary embodiments, the display control unit25may generate an image in which a mark indicating the position of the lesion specified by the derivation unit22is added to the medical image G0. In the example ofFIG.5, the nodular shadow N included in the medical image G0is surrounded by a broken-line rectangle mark38. This makes it easier, for example, for a reader of the interpretation report to see the region in the image that is the basis of the lesion, without the need for the radiologist to provide comments on findings related to the position of the lesion. Therefore, it is possible to support the creation of a document such as an interpretation report. The mark38indicating the positions of the lesion is not limited to the broken-line rectangle, but may be various marks such as, for example, a polygon, a circle, an arrow, or the like, and the line type of the mark (a solid line, a broken line, and a dotted line), line color, line thickness, or the like may be changed as appropriate.

Further, in the above exemplary embodiment, the present disclosure is applied in the case where the interpretation report is created as a document and the comments on findings are generated as a character string, but the present disclosure is not limited thereto. For example, the present disclosure may be applied in the case of creating a medical document other than an interpretation report, such as an electronic medical record and a diagnosis report, and other documents including a character string related to an image. Further, for example, instead of the comment on findings, a keyword as shown in the list54of the property item44inFIG.5may be generated.

Further, although the various processes are performed using the medical image G0with a lung as the diagnosis target in the above exemplary embodiments, the diagnosis target is not limited to the lung. In addition to the lung, any part of a human body such as a heart, liver, brain, and limbs can be diagnosed. Further, although various processes are performed using one medical image G0in the above exemplary embodiments, various processes may be performed using a plurality of images such as a plurality of tomographic images relating to the same diagnosis target.

Further, in each of the above exemplary embodiments, each process of the derivation unit22and the generation unit24in the information processing apparatus20encompassed in the interpretation WS3may be performed by an external device, for example, another analysis server connected to the network10. In this case, the external device acquires the medical image G0from the image server5, and derives the property information indicating the property of the property item for each predetermined property item from the medical image G0. Further, the comment-on-findings candidate52is generated based on the information that has been selected as the description regarding the property item being necessary, unnecessary, or optional in the document, for each property item received by the information processing apparatus20. In the information processing apparatus20, the display control unit25controls the display content to be displayed on the display14based on the property information derived by the external device and the comment-on-findings candidate52generated by the external device.

In the above exemplary embodiments, for example, as hardware structures of processing units that execute various kinds of processing, such as the acquisition unit21, the derivation unit22, the reception unit23, the generation unit24, and the display control unit25, various processors shown below can be used. As described above, the various processors include a programmable logic device (PLD) as a processor of which the circuit configuration can be changed after manufacture, such as a field programmable gate array (FPGA), a dedicated electrical circuit as a processor having a dedicated circuit configuration for executing specific processing such as an application specific integrated circuit (ASIC), and the like, in addition to the CPU as a general-purpose processor that functions as various processing units by executing software (programs).

One processing unit may be configured by one of the various processors, or may be configured by a combination of the same or different kinds of two or more processors (for example, a combination of a plurality of FPGAs or a combination of the CPU and the FPGA). In addition, a plurality of processing units may be configured by one processor. As an example where a plurality of processing units are configured by one processor, first, there is a form in which one processor is configured by a combination of one or more CPUs and software as typified by a computer, such as a client or a server, and this processor functions as a plurality of processing units. Second, there is a form in which a processor for realizing the function of the entire system including a plurality of processing units via one integrated circuit (IC) chip as typified by a system on chip (SoC) or the like is used. In this way, various processing units are configured by one or more of the above-described various processors as hardware structures.

Furthermore, as the hardware structure of the various processors, more specifically, an electrical circuit (circuitry) in which circuit elements such as semiconductor elements are combined can be used.

The disclosure of JP2020-036291 filed on Mar. 3, 2020 is incorporated herein by reference in its entirety. All literatures, patent applications, and technical standards described herein are incorporated by reference to the same extent as if the individual literature, patent applications, and technical standards were specifically and individually stated to be incorporated by reference.