Patent Description:
In recent years, advances in medical devices, such as computed tomography (CT) apparatuses and magnetic resonance imaging (MRI) apparatuses, have enabled image diagnosis using high-resolution medical images with higher quality. In particular, since a region of a lesion can be accurately specified by image diagnosis using CT images, MRI images, and the like, appropriate treatment is being performed based on the specified result.

In addition, image diagnosis is made by analyzing a medical image via computer-aided diagnosis (CAD) using a learning model in which machine learning is performed by deep learning or the like, and detecting diseased regions such as a lesion included in the medical image as regions of interest from the medical image. In this way, the analysis result generated by the analysis process via CAD is saved in a database in association with examination information, such as a patient name, gender, age, and a modality which has acquired a medical image, and provided for diagnosis. A doctor interprets a medical image by referring to a distributed medical image and analysis result in his or her own interpretation terminal. At this time, in the interpretation terminal, an annotation is superimposed and displayed on the region of interest including the disease included in the medical image based on the analysis result. An annotation is a general term for figures, characters, and symbols attached to a medical image. For example, figures such as a rectangle surrounding the region of interest and an arrow indicating the region of interest, and text such as the type and size of the disease are superimposed and displayed as annotations. A radiologist refers to the annotations added to the region of interest to create an interpretation report.

Incidentally, in a case where a plurality of lesions are included in the medical image, an annotation is added to each of the lesions. Therefore, in a case where the medical image to which the annotation is added is displayed, a plurality of annotations are superimposed and displayed on the medical image. In such a case where a plurality of annotations are superimposed and displayed, it is difficult to clearly grasp the position of the lesion indicated by the displayed annotations. Therefore, a method has been proposed in which a plurality of annotations are grouped and associated with a medical image in a unit of a group to clarify the association between the plurality of annotations and the medical image (see, for example, <CIT>). Further, in the method described in <CIT>, there has been also proposed a method of further associating a character string of a finding included in an interpretation report with a medical image and an annotation.

<CIT> discloses an image interpretation support apparatus and method. The apparatus includes a line-of-sight detecting unit that detects a line of sight of a person who reads the image and a gaze determining unit that determines, in accordance with a detection result obtained by the line-of-sight detecting unit, whether a candidate lesion portion in an image to be interpreted that is displayed on a display unit has been gazed at by the person who reads the image. The apparatus further includes a completion detecting unit that detects completion of image interpretation, by the person who reads the image, of the image to be interpreted that is displayed on the display unit, and a display control unit that, in a case where the completion detecting unit detects the completion of the image interpretation and the gaze determining unit determines that the candidate lesion portion has not been gazed at, switches from a first display state to a second display state.

<CIT> discloses a computer assisted data reconciliation method and apparatus. <CIT> discloses an image management device and image display device. <CIT> discloses a medical scan interface feature evaluating system. <CIT> discloses an image processing method and computer readable medium for image processing.

On the other hand, in a case of interpreting a medical image to which a plurality of annotations are added, annotations other than the annotations added to a region of attention in the medical image interfere with the interpretation for a radiologist. In a case where the method described in <CIT> is used, it is possible to display only annotations included in a specific group, but some of them may include annotations that are not necessary for interpretation. In such a case, unnecessary annotations can be hidden and only necessary annotations can be displayed by performing an operation of clicking unnecessary annotations or clicking a region to which unnecessary annotations are added. However, since the radiologist interprets a large number of medical images every day, such an operation is very troublesome for the radiologist.

The present disclosure has been made in view of the above circumstances, and an object of the present disclosure is to enable a radiologist to refer only to an annotation about a region of attention without performing a troublesome operation.

According to an aspect of the present disclosure, there is provided an image display apparatus according to claim <NUM> of the appended claims.

In addition, in the image display apparatus according to the aspect of the present disclosure, the processor may be configured to switch to hide or weakly display the relevant information related to the regions of interest other than the region of attention.

In addition, in the image display apparatus according to the aspect of the present disclosure, the other information may be an observation condition of the medical image.

In addition, in the image display apparatus according to the aspect of the present disclosure, the processor may be configured to: display the medical image in which all relevant information is hidden; and display the relevant information related to the region of attention in a case where the region of attention is specified based on the detection result of the other operation.

In addition, in the image display apparatus according to the aspect of the present disclosure, the other operation may be an operation of inputting a comment on findings related to the medical image, and the processor may be configured to specify a region of interest related to a term included in the comment on findings as the region of attention.

In addition, in the image display apparatus according to the aspect of the present disclosure, the other operation may be an operation of further adding the relevant information to the region of interest included in the medical image, and the processor may be configured to specify the region of interest to which the relevant information is added as the region of attention.

In addition, in the image display apparatus according to the aspect of the present disclosure, the other operation may be an operation of changing a size of the medical image with reference to a designated region of interest, and the processor may be configured to specify the designated region of interest as the region of attention.

In addition, in the image display apparatus according to the aspect of the present disclosure, the processor may be configured to switch to hide the relevant information superimposed on the region of attention by the other operation.

In addition, in the image display apparatus according to the aspect of the present disclosure, the specification information may include only the other information.

In addition, in the image display apparatus according to the aspect of the present disclosure, the relevant information may include an annotation related to the region of interest.

In addition, in the image display apparatus according to the aspect of the present disclosure, the annotation may include at least one of a recognition result of the region of interest or character information related to the region of interest.

In addition, in the image display apparatus according to the aspect of the present disclosure, the region of interest may be a lesion or an organ included in the medical image.

According to another aspect, there is provided an image display method according to claim <NUM> of the appended claims.

According to another aspect, there is provided a computer-readable storage medium according to claim <NUM> of the appended claims.

According to the aspects of the present disclosure, the radiologist can refer only to the annotation about the region of attention without performing a troublesome operation.

First, a configuration of a medical information system <NUM> to which an image display apparatus according to the present embodiment is applied will be described. <FIG> is a diagram showing a schematic configuration of the medical information system <NUM>. The medical information system <NUM> shown in <FIG> is, based on an examination order from a doctor in a medical department using a known ordering system, a system for imaging an examination target part of a subject, storing a medical image acquired by the imaging, interpreting the medical image by a radiologist and creating an interpretation report, and viewing the interpretation report and observing the medical image to be interpreted in detail by the doctor in the medical department that is a request source.

As shown in <FIG>, in the medical information system <NUM>, a plurality of imaging apparatuses <NUM>, a plurality of interpretation workstations (WSs) <NUM> that are interpretation terminals, a medical care WS <NUM>, an image server <NUM>, an image database (hereinafter referred to as an image DB) <NUM>, a report server <NUM>, and a report database (hereinafter referred to as a report DB) <NUM> are communicably connected to each other through a wired or wireless network <NUM>.

Each apparatus is a computer on which an application program for causing each apparatus to function as a component of the medical information system <NUM> is installed. The application program is stored in a storage apparatus of a server computer connected to the network <NUM> or in a network storage in a state in which it can be accessed from the outside, and is downloaded to and installed on the computer in response to a request. Alternatively, the application program is recorded on a recording medium, such as a digital versatile disc (DVD) and a compact disc read only memory (CD-ROM), and distributed, and is installed on the computer from the recording medium.

The imaging apparatus <NUM> is an apparatus (modality) that generates a medical image showing a diagnosis target part of the subj ect by imaging the diagnosis target part. Specifically, examples of the modality include a simple X-ray imaging apparatus, a CT apparatus, an MRI apparatus, a positron emission tomography (PET) apparatus, and the like. The medical image generated by the imaging apparatus <NUM> is transmitted to the image server <NUM> and is saved in the image DB <NUM>.

The interpretation WS <NUM> is a computer used by, for example, a radiologist of a radiology department to interpret a medical image and to create an interpretation report, and encompasses an image display apparatus according to the present embodiment. In the interpretation WS <NUM>, a viewing request for a medical image to the image server <NUM>, various image processing for the medical image received from the image server <NUM>, display of the medical image, input reception of comments on findings regarding the medical image, and the like are performed. In the interpretation WS <NUM>, creation of an interpretation report, a registration request and a viewing request for the interpretation report to the report server <NUM>, display of the interpretation report received from the report server <NUM>, and the like are performed. The above processes are performed by the interpretation WS <NUM> executing software programs for respective processes.

The medical care WS <NUM> is a computer used by a doctor in a medical department to observe an image in detail, view an interpretation report, create an electronic medical record, and the like, and is configured to include a processing apparatus, a display apparatus such as a display, and an input apparatus such as a keyboard and a mouse. In the medical care WS <NUM>, a viewing request for the image to the image server <NUM>, display of the image received from the image server <NUM>, a viewing request for the interpretation report to the report server <NUM>, and display of the interpretation report received from the report server <NUM> are performed. The above processes are performed by the medical care WS <NUM> executing software programs for respective processes.

The image server <NUM> is a general-purpose computer on which a software program that provides a function of a database management system (DBMS) is installed. The image server <NUM> comprises a storage in which the image DB <NUM> is configured. This storage may be a hard disk apparatus connected to the image server <NUM> by a data bus, or may be a disk apparatus connected to a storage area network (SAN) or a network attached storage (NAS) connected to the network <NUM>. In a case where the image server <NUM> receives a request to register a medical image from the imaging apparatus <NUM>, the image server <NUM> prepares the medical image in a format for a database and registers the medical image in the image DB <NUM>.

Image data of the medical image acquired by the imaging apparatus <NUM> and accessory information are registered in the image DB <NUM>. The accessory information includes, for example, an image identification (ID) for identifying each medical image, a patient ID for identifying a subject, an examination ID for identifying an examination, a unique ID (unique identification (UID)) allocated for each medical image, examination date and examination time at which a medical image is generated, the type of imaging apparatus used in an examination for acquiring a medical image, patient information such as the name, age, and gender of a patient, an examination part (an imaging part), imaging information (an imaging protocol, an imaging sequence, an imaging method, imaging conditions, the use of a contrast medium, and the like), and information such as a series number or a collection number in a case where a plurality of medical images are acquired in one examination.

In addition, in a case where the viewing request from the interpretation WS <NUM> and the medical care WS <NUM> is received through the network <NUM>, the image server <NUM> searches for a medical image registered in the image DB <NUM> and transmits the searched for medical image to the interpretation WS <NUM> and to the medical care WS <NUM> that are request sources.

The report server <NUM> incorporates a software program for providing a function of a database management system to a general-purpose computer. In a case where the report server <NUM> receives a request to register the interpretation report from the interpretation WS <NUM>, the report server <NUM> prepares the interpretation report in a format for a database and registers the interpretation report in the report DB <NUM>.

In the report DB <NUM>, an interpretation report created by the radiologist using the interpretation WS <NUM> is registered. The interpretation report may include, for example, a medical image to be interpreted, an image ID for identifying the medical image, a radiologist ID for identifying the radiologist who performed the interpretation, a disease name, disease position information, information for accessing a medical image, and the like.

Further, in a case where the report server <NUM> receives the viewing request for the interpretation report from the interpretation WS <NUM> and the medical care WS <NUM> through the network <NUM>, the report server <NUM> searches for the interpretation report registered in the report DB <NUM>, and transmits the searched for interpretation report to the interpretation WS <NUM> and to the medical care WS <NUM> that are request sources.

In the present embodiment, the diagnosis target is the thoracoabdominal region of a human body, and the medical image is a three-dimensional CT image consisting of a plurality of tomographic images including the thoracoabdominal region. Then, in the interpretation WS <NUM>, the radiologist interprets the CT image to create an interpretation report including comments on findings about diseases such as lungs and livers included in the thoracoabdominal region. The medical image is not limited to the CT image, and any medical image such as an MRI image and a simple X-ray image acquired by a simple X-ray imaging apparatus can be used.

In the present embodiment, in creating the interpretation report, a medical image is analyzed and a region of interest included in the medical image is detected. The radiologist interprets the image while referring to the detection result. The detection of the region of interest will be described later.

The network <NUM> is a wired or wireless local area network that connects various apparatuses in a hospital to each other. In a case where the interpretation WS <NUM> is installed in another hospital or clinic, the network <NUM> may be configured to connect local area networks of respective hospitals through the Internet or a dedicated line.

Next, the image display apparatus according to the present embodiment will be described. <FIG> illustrates a hardware configuration of the image display apparatus according to the present embodiment. As shown in <FIG>, an image display apparatus <NUM> includes a central processing unit (CPU) <NUM>, a non-volatile storage <NUM>, and a memory <NUM> as a temporary storage area. Further, the image display apparatus <NUM> includes a display <NUM> such as a liquid crystal display, an input device <NUM> consisting of a pointing device such as a keyboard and a mouse, and a network interface (I/F) <NUM> connected to the network <NUM>. The CPU <NUM>, the storage <NUM>, the display <NUM>, the input device <NUM>, the memory <NUM>, and the network I/F <NUM> are connected to a bus <NUM>. The CPU <NUM> is an example of a processor in the present disclosure.

The storage <NUM> is realized by a hard disk drive (HDD), a solid state drive (SSD), a flash memory, and the like. An image display program <NUM> is stored in the storage <NUM> as the storage medium. The CPU <NUM> reads out the image display program <NUM> from the storage <NUM>, loads the read-out program into the memory <NUM>, and executes the loaded image display program <NUM>.

Next, a functional configuration of the image display apparatus according to the present embodiment will be described. <FIG> is a diagram showing a functional configuration of the image display apparatus according to the present embodiment. As shown in <FIG>, the image display apparatus <NUM> comprises an image acquisition unit <NUM>, an analysis unit <NUM>, a display control unit <NUM>, and a specifying unit <NUM>. Then, the CPU <NUM> executes the image display program <NUM>, so that the CPU <NUM> functions as the image acquisition unit <NUM>, the analysis unit <NUM>, the display control unit <NUM>, and the specifying unit <NUM>.

The image acquisition unit <NUM> acquires a medical image to be interpreted for creating an interpretation report from the image server <NUM> according to an instruction from the input device <NUM> by the radiologist who is an operator. A medical image to be interpreted is referred to as a target image G0 in the following description. In addition, a plurality of images may be acquired in one examination. Since all the images acquired in one examination are interpreted, there may be a plurality of target images G0. In addition, a plurality of images of a patient to be examined for interpreting the target image G0 may be acquired and saved in the image server <NUM> in the past. In the present embodiment, in a case of interpreting the target image G0 of the patient to be examined, all the images acquired by imaging the patient to be examined, including the target image G0, are acquired from the image server <NUM>. In the following description, among all images of the patient acquired from the image server <NUM>, images other than the target image G0 are acquired before the target image G0 in a timely manner, and thus are referred to as past images GP0.

In the present embodiment, the target image G0 is a three-dimensional CT image consisting of a plurality of tomographic images acquired by imaging the thoracoabdominal region of the patient to be examined. Further, in a case where a plurality of target images G0 are acquired in one examination, each target image G0 may include an X-ray image acquired by simple X-ray imaging in addition to the CT image. Moreover, the past image GP0 may also include an X-ray image acquired by simple X-ray imaging in addition to the CT image.

The analysis unit <NUM> detects a region of the abnormal shadow included in the target image G0 as a region of interest, and derives an annotation for the detected region of interest. The analysis unit <NUM> detects regions of shadows of a plurality of types of diseases as regions of interest from the target image G0 using a known computer-aided diagnosis (that is, CAD) algorithm, and derives annotations by deriving the properties of the regions of interest.

In order to detect the regions of interest and derive the annotations, the analysis unit <NUM> has a learning model 22Ain which machine learning is performed to detect abnormal shadows of a plurality of types of diseases as the regions of interest from the target image G0, and further derive properties of the abnormal shadows.

The learning model 22A consists of a convolutional neural network (CNN) in which deep learning has been performed using supervised training data to discriminate whether or not each pixel (voxel) in the target image G0 represents a shadow of various diseases or an abnormal shadow.

The learning model 22A is constructed by training CNN using, for example, supervised training data consisting of supervised training images that include abnormal shadows, a region of the abnormal shadows in the supervised training image, and correct answer data representing the properties of the abnormal shadows, and supervised training data consisting of supervised training images that do not include abnormal shadows. The learning model 22A derives the confidence degree (likelihood) indicating that each pixel in the medical image is an abnormal shadow, and detects a region consisting of pixels whose confidence degree is equal to or higher than a predetermined threshold value as a region of interest. Here, the confidence degree is a value of <NUM> or more and <NUM> or less. Further, the learning model 22A derives the properties of the detected region of interest. The properties include the position and the size of the region of interest, the type of disease, and the like.

The learning model 22A may detect a region of interest from a three-dimensional target image G0, or may detect a region of interest from each of a plurality of tomographic images constituting the target image G0. Further, as the learning model 22A, any learning model such as, for example, a support vector machine (SVM) can be used in addition to the convolutional neural network.

The analysis unit <NUM> derives, as annotations, a figure surrounding the region of interest detected by the learning model, text representing the property, a line connecting the figure and the text, and the like. The annotation is derived as an image of a channel different from that of the target image G0 and is added to the target image G0. Therefore, in a case where the image of the channel of the annotation is superimposed and displayed on the target image G0, the annotation is superimposed and displayed on the target image G0.

<FIG> is a diagram schematically showing a relationship between the target image G0 and the annotation. As shown in <FIG>, the target image G0 includes a plurality of tomographic images Dj. Further, it is assumed that an abnormal shadow is detected as a region of interest in each of tomographic image groups GA1, GA2, and GA3 of the plurality of tomographic images Dj, and an annotation is added to the detected region of interest. In the present embodiment, the tomographic image Dj represents an axial cross section of a human body, but the present disclosure is not limited thereto.

<FIG> is a diagram showing an annotation added to the tomographic image groups GA1 to GA3 shown in <FIG>. As shown in <FIG>, in a tomographic image DA1 included in the tomographic image group GA1, a region of interest R1 is detected, and an annotation <NUM> is added to the region of interest R1. The annotation <NUM> is character information related to the region of interest R1, and the content thereof is "SOL of <NUM> is found in S1". The SOL is an abbreviation for Space Occupying Lesion. The annotation <NUM> is added to all the tomographic images in which the region of interest R1 is detected, which are included in the tomographic image group GA1.

In addition, in a tomographic image DA2 included in the tomographic image group GA2, a region of interest R2 is detected in the left lung, and a region of interest R3 is detected in the right lung. In the tomographic image DA2, an annotation <NUM> and an annotation <NUM> are added to the region of interest R2 of the left lung and the region of interest R3 of the right lung, respectively. The annotation <NUM> consists of a figure 32A surrounding a region of interest of the left lung included in the tomographic image DA2, a text 32B that represents the properties of the region of interest surrounded by the figure 32A, and a leader line 32C connecting the figure 32A and the text 32B. The annotation <NUM> consists of a figure 33A surrounding a region of interest of the right lung included in the tomographic image DA2, a text 33B that represents the properties of the region of interest surrounded by the figure 33A, and a leader line 33C connecting the figure 33A and the text 33B. The texts 32B and 33B related to the region of interest are, for example, the position, the size, the property, or the like of the region of interest. The annotations <NUM> and <NUM> are added to all the tomographic images in which the regions of interest R2 and R3 are detected, respectively, which are included in the tomographic image group GA2.

Further, in a tomographic image DA3 included in the tomographic image group GA3, a region of interest R4 included in the liver is detected, and an annotation <NUM> is added to the region of interest R4 of the liver. The annotation <NUM> consists of a figure 34A surrounding a region of interest of the liver included in the tomographic image DA3, a text 34B that represents the properties of the region of interest surrounded by the figure 34A, and a leader line 34C connecting the figure 34A and the text 34B. The annotation <NUM> is added to all the tomographic images in which the region of interest R4 is detected, which are included in the tomographic image group GA3.

The display control unit <NUM> displays an image of the patient to be examined including the target image G0 and the past image GP0 on the display <NUM>. <FIG> is a diagram showing a display screen of an image. As shown in <FIG>, a display screen <NUM> includes an examination list area <NUM> for displaying an examination list, a thumbnail image area <NUM> for displaying thumbnail images of the target image G0 included in the selected examination and past images of the patient to be examined, and an image display area <NUM> for displaying an image.

In the examination list area <NUM>, a patient name, a patient ID, and an examination date and time are displayed for each examination. The operator can select the patient to be examined by clicking on a desired examination list. It should be noted that, in <FIG>, diagonal lines are added to the rows of the patient to be examined selected in the examination list area <NUM>.

In the thumbnail image area <NUM>, thumbnail images of the target images G0 of the selected patient to be examined and representative images of the past images GP0 are displayed. Here, assuming that two target images G1 and G2 and four past images GP1 to GP4 are acquired for the patient to be examined, as shown in <FIG>, six thumbnail images C1 to C6 are displayed in the thumbnail image area <NUM>. In a case where the image is a three-dimensional image, a minified image of a tomographic image of a predetermined tomographic plane included in the three-dimensional image can be used as a representative image. In the case of a simple X-ray image, the minified image can be used as a representative image. In addition, in the present embodiment, it is assumed that the thumbnail images C1 and C2 are the thumbnail images of the target images G1 and G2, and the thumbnail images C3 to C6 are the thumbnail images of the past images GP1 to GP4. Note that it is assumed that the target image G1 corresponds to the target image G0 shown in <FIG>, abnormal shadows are detected as regions of interest in the tomographic image groups GA1, GA2, and GA3 of the plurality of tomographic images Dj included in the target image G1, and annotations are added to the detected regions of interest.

The image selected in the thumbnail image area <NUM> is displayed in the image display area <NUM>. As shown in <FIG>, the image display area <NUM> includes four display areas 43A to 43D. In <FIG>, representative images of the four images selected from the six thumbnail images C1 to C6 are displayed in the display areas 43Ato 43D of the image display area <NUM>, respectively. In addition, in the thumbnail image area <NUM>, diagonal lines are added to the thumbnail images other than the images displayed in the image display area <NUM> in order to clearly indicate that the thumbnail images are not displayed in the image display area <NUM>. In the present embodiment, since the thumbnail images C1 and C2 correspond to the target images G1 and G2, respectively, the target images G1 and G2 are displayed in the display areas 43A and 43B, respectively. It is also assumed that the past images GP1 and GP2 are displayed in the display areas 43C and 43D.

Although annotations are added to the target image G1 as shown in <FIG>, the annotations are not superimposed and displayed on the representative image of the target image G1 displayed on the initial display screen <NUM> immediately after being displayed on the display <NUM>.

In the present embodiment, the radiologist interprets the target image G1 displayed on the display screen <NUM> to confirm the region of interest detected by the analysis unit <NUM>, or newly specify an abnormal shadow that the analysis unit <NUM> could not detect. At this time, the radiologist can perform a paging operation of switching the tomographic plane of the target image G1 displayed in the display area 43A by moving a mouse cursor <NUM> to the display area 43A and rotating the mouse wheel. Further, the radiologist can perform an operation of changing the size of the displayed target image G1 by performing a predetermined size change operation using the input device <NUM>. Examples of the size change operation include an operation of moving the mouse cursor <NUM> to a reference position where the size change is desired in the target image G1, and rotating the mouse wheel while pressing the ctrl key. In this case, the size of the target image G1 being displayed is changed around the cursor position. A paging operation and a size change operation are examples of other operations related to the display of the medical image.

In addition, as necessary, the radiologist can perform an operation of adding tag information indicating the property of the newly found abnormal shadow to the target image, or an operation of inputting a comment on findings about the target image G1. The operation of adding the tag information and the operation of inputting the comment on findings are examples of other operations related to the interpretation of the medical image.

The operation of adding the tag information indicating the property to the target image is an operation of displaying a property input window <NUM> shown in <FIG> in a pop-up manner by, for example, specifying a newly found abnormal shadow with the mouse cursor <NUM> and right-clicking the specified abnormal shadow. In the property input window <NUM>, for example, items of a boundary, a shape, a serrated shape, a spicula, a lobule, a straight line, an absorption value, an air bronchogram, a cavity, a calcification, a fat, a pleural invagination, and a pleural contact are shown as the properties of the abnormal shadow of the lung. Negative (-) or positive (+) can be selected for the items of the boundary, serrated shape, spicula, lobule, straight line, air bronchogram, cavity, calcification, fat, pleural invagination, and pleural contact. As for the shape item, irregular and well shape items can be selected. For the absorption value, a solid type or a frosted glass type can be selected. In addition, the operation of inputting the comment on findings is an operation of displaying a comment-on-findings input window <NUM> shown in <FIG> in a pop-up manner by right-clicking the mouse on the displayed target image.

The specifying unit <NUM> specifies a region of attention that the radiologist pays attention to among a plurality of regions of interest included in the target image. In the present embodiment, the specifying unit <NUM> specifies at least one region of attention based on specification information including other information other than information dedicated to specifying regions of attention. In the present embodiment, it is assumed that a detection result of another operation related to the display or interpretation of the medical image is used as other information. The specifying unit <NUM> specifies the region of attention based on specification information including detection results of other operations as other information. In addition, examples of the information dedicated to specifying the region of attention include a detection result of an operation of clicking the region of attention using a mouse.

For example, in the interpretation of the target image G1, in a case where the radiologist finds an abnormal shadow in a certain tomographic image, a paging operation for repeatedly interpreting the tomographic images before and after the tomographic image including the abnormal shadow is performed. The paging operation for repeatedly interpreting the images is an operation of repeatedly switching the tomographic plane back and forth in the body axis direction of the human body by rotating the mouse wheel back and forth. In a case where such a paging operation is detected and the tomographic image repeatedly displayed during the paging operation includes an annotation, the specifying unit <NUM> specifies a region of interest to which the annotation is added as the region of attention that the radiologist pays attention to.

For example, in a case where the paging operation is performed in the vicinity of the tomographic image group GA1 of the target image G1 shown in <FIG>, the specifying unit <NUM> specifies the region of interest R1 included in the tomographic image DA1 shown in <FIG> as the region of attention. Further, in a case where the paging operation is performed in the vicinity of the tomographic image group GA2 in the target image G1 shown in <FIG>, the specifying unit <NUM> specifies the region of interest R2 and the region of interest R3 included in the tomographic image DA2 shown in <FIG> as the regions of attention. Further, in a case where the paging operation is performed in the vicinity of the tomographic image group GA3 in the target image G1 shown in <FIG>, the specifying unit <NUM> specifies the region of interest R4 included in the tomographic image DA3 shown in <FIG> as the region of attention.

The display control unit <NUM> displays an annotation added to the region of attention specified by the specifying unit <NUM> on the display screen <NUM>. That is, in a case where the paging operation is performed in the vicinity of the tomographic image group GA1 in the target image G1 displayed in the display area 43A, the annotation <NUM> is superimposed and displayed on the target image G1 (the tomographic image DA1) as shown in <FIG>. In addition, in a case where the paging operation is performed in the vicinity of the tomographic image group GA2 in the target image G1, the annotations <NUM> and <NUM> are superimposed and displayed on the target image G1 (the tomographic image DA2) as shown in <FIG>. In a case where the paging operation is performed in the vicinity of the tomographic image group GA3 in the target image G1, the annotation <NUM> is superimposed and displayed on the target image G1 (the tomographic image DA3) as shown in <FIG>.

In addition, in a case where the paging operation is stopped, the display control unit <NUM> may continue to display the annotation or may hide the annotation. Alternatively, the annotation may be displayed for a predetermined time and then switched to be hidden.

On the other hand, in a case where the radiologist finds the abnormal shadow in the target image G1, the radiologist may enlarge and display the abnormal shadow. In that case, the radiologist performs an operation of enlarging the displayed target image G1. For example, in the target image G1 (the tomographic image DA2) on which the annotations <NUM> and <NUM> are superimposed and displayed as shown in <FIG>, in a case where the radiologist desires to enlarge the target image G1 with reference to the region of interest R3 to which the annotation <NUM> is added, as shown in <FIG>, the radiologist moves the mouse cursor <NUM> to the region of interest R3 and rotates the mouse wheel while pressing a specific key such as the ctrl key. The specifying unit <NUM> specifies the region of interest R3 where the mouse cursor <NUM> is located as the region of attention based on the detection result of such an operation of changing the size.

The display control unit <NUM> displays an annotation added to the region of attention specified by the specifying unit <NUM> on the display screen <NUM>. Here, as shown in <FIG>, in a case where two regions of interest R2 and R3 are included in the tomographic image DA2 being displayed and the annotations <NUM> and <NUM> are superimposed and displayed on each of the regions of interest R2 and R3, it is assumed that the region of interest R3 is specified as the region of attention. In this case, the display control unit <NUM> displays only the annotation <NUM> for the region of interest R3 and hides the annotation <NUM> for the region of interest R2. Accordingly, as shown in <FIG>, the target image G1 (the tomographic image DA2) enlarged centering on the region of interest R3 is displayed in the display area 43A, the annotation <NUM> is hidden, and only the annotation <NUM> added to the region of interest R3 is displayed.

In addition, in a case where the operation of changing the size is stopped, the display control unit <NUM> may continue to display the annotation or may hide the annotation. Alternatively, the annotation may be displayed for a predetermined time and then switched to be hidden.

In addition, in a case where the radiologist finds an abnormal shadow in the target image G1, the radiologist may display the comment-on-findings input window <NUM> shown in <FIG> in a pop-up manner and input a comment on findings about the abnormal shadow. The specifying unit <NUM> may specify a region of attention based on a detection result of an operation of inputting a comment on findings by the radiologist. Specifically, the specifying unit <NUM> may analyze the input comment on findings and specify a region of interest corresponding to the term included in the comment on findings as the region of attention. Here, as shown in <FIG>, in a case where two regions of interest R2 and R3 are included in the tomographic image DA2 being displayed and the annotations <NUM> and <NUM> are superimposed and displayed on each of the regions of interest R2 and R3, it is assumed that the input comment on findings is "a nodule is seen in the left lung". The specifying unit <NUM> analyzes the comment on findings and specifies a part "left lung". The specifying unit <NUM> specifies the region of interest R2 of the left lung included in the tomographic image DA2 as the region of attention based on the specified term "left lung". Note that the specifying unit <NUM> may specify a term "nodule" included in the comment on findings, refer to the description of the texts 32B and 33B of the annotations <NUM> and <NUM> added to the regions of interest R2 and R3, and specify a region of interest to which an annotation including a description regarding a nodule is added as the region of attention.

In this case, the display control unit <NUM> displays only the annotation <NUM> for the region of interest R2 and hides the annotation <NUM> for the region of interest R3. Accordingly, as shown in <FIG>, the comment-on-findings input window <NUM> is displayed in the display area 43A, the annotation <NUM> added to the region of interest R3 is hidden based on the comment on findings of "a nodule is seen in the left lung" input in the comment-on-findings input window <NUM>, and only the annotation <NUM> added to the region of interest R2 is displayed.

It is assumed that the radiologist further inputs a comment on findings of "a cavity is also seen" in the comment-on-findings input window <NUM>. In this case, the specifying unit <NUM> analyzes the newly input comment on findings and specifies the term "cavity". In this case, as shown in <FIG>, in a case where the text 33B of the annotation <NUM> added to the region of interest R3 includes a description regarding the cavity, the annotation <NUM> added to the region of interest R3 may be further displayed.

In this case, in a case where an input cursor <NUM> is located at the position of "a cavity is also seen" in the comment-on-findings input window <NUM>, as shown in <FIG>, the display control unit <NUM> may display the annotation <NUM> added to the region of interest R3 corresponding to the comment on findings of "a cavity is also seen", and hide the annotation <NUM> added to the region of interest R2. In addition, in a case where the input cursor <NUM> is located in the comment on findings of "a nodule is seen in the left lung", the display control unit <NUM> may display the annotation <NUM> added to the region of interest R2 corresponding to the comment on findings of "a nodule is seen in the left lung", and hide the annotation <NUM> added to the region of interest R3.

In a case where the operation of inputting the comment on findings is stopped, the display control unit <NUM> may continue to display the annotation or may hide the annotation. Alternatively, the annotation may be displayed for a predetermined time and then hidden.

Further, the radiologist may perform an operation of inputting a property in a case where an abnormal shadow is found in the target image G1. In this case, the radiologist performs an operation of displaying the property input window <NUM> shown in <FIG> in a pop-up manner by specifying a found abnormal shadow with the mouse cursor <NUM> and right-clicking the specified abnormal shadow. The specifying unit <NUM> specifies the region of interest where the mouse cursor <NUM> is located as the region of attention based on the detection result of the operation. For example, in a case where an operation of inputting properties to the region of interest R2 is performed in the target image G1 (tomographic image DA2) on which the annotations <NUM> and <NUM> are superimposed and displayed as shown in <FIG>, the radiologist moves the mouse cursor <NUM> to the region of interest R2 and right-clicks. Upon detecting this operation, the specifying unit <NUM> specifies the region of interest R2 where the mouse cursor <NUM> is located as the region of attention.

In this case, as shown in <FIG>, the display control unit <NUM> hides the annotation <NUM> added to the region of interest R3 in the target image G0 (the tomographic image DA2), and displays only the annotation <NUM> added to the region of interest R2. The radiologist can see the annotation <NUM> displayed for the region of interest R2 and add the missing property, or correct the erroneous property.

In addition, in a case where the operation of inputting the property is stopped, the display control unit <NUM> may continue to display the annotation or may hide the annotation. Alternatively, the annotation may be displayed for a predetermined time and then hidden.

Next, a process performed in the present embodiment will be described. <FIG> is a flowchart showing a process performed in the present embodiment. It is assumed that a plurality of images including the target image G0 and the past image GP0 of the patient to be examined are acquired from the image server <NUM> and are saved in the storage <NUM>. Further, it is assumed that the display control unit <NUM> displays the display screen <NUM> on the display <NUM>, the examination list is selected on the display screen <NUM>, and the image related to the selected examination is displayed in the thumbnail image area <NUM>. In the thumbnail image area <NUM>, in a case where an image to be displayed in the image display area <NUM> is selected (Step ST1; affirmative), the display control unit <NUM> displays the selected image in the image display area <NUM> (Step ST2). At this time, the display control unit <NUM> first hides the annotation even on the image in which the region of interest is detected (Step ST3).

In this state, the radiologist can perform operations such as performing a paging operation or the like, inputting a comment on findings, or adding a property tag to the newly found abnormal shadow.

Next, the specifying unit <NUM> determines whether or not another operation related to the display of the target image or another operation related to the interpretation of the image is detected (another operation detection; Step ST4). In a case where the determination in Step ST4 is affirmative, the specifying unit <NUM> specifies a region of attention among a plurality of regions of interest based on specification information including other information that is a detection result of another operation (Step ST5).

The display control unit <NUM> switches to display the annotation for the specified region of attention (Step ST6). Accordingly, the radiologist can interpret the region of interest with reference to the displayed annotation. In addition, it is possible to perform operations such as inputting a comment on findings or adding a property tag to the newly found abnormal shadow. The input comment on findings is saved in the storage <NUM> as an interpretation report. Further, the property tag is added to the target image G0 as tag information.

Subsequently, the display control unit <NUM> determines whether or not another operation has been ended (Step ST7), and in a case where the determination in Step ST7 is negative, the process returns to Step ST6. In a case where the determination in Step ST7 is affirmative, the annotation is switched to be hidden (Step ST8). In a case where the determinations in step ST4 are negative, and in a case where, after Step ST8, it is determined whether or not an instruction to end interpretation has been given (Step ST9) and the determination in Step ST9 is negative, the process returns to Step ST4. In a case where the determination in Step ST9 is affirmative, the process ends. The interpretation report generated by the interpretation of the target image G0 and the target image G0 to which the property tag is added are transferred to the report server <NUM>. In the report server <NUM>, the transferred interpretation report is saved together with the target image G0.

In a case where the determination in Step ST7 is affirmative, the annotation may be switched to be hidden after a predetermined time has elapsed. Alternatively, the display of the annotation may be continued. In a case where the display of the annotation is continued, another operation is detected, the region of attention is specified according to the detected operation, and the annotation related to the specified region of attention is displayed.

In this way, in the present embodiment, at least one region of attention is specified and the annotation for the specified region of attention is switched to be displayed based on specification information including other information other than the information dedicated to specifying the region of attention that the user pays attention to. Therefore, the radiologist can refer only to the annotation about the region of attention without performing a troublesome operation.

In particular, by hiding the annotations related to the region of interest other than the region of attention, it is possible to prevent the interpretation of the region of interest of attention from being hindered by the annotation about the region of interest that is not of attention.

In addition, by specifying the region of attention based on the detection results of other operations related to the display of the medical image or the interpretation of the medical image, the radiologist can specify the region of attention and display an annotation related to the region of attention without performing a dedicated operation for specifying the region of attention.

In the above embodiment, all the annotations are hidden in a case where the target image G0 is displayed, but the present disclosure is not limited thereto. In a case where the target image G0 is displayed, all the annotations may be displayed. In this case, in a case where the region of attention is specified based on specification information such as a detection result of another operation, it is sufficient that the display control unit <NUM> displays only the annotations related to the region of attention by switching to hide the annotations related to the region of interest other than the region of attention.

In addition, in the above-described embodiment, the annotation related to the region of attention is displayed and the annotation related to the region of interest other than the region of attention is hidden, but the present disclosure is not limited thereto. Annotations related to regions of interest other than the region of attention may be weakly displayed. Here, the "weak display" refers to making the degree of display weaker than that of the displayed annotation, such as increasing the transparency of the annotation, displaying the annotation with a broken line, making the density lower than that of the displayed annotation, making the color lighter, and the like. For example, in the tomographic image DA2 shown in <FIG>, in a case where the region of interest R2 is specified as the region of attention, as shown in <FIG>, the annotation <NUM> related to the region of interest R2 may be displayed and the annotation <NUM> related to the region of interest R3 may be displayed with a broken line.

In addition, in the above-described embodiment, the detection result of another operation is used as other information included in the specification information, but the present disclosure is not limited thereto. The region of attention may be specified by using the observation condition of the target image G0 as other information. Here, in a case where the target image G0 is a CT image, a window level (WL) and a window width (WW) are used as observation conditions in the case of displaying the CT image. WL is a CT value that is the center of the part to be observed in the gradation displayable by the display <NUM> in a case where the CT image is displayed on the display <NUM>. WW is a width between a lower limit value and an upper limit value of the CT value of the part to be observed.

As such observation conditions, there are known observation conditions according to parts, such as a mediastinum condition that facilitates observation of bones and abdominal organs and a lung field condition that facilitates observation of lung fields. In addition to CT images, other medical images such as MRI images and radiation images acquired by simple radiography are also displayed under observation conditions that facilitate observation of specific parts. Therefore, in the specifying unit <NUM>, an organ to be observed may be specified according to the observation condition, and the region of interest included in the specified organ may be specified as the region of attention.

For example, in a tomographic image DA4 including a part of the liver and the left lung as shown in <FIG>, it is assumed that a region of interest R5 is detected in the left lung and a region of interest R6 is detected in the liver. Further, it is assumed that an annotation <NUM> is added to the region of interest R5 of the left lung and an annotation <NUM> is added to the region of interest R6 of the liver. In this case, in a case where the observation condition is the mediastinum condition, the specifying unit <NUM> specifies the region of interest R6 of the liver as the region of attention. Therefore, as shown in <FIG>, the display control unit <NUM> displays the annotation <NUM> added to the region of interest R6 and hides the annotation <NUM> added to the region of interest R5.

In addition, although the annotation related to the region of attention is displayed in the above-described embodiment, the annotation superimposed on the region of attention may be switched to be hidden in the annotation related to the region of attention. For example, as shown in <FIG>, in a case where the region of interest R4 included in the tomographic image DA3 is specified as the region of attention, in the annotation <NUM>, the figure 34A and the leader line 34C surrounding the region of interest R4 may be hidden, and only the text 34B outside the region of the human body may be displayed.

Further, in the above-described embodiment, the annotation is used as relevant information, but the present disclosure is not limited thereto. In addition to the annotation, examples of relevant information related to the region of interest include a region extraction result of a structure included in the target image G0 or the like. For example, as shown in <FIG>, a mask <NUM> for a region of the liver included in the target image G0 may be superimposed and displayed on a tomographic image DA5 of the target image as a region extraction result. In this case, in a case where the region of interest is the liver and the relevant information is the mask <NUM> for the liver, the mask <NUM> may be displayed on the liver based on detection results of another operation related to the display of the image or another operation related to the interpretation of the image.

In addition, in the above-described embodiment, the display and hiding of the annotation only for the target image G0 (G1) displayed in the display area 43A has been described, but in a case where annotations are also added to the images displayed in the other display areas 43B to 43D, the annotations may be switched between display and hiding in the same manner as the display area 43A.

In addition, in the above-described embodiment, the region of attention is specified as the specification information based on information other than the information dedicated to specifying the region of attention that the user pays attention to, but the specification information is not limited thereto. The detection result of a dedicated operation for specifying the region of attention, such as a mouse click by the radiologist, may also be used as the specification information. For example, as shown in <FIG>, it is assumed that the pleura of the lung field included in a tomographic image DA6 includes a region of interest R7, and the mediastinum includes a region of interest R8 adjacent to the region of interest R7. In this case, in a case where the radiologist selects the region of interest R7 in the lung field by clicking the mouse, the radiologist may erroneously select the region of interest R8 adjacent to the region of interest R7. In such a case, the specifying unit <NUM> may use the detection result of the operation of the radiologist clicking the mouse to select the region of interest and the observation condition of the target image G0 including the tomographic image DA6 as specification information, and display the annotation.

For example, in a case where the observation condition of the target image G0 is the lung field condition, the region of interest that the radiologist pays attention to is included in the lung field. Therefore, in a case where the observation condition is the lung field condition, the specifying unit <NUM> determines that the selected region of interest is the region of interest R7 included in the lung field, and specifies the region of interest R7 as the region of attention based on the detection results of the operation of selecting the observation condition and the region of attention. In this case, as shown in <FIG>, an annotation <NUM> added to the region of interest R7 is displayed. Also, in a case where the observation condition is the mediastinum condition, the specifying unit <NUM> determines that the selected region of interest is the region of interest R8 included in the mediastinum, and specifies the region of interest R8 as the region of attention based on the detection results of the operation of selecting the observation condition and the region of attention. In this case, it is sufficient to display only the annotation added to the region of interest R8.

Further, in the above-described embodiment, a plurality of images are displayed on the display screen <NUM> at the same time, but the present disclosure is not limited thereto. The technology of the present disclosure can also be applied to a case where only one image is displayed on the display screen <NUM>.

Further, in the above-described embodiment, the image display apparatus included in the interpretation WS <NUM> according to the present embodiment comprises the analysis unit <NUM> so that the interpretation WS <NUM> analyzes the target image, but the present disclosure is not limited thereto. The target image may be analyzed by an external analysis device different from the interpretation WS <NUM>, and the analysis result may be acquired by the interpretation WS <NUM>. In addition, an annotation may already be superimposed on the image saved in the image server <NUM>. Even in such a case, as in the above-described embodiment, it is possible to display the image on which the annotation is superimposed in the interpretation WS <NUM> and to switch to hide the annotation based on detection results of another operation related to the display of the image or another operation related to the interpretation of the image.

In each of the above embodiments, for example, as hardware structures of processing units that execute various kinds of processing, such as the image acquisition unit <NUM>, the analysis unit <NUM>, the display control unit <NUM>, and the specifying unit <NUM>, various processors shown below can be used. As described above, the various processors include a programmable logic device.

(PLD) as a processor of which the circuit configuration can be changed after manufacture, such as a field programmable gate array (FPGA), a dedicated electrical circuit as a processor having a dedicated circuit configuration for executing specific processing such as an application specific integrated circuit (ASIC), and the like, in addition to the CPU as a general-purpose processor that functions as various processing units by executing software (programs).

One processing unit may be configured by one of the various processors, or may be configured by a combination of the same or different kinds of two or more processors (for example, a combination of a plurality of FPGAs or a combination of the CPU and the FPGA). In addition, a plurality of processing units may be configured by one processor.

As an example in which a plurality of processing units are configured by one processor, first, there is a form in which one processor is configured by a combination of one or more CPUs and software as typified by a computer, such as a client or a server, and this processor functions as a plurality of processing units. Second, there is a form in which a processor for realizing the function of the entire system including a plurality of processing units via one integrated circuit (IC) chip as typified by a system on chip (SoC) or the like is used. As described above, various processing units are configured by using one or more of the above-described various processors as hardware structures.

Claim 1:
An image display apparatus (<NUM>) comprising at least one processor (<NUM>),
wherein the processor (<NUM>) is configured to:
superimpose and display (ST2) relevant information related to each of a plurality of regions of interest included in a medical image on the medical image to be switchable between display and hiding, wherein the medical image is a three-dimensional image including a plurality of tomographic images;
specify (ST5) at least one region of attention based on specification information including other information other than information dedicated to specifying regions of attention that a user pays attention to among the plurality of regions of interest, wherein the other information is a detection result of another operation related to display or interpretation of the medical image; and
switch (ST6) to display relevant information related to the specified region of attention;
wherein the other operation is a paging operation on the tomographic image, and
the processor (<NUM>) is configured to specify a region of interest included in the tomographic image being paging-operated as the region of attention.