Patent ID: 12239487

DETAILED DESCRIPTION

Hereinafter, an embodiment of the present disclosure will be described with reference to the drawings.FIG.1is a hardware configuration diagram illustrating the outline of a diagnosis support system to which an ultrasonography apparatus according to the embodiment of the present disclosure is applied. As illustrated inFIG.1, in a diagnosis support system1, an ultrasonography apparatus2and an image storage server3according to this embodiment are connected to each other through a network4such that they can communicate with each other.

An ultrasound probe5is connected to the ultrasonography apparatus2. The ultrasound probe5transmits ultrasonic waves to a part of a subject H to be measured in a state in which the ultrasound probe5is in contact with a body surface of the subject H, receives the ultrasonic waves reflected in the subject H, converts the received ultrasonic waves into an electric signal, and transmits the electric signal to the ultrasonography apparatus2. The ultrasonography apparatus2images the received electric signal to acquire an ultrasound image. The ultrasound image is a tomographic image of a cross section which extends from an imaging position of the ultrasound probe5in a depth direction in the subject H.

The image storage server3is a computer that stores and manages various types of data and that comprises a high-capacity external storage device and database management software. The image storage server3communicates with the ultrasonography apparatus2through the wired or wireless network4to transmit and receive, for example, the ultrasound image acquired by the ultrasonography apparatus2and a diagnostic log using the ultrasound image created by the ultrasonography apparatus2. The image storage server3stores the received ultrasound image, diagnostic log, and the like in a recording medium, such as a high-capacity external storage device, and manages them. In addition, the storage format of image data of the ultrasound image and the communication between the apparatuses through the network4are based on a protocol such as digital imaging and communication in medicine (DICOM). Further, the image storage server3is an example of an external device.

The ultrasound image and the diagnostic log are stored in the image storage server3in association with each other for each examination. In one examination, a plurality of ultrasound images are acquired as an ultrasound image group. Therefore, the diagnostic log is stored in the image storage server3in association with the ultrasound image group. Here, the plurality of ultrasound images stored in the image storage server3are used for diagnosis, and the diagnostic log is created. Therefore, in the following description, a plurality of ultrasound images acquired and stored in the image storage server3in one examination are referred to as a diagnosed ultrasound image group. In addition, one diagnosed ultrasound image group may include a sub-image group that consists of a plurality of ultrasound images acquired at different imaging positions on the body surface of the subject H.

Findings which are interpretation results of the ultrasound images by an examiner are described in the diagnostic log. The findings include information of, for example, the location of anatomical features and the type and size of diseases observed in the ultrasound image group. For example, in the case of an ultrasound image of a heart, findings including information indicating the anatomical features of the heart, such as the size of a left auricle, and the location of a disease, such as a thrombus, in a case in which the disease is observed in the ultrasound image are described in the diagnostic log. In addition, the diagnostic log is created by the ultrasonography apparatus2.

Next, the ultrasonography apparatus according to this embodiment will be described.FIG.2illustrates a hardware configuration of the ultrasonography apparatus according to this embodiment. As illustrated inFIG.2, the ultrasonography apparatus2includes a central processing unit (CPU)11, a non-volatile storage13, and a memory16as a temporary storage area. In addition, the ultrasonography apparatus2includes a display14, such as a liquid crystal display, an input device15, such as a keyboard and a mouse, and a network interface (I/F)17that is connected to the network4. The CPU11, the storage13, the display14, the input device15, the memory16, and the network I/F17are connected to a bus18. In addition, the CPU11is an example of a processor.

The storage13is implemented by, for example, a hard disk drive (HDD), a solid state drive (SSD), and a flash memory. An ultrasonography program12is stored in the storage13as a storage medium. The CPU11reads the ultrasonography program12from the storage13, develops the ultrasonography program12in the memory16, and executes the developed ultrasonography program12.

Next, a functional configuration of the ultrasonography apparatus according to this embodiment will be described.FIG.3is a diagram illustrating the functional configuration of the ultrasonography apparatus according to this embodiment. As illustrated inFIG.3, the ultrasonography apparatus2comprises an imaging unit21, a search unit22, an imaging position specification unit23, a probe position specification unit24, a notification unit25, a diagnostic log creation unit26, and a communication unit27. The CPU11executes the ultrasonography program12to function as the imaging unit21, the search unit22, the imaging position specification unit23, the probe position specification unit24, the notification unit25, the diagnostic log creation unit26, and the communication unit27.

The imaging unit21images the electric signals input from the ultrasound probe5, which is moved along the body surface of the subject H, at a predetermined frame rate (for example, 30 fps) and sequentially acquires a plurality of ultrasound images. In addition, the imaging unit21stores, as an ultrasound image group, a plurality of ultrasound images acquired for a period from the start of storage to the end of the storage in the storage13in response to storage start and end instructions input by the examiner through the input device15. In this case, the acquired ultrasound image group is stored as one image file in the storage13in association with an examination ID for specifying the examination that is currently being performed. In addition, the ultrasound image group for the examination that is currently being performed is referred to as an ultrasound image group during diagnosis.

The search unit22searches for a similar ultrasound image group that is similar to the features of the ultrasound image group during diagnosis acquired by the imaging unit21with reference to the image storage server3.FIG.4is a diagram schematically illustrating a file configuration of a plurality of diagnosed ultrasound image groups stored in the image storage server3. As illustrated inFIG.4, a plurality of diagnosed ultrasound image groups30-1,30-2,30-3, . . . are stored in the image storage server3. One diagnosed ultrasound image group includes a plurality of ultrasound images acquired in one examination. In one examination, a plurality of ultrasound images are acquired at different imaging positions. Therefore, one diagnosed ultrasound image group includes a plurality of sub-image groups31-1,31-2,31-3, . . . that include a plurality of ultrasound images acquired at different imaging positions. In addition,FIG.4illustrates that the sub-image group31-1includes a plurality of ultrasound images32-1,32-2,32-3, . . . . In addition, a diagnostic log33for the examination is associated with one diagnosed ultrasound image group. InFIG.4, the diagnostic log33is illustrated to be included in the diagnosed ultrasound image group. However, the diagnostic log and the diagnosed ultrasound image group may be stored as separate files in the image storage server3.

Further, in this embodiment, one diagnosed ultrasound image group may constitute one image file. In this case, the number of ultrasound images in the diagnosed ultrasound image group, the number of ultrasound images in each of the sub-image groups, and the imaging date and time of each of the sub-image groups are described in a tag of the image file. Meanwhile, each of the plurality of sub-image groups may constitute one image file, and the plurality of sub-image groups may be stored in a folder for one examination.

The search unit22specifies the features of the ultrasound image group during diagnosis in order to search for a similar ultrasound image group. For example, the number of ultrasound images included in the ultrasound image group during diagnosis, anatomical features included in a plurality of ultrasound images, and the plurality of ultrasound images can be used as the features of the ultrasound image group during diagnosis.

Here, the ultrasound image group during diagnosis is stored as one image file consisting of a plurality of ultrasound images in the storage13. Tag information is given to the image file. For example, the imaging date and time of the ultrasound image group during diagnosis and the number of images are described in the tag information. The search unit22acquires the number of ultrasound images included in the ultrasound image group during diagnosis with reference to the tag information given to the image file of the ultrasound image group during diagnosis. In addition, in some cases, for a specific disease, a larger number of ultrasound images are acquired as compared to other diseases. Therefore, the number of ultrasound images is a feature for the ultrasound image group.

Further, the search unit22performs a region detection process on the ultrasound images included in the ultrasound image group during diagnosis to detect anatomical features included in a plurality of ultrasound images. Furthermore, the search unit22has a trained model, such as a neural network, which has been subjected to machine learning to detect anatomical features included in the heart, a liver, a kidney, blood vessels, and the like and detects the anatomical features included in the ultrasound images using the trained model. In addition, the search unit22may perform template matching using a template that indicates the shape of the anatomical features included in the heart, the liver, the kidney, the blood vessels, and the like to specify the anatomical features, instead of using the trained model. In addition, the search unit22may detect the anatomical features from only one ultrasound image included in the ultrasound image group during diagnosis. However, in a case in which the anatomical features are detected from only one ultrasound image, the accuracy of detecting the anatomical features may be low. Therefore, the anatomical features may be detected from some of the plurality of ultrasound images.

The search unit22compares the number of ultrasound images included in the ultrasound image group during diagnosis with the number of ultrasound images included in each of the plurality of diagnosed ultrasound image groups stored in the image storage server3or with the number of ultrasound images included in each of the sub-image groups constituting the diagnosed ultrasound image group and derives a similarity that has a larger value as the difference between the numbers of ultrasound images becomes smaller as a first similarity S1with the diagnosed ultrasound image group or with the sub-image group. For example, the reciprocal of (the absolute value of the difference between the numbers of ultrasound images+1) can be used as the first similarity S1. In a case in which the difference between the numbers of ultrasound images is 0, the first similarity S1is 1 which is the maximum value. In addition, in this embodiment, the search unit22compares the number of ultrasound images included in the ultrasound image group during diagnosis with the number of ultrasound images included in the sub-image group constituting the diagnosed ultrasound image group. However, the present disclosure is not limited thereto.

Further, the search unit22specifies a diagnosed ultrasound image group, in which anatomical features matched with the anatomical features detected for the ultrasound image group during diagnosis are described in the diagnostic log, with reference to the diagnostic log for each of the plurality of diagnosed ultrasound image groups stored in the image storage server3. Furthermore, the search unit22derives a second similarity based on a correlation value between the ultrasound images included in the ultrasound image group during diagnosis and the ultrasound images included in each of the plurality of diagnosed ultrasound image groups. For example, the search unit22registers each of the ultrasound images included in the ultrasound image group during diagnosis with each of the ultrasound images included in the diagnosed ultrasound image group or the sub-image group and derives, as the correlation value, the absolute value of the difference between the pixel values of the corresponding pixels of both images after the registration. Then, the search unit22normalizes the correlation value to a value of 0 to 1 to derive the second similarity S2. In addition, the search unit22derives the second similarity S2based on the correlation value between the ultrasound image included in the ultrasound image group during diagnosis and the ultrasound image included in the sub-image group constituting the diagnosed ultrasound image group. However, the present disclosure is not limited thereto.

The search unit22calculates the sum S1+S2of the first and second similarities for the diagnosed ultrasound image group, in which the anatomical features matched with the anatomical features detected for the ultrasound image group during diagnosis are described in the diagnostic log, and sorts the diagnosed ultrasound image groups in descending order of the sum S1+S2. Then, the search unit22searches for a predetermined number of (one or more) diagnosed ultrasound image groups having the largest sum S1+S2as the similar ultrasound image groups. In addition, the search unit22may search for the similar ultrasound image group using only the first similarity S1or only the second similarity S2.

The imaging position specification unit23specifies the imaging position on the body surface of the subject H where a disease predicted in the subject H can be imaged, using positional information indicating the position of the disease included in the diagnostic log for the similar ultrasound image group. That is, the imaging position specification unit23determines whether or not a disease name is included, with reference to the diagnostic logs for all of the searched similar ultrasound image groups. In a case in which the disease name is included, a description of positional information indicating the position of the disease is included in the diagnostic log. Therefore, in a case in which the disease name is included in the diagnostic log for the similar ultrasound image group, the imaging position specification unit23acquires the disease name and the positional information from the diagnostic log. In a case in which the disease name is not included in the diagnostic logs for all of the searched similar ultrasound image groups, the imaging position specification unit23does not specify the imaging position.

FIG.5is a diagram illustrating a diagnostic log for a certain similar ultrasound image group. As illustrated inFIG.5, “Intracardiac thrombus is observed at the distal end of the cardiac apex” is described in a diagnostic log35. In this case, the imaging position specification unit23acquires “intracardiac thrombus” as the disease name and information of “the distal end of the cardiac apex” as the positional information from the diagnostic log35. Therefore, the occurrence of the intracardiac thrombus at the distal end of the cardiac apex is predicted in the subject H from the diagnostic log35for the similar ultrasound image group.

Further, the imaging position specification unit23specifies the position on the body surface of the subject H where the disease predicted in the subject H can be imaged, using the positional information acquired from the diagnostic log35.FIG.6is a diagram illustrating the specification of the position on the body surface of the subject H where anatomical features can be imaged. In addition, here, it is assumed that the positional information acquired from the diagnostic log35is the “distal end of the cardiac apex”. As illustrated inFIG.6, the imaging position specification unit23specifies an imaging position38where the distal end of the cardiac apex can be imaged in a schema37which is an image schematically illustrating a chest of the subject H.

The probe position specification unit24specifies the current position of the ultrasound probe5on the body surface of the subject. In this embodiment, as illustrated inFIG.1, a sensor6that detects the position of the ultrasound probe5is attached to the ultrasound probe5. The sensor6is a 6-axis sensor that detects a moving direction, orientation, and rotation of the ultrasound probe5and that calculates a moving distance, a moving speed, and the like. Specifically, the 6-axis sensor is implemented by a combination of an acceleration sensor that can detect three directions of a front-rear direction, a left-right direction, and an up-down direction and a geomagnetic sensor that can detect north, south, east, and west or by a combination of an acceleration sensor and a gyro sensor that can detect the speed of rotation.

The probe position specification unit24acquires the current position of the ultrasound probe5on the body surface of the subject H on the basis of output information from the sensor6.

In addition, instead of the 6-axis sensor, such as the sensor6, the magnetic sensor described in JP2011-167331A may be used. In addition, a marker that can detect a position in the directions of three axes in a space may be attached to the ultrasound probe5, and a captured image of the marker may be analyzed to specify the position of the ultrasound probe5. Further, a method for specifying the position of the ultrasound probe5is not limited thereto, and any method can be used.

The notification unit25notifies the examiner of the imaging position specified by the imaging position specification unit23. Specifically, the notification unit25notifies of the imaging position by displaying the schema which schematically shows the body surface of the subject H on the display14and superimposing the current position of the ultrasound probe5and the imaging position on the displayed schema.FIG.7is a diagram illustrating an imaging position notification screen. As illustrated inFIG.7, a notification screen40has a first image region41and a second image region42. An ultrasound image that is currently being captured is displayed in the first image region41. The schema37illustrated inFIG.6is displayed in the second image region42. A mark43is superimposed on the schema37at the imaging position38illustrated inFIG.6. Further, in addition to the mark43, an icon44indicating the ultrasound probe5is displayed at the current position of the ultrasound probe5specified by the probe position specification unit24. In addition, a findings region46in which the examiner describes findings about the ultrasound image group is displayed on the notification screen40. The examiner can input the findings about the ultrasound image group to the findings region46using the input device15.

The examiner can see the notification screen40to visually recognize at a glance how the ultrasound probe5can be moved to the imaging position where a disease is expected.

The diagnostic log creation unit26describes the findings, which have been input to the findings region46by the examiner, in the diagnostic log to create the diagnostic log. For example, inFIG.7, since “a thrombus is observed in the left auricle” is input as the findings, the diagnostic log creation unit26describes “a thrombus is observed in the left auricle” in the diagnostic log.

In addition, in this embodiment, whenever the imaging position is notified of, the ultrasound image group is acquired at each of a plurality of imaging positions on the body surface of the subject H. Therefore, in this embodiment, whenever a new ultrasound image group is acquired, the search unit22searches for a new similar ultrasound image group. For example, in a case in which the examiner moves the ultrasound probe5to the imaging position indicated by the mark43to acquire a new ultrasound image group, the search unit22searches for a new similar ultrasound image group, using the previously acquired ultrasound image group and the new ultrasound image group. In this case, there is a high probability that the new similar ultrasound image group searched for on the basis of the features of the new ultrasound image group will include the ultrasound image obtained by imaging a part which is different from that in the previous similar ultrasound image group. Therefore, there is a high probability that the diagnostic log of the new similar ultrasound image group will include a disease which is not included in the diagnostic log of the previous similar ultrasound image group. Therefore, the imaging position specification unit23can specify a new imaging position, where a predicted disease can be imaged, with reference to the diagnostic log of the new similar ultrasound image group. Then, the notification unit25notifies of the new imaging position such that the examiner can move the ultrasound probe5to the new imaging position to further acquire a new ultrasound image group. The newly acquired ultrasound image group becomes an ultrasound image group during diagnosis at the new imaging position.

For example, it is assumed that the diagnostic log of the searched similar ultrasound image group includes a description of a disease related to the liver in a case in which the ultrasound probe5is moved to the imaging position indicated by the mark43illustrated inFIG.7to acquire an ultrasound image. In this case, the imaging position specification unit23specifies the liver at the imaging position. Therefore, as illustrated inFIG.8, a mark48indicating a new imaging position is superimposed on the notification screen40at the position of the liver in the schema37. In addition, an icon49indicating the current position of the ultrasound probe5is displayed inFIG.8. Therefore, it is possible to acquire the ultrasound image of the subject H at the position where the ultrasound image could not be captured in only the previous similar ultrasound image group and to check whether or not a disease occurs.

In addition, in a case in which the findings about the new ultrasound image group are input to the findings region46, the diagnostic log creation unit26updates a diagnostic log. For example, in a case in which the ultrasound probe5is moved to the imaging position indicated by the mark43illustrated inFIG.7to acquire a new ultrasound image group, the examiner inputs findings about the new ultrasound image group to the findings region46of the notification screen40. In a case in which findings based on the new ultrasound image group are input to the findings region46, the diagnostic log creation unit26adds the input findings to the diagnostic log to update the diagnostic log.

In a case in which the examiner inputs an instruction to end the examination, the communication unit27generates one diagnosed ultrasound image group having, as a sub-image group, each of a plurality of ultrasound image groups acquired at each of a plurality of imaging positions and transmits the one diagnosed ultrasound image group to the image storage server3together with the diagnostic log described up to the end of the examination. The image storage server3stores the transmitted diagnosed ultrasound image group and diagnostic log as a new diagnosed ultrasound image group.

Next, a process performed in this embodiment will be described.FIG.9is a flowchart illustrating the process performed in this embodiment. The process is started in response to the input of an examination start instruction, and the imaging unit21sequentially acquires ultrasound images (Step ST1) and stores an ultrasound image group including a plurality of ultrasound images acquired between a storage start instruction and a storage end instruction from the examiner in the storage13(Step ST2).

Then, the search unit22searches for a similar ultrasound image group that is similar to the features of the acquired ultrasound image group with reference to the image storage server3(Step ST3). Then, the imaging position specification unit23specifies the imaging position on the body surface of the subject where the disease predicted in the subject can be imaged, using the diagnostic log for the similar ultrasound image group (Step ST4). In a case in which the imaging position is specified (Step ST5; YES), the notification unit25notifies of the specified imaging position (Step ST6).

In a case in which the determination result in Step ST5is “NO” and following Step ST6, the diagnostic log creation unit26receives the input of findings by the examiner and describes the input findings in the diagnostic log to create the diagnostic log (Step ST7). Then, it is determined whether or not an end instruction is input (Step ST8). In a case in which the determination result in Step ST8is “NO”, the process returns to Step ST1. Then, the processes from Step ST1to Step ST7are repeated. In a case in which the determination result in Step ST8is “YES”, the communication unit27derives one diagnosed ultrasound image group having, as sub-image groups, the ultrasound image groups during diagnosis acquired at each of the specified imaging positions, and transmits the diagnosed ultrasound image group and the diagnostic log to the image storage server3(Step ST9). Then, the process is ended. Further, in a case in which the processes from Step ST1to Step ST7are repeated, the diagnostic log creation unit26updates the diagnostic log in the process in Step ST7.

As described above, in this embodiment, a similar ultrasound image group that is similar to the features of the ultrasound image group during diagnosis is searched for, and the imaging position on the body surface of the subject where the disease predicted in the subject can be imaged is specified using the positional information of the disease included in the diagnostic log for the similar ultrasound image group. Then, the specified imaging position is notified of. Therefore, the operator can move the ultrasound probe5to the notified of imaging position, acquire an ultrasound image group, and interpret the acquired ultrasound image group to check whether or not a disease occurs. Therefore, it is possible to acquire an ultrasound image without overlooking a disease, using the similar ultrasound image group. As a result, it is possible to improve the accuracy of ultrasound diagnosis.

In addition, the acquisition of a new ultrasound image group at the specified imaging position, the searching of a new similar ultrasound image group based on the new ultrasound image group, the specification of a new imaging position, the notification of the new imaging position, and the update of the diagnostic log based on findings about the new ultrasound image group can be repeatedly performed to acquire the ultrasound image of the subject at a position where the ultrasound image could not be captured in only the similar ultrasound image group and to check whether or not a disease occurs. Therefore, it is possible to examine the disease that is likely to occur, without omission.

In addition, the ultrasonography apparatus according to this embodiment may include an analysis unit28that analyzes an ultrasound image group to detect an abnormal shadow from the ultrasound image group, like another ultrasonography apparatus2A according to this embodiment illustrated inFIG.10. The analysis unit28analyzes an ultrasound image group to detect an abnormal shadow, such as a disease, included in the ultrasound image group. The analysis unit28detects shadows of a plurality of types of diseases as abnormal shadows from the ultrasound image group using a known computer-aided diagnosis (that is, CAD) algorithm.

As described above, since the ultrasonography apparatus2A according to this embodiment includes the analysis unit28, it is possible to display an analysis result on the notification screen40or to describe the analysis result in the diagnostic log. Therefore, it is possible to efficiently perform a diagnosis using the ultrasound image group.

In addition, in the above-described embodiments, for example, the following various processors can be used as a hardware structure of processing units performing various processes, such as the imaging unit21, the search unit22, the imaging position specification unit23, the probe position specification unit24, the notification unit25, the diagnostic log creation unit26, the communication unit27, and the analysis unit28. The various processors include, for example, a CPU which is a general-purpose processor executing software (programs) to function as various processing units as described above, a programmable logic device (PLD), such as a field programmable gate array (FPGA), which is a processor whose circuit configuration can be changed after manufacture, and a dedicated electric circuit, such as an application specific integrated circuit (ASIC), which is a processor having a dedicated circuit configuration designed to perform a specific process.

One processing unit may be configured by one of the various processors or by a combination of two or more processors of the same type or different types (for example, a combination of a plurality of FPGAs or a combination of a CPU and an FPGA). In addition, a plurality of processing units may be configured by one processor.

A first example of the configuration in which a plurality of processing units are configured by one processor is an aspect in which one processor is configured by a combination of one or more CPUs and software and functions as a plurality of processing units. A representative example of this aspect is a client computer or a server computer. A second example of the configuration is an aspect in which a processor that implements the functions of the entire system including a plurality of processing units using one integrated circuit (IC) chip is used. A representative example of this aspect is a system-on-chip (SoC). As described above, various processing units are configured by one or more of the various processors as a hardware structure.

In addition, specifically, an electric circuit (circuitry) obtained by combining circuit elements, such as semiconductor elements, can be used as the hardware structure of the various processors.