Patent Description:
An ultrasound diagnostic apparatus that captures an ultrasound image of a subject by using an ultrasound probe that receives an ultrasound echo of an ultrasonic wave transmitted to the subject and outputs a reception signal based on the received ultrasound echo is known.

A technique of determining whether or not a captured ultrasound image is suitable for a medical examination is known. For example, in <CIT>, a technique of acquiring a plurality of ultrasound images of a lung of a subject and determining whether or not the ultrasound image is suitable for a diagnosis is known.

With the technique disclosed in <CIT>, it is possible to determine whether or not the ultrasound image is suitable for a diagnosis of a lung, but it is not possible to determine whether or not the ultrasound image is suitable for a diagnosis of parts other than the lung. For example, with the technique disclosed in <CIT>, it is not possible to determine whether or not the ultrasound image is suitable for a diagnosis of a blood vessel in an arm. <CIT> discloses an ultrasonic diagnostic imaging system including a measuring unit that measures a relative position and a relative posture of the ultrasonic probe with respect to an examinee using image information on the examinee acquired by the ultrasonic probe, and a control amount calculation unit that calculates an amount of control of the position and posture of the ultrasonic probe based on the measurement result of the measuring unit and a guiding information presentation unit that presents information for guiding movement of the position and posture of the ultrasonic probe using the amount of control calculated by the control amount calculation unit.

<CIT> discloses an ultrasonic user interface device. <CIT> discloses a subject information acquisition device and a control method thereof. <NPL> relates to EchoFelx: Hand gesture recognition using ultrasound imaging. <NPL> relates to low-cost biometric recognition system based on NIR palm vein image.

The present disclosure has been made in view of the above circumstances, and an object of the present disclosure is to provide an ultrasonography apparatus according to claim <NUM>, an image processing apparatus according to claim <NUM>, an ultrasound image capturing method according to claim <NUM>, and an ultrasound image capturing program capable of obtaining an ultrasound image suitable for a diagnosis according to claim <NUM>.

In order to achieve the above-described object, a first aspect according to the present disclosure provides an ultrasonography apparatus comprising: at least one processor, in which the processor acquires, from a detector that detects a posture of a palm of a subject in capturing an ultrasound image, information representing a posture of the palm at a first time point as a first detection result, acquires information representing a posture of the palm at a second time point, which is after the first time point and is a time point during capturing of the ultrasound image, as a second detection result, derives a difference between the posture of the palm at the first time point and the posture of the palm at the second time point based on the first detection result and the second detection result, and performs notification in a case in which the derived difference is equal to or greater than a threshold value.

In an embodiment, the first time point is a start time point of capturing the ultrasound image.

In an embodiment, the processor adds information representing a warning to the ultrasound image captured at the second time point in a case in which the derived difference is equal to or greater than the threshold value.

A further aspect according to the present disclosure provides an image processing apparatus performing image processing on a plurality of ultrasound images captured by an ultrasonography apparatus that acquires, from a detector that detects a posture of a palm of a subject in capturing an ultrasound image, information representing a posture of the palm at a first time point as a first detection result, acquires information representing a posture of the palm at a second time point, which is after the first time point and is a time point during capturing of the ultrasound image, as a second detection result, derives a difference between the posture of the palm at the first time point and the posture of the palm at the second time point based on the first detection result and the second detection result, and adds information representing a warning to the ultrasound image captured at the second time point in a case in which the derived difference is equal to or greater than a threshold value, the image processing apparatus comprising: at least one processor, in which the processor acquires the plurality of ultrasound images captured by the ultrasonography apparatus, and generates an object-of-interest image from the plurality of ultrasound images.

In an embodiment, in a case in which the acquired plurality of the ultrasound images include the ultrasound image to which the information representing the warning is added, the processor generates the object-of-interest image from the plurality of ultrasound images excluding the ultrasound image to which the information representing the warning is added.

In an embodiment, the processor displays a region generated from the ultrasound image to which the information representing the warning is added and a region generated from an ultrasound image other than the ultrasound image to which the information representing the warning is added, of the generated object-of-interest image, in an identifiable manner.

In an embodiment, the processor generates the object-of-interest image by making a weighting of the ultrasound image to which the information representing the warning is added different from a weighting of an ultrasound image other than the ultrasound image to which the information representing the warning is added.

In addition, in order to achieve the above-described object, a further aspect according to the present disclosure provides an ultrasound image capturing method executed by a processor, the method comprising: acquiring, from a detector that detects a posture of a palm of a subject in capturing an ultrasound image, information representing a posture of the palm at a first time point as a first detection result; acquiring information representing a posture of the palm at a second time point, which is after the first time point and is a time point during capturing of the ultrasound image, as a second detection result; deriving a difference between the posture of the palm at the first time point and the posture of the palm at the second time point based on the first detection result and the second detection result; and performing notification in a case in which the derived difference is equal to or greater than a threshold value.

In addition, in order to achieve the above-described object, a further aspect according to the present disclosure provides an ultrasound image capturing program for causing a processor to execute a process comprising: acquiring, from a detector that detects a posture of a palm of a subject in capturing an ultrasound image, information representing a posture of the palm at a first time point as a first detection result; acquiring information representing a posture of the palm at a second time point, which is after the first time point and is a time point during capturing of the ultrasound image, as a second detection result; deriving a difference between the posture of the palm at the first time point and the posture of the palm at the second time point based on the first detection result and the second detection result; and performing notification in a case in which the derived difference is equal to or greater than a threshold value.

According to the present disclosure, it is possible to obtain an ultrasound image suitable for a diagnosis.

Hereinafter, exemplary embodiments will be described in detail with reference to the drawings. The present embodiment does not limit the present invention, as defined by the appended claims.

First, an example of an overall configuration of a medical image capturing system <NUM> according to the present embodiment will be described. <FIG> shows a block diagram showing an example of the overall configuration of the medical image capturing system <NUM> according to the present embodiment. As shown in <FIG>, the medical image capturing system <NUM> according to the present embodiment comprises an ultrasonography apparatus <NUM> and a detector <NUM>.

As shown in <FIG>, the ultrasonography apparatus <NUM> according to the present embodiment comprises an ultrasound probe <NUM> and a main body portion <NUM>.

The ultrasound probe <NUM> comprises a transducer array <NUM> and a transmission/reception circuit <NUM> including a transmission circuit <NUM> and a reception circuit <NUM>. The transducer array <NUM> comprises a plurality of transducers (not shown) arranged in a one-dimensional or two-dimensional manner. As an example, in the present embodiment, an aspect in which the ultrasound probe <NUM> is a linear-type ultrasound probe in which a plurality of transducers are linearly arranged will be described. The ultrasound probe <NUM> is not limited to this aspect, and may be a convex-type or sector-type ultrasound probe in which the transducers are arranged in a curved manner. Each of the plurality of transducers transmits an ultrasonic wave based on a drive signal applied from the transmission circuit <NUM>, receives an ultrasound echo generated in a subject, and outputs an electric signal in response to the received ultrasound echo.

Each of the plurality of transducer is configured by forming electrodes at both ends of a piezoelectric body which is a piezoelectric material, such as piezoelectric ceramic represented by lead zirconate titanate (PZT), a polymeric piezoelectric element represented by poly vinylidene di fluoride (PVDF), and piezoelectric single crystal represented by lead magnesium niobate-lead titanate (PMN-PT).

The transmission circuit <NUM> causes the transducer array <NUM> to transmit an ultrasound beam toward the subject. Specifically, the transmission circuit <NUM> includes, for example, a plurality of pulse generators (not shown), and, based on a transmission delay pattern selected in response to a control signal from an imaging controller <NUM> of the main body portion <NUM>, each delay amount is adjusted to supply the drive signal and apply a voltage to each of the plurality of transducers included in the transducer array <NUM>. Each drive signal is a pulse-like or continuous wave-like voltage signal, and in a case in which a voltage is applied to the electrodes of the transducers of the transducer array <NUM>, the piezoelectric body expands and contracts. As a result of the expansion and contraction, pulsed or continuous ultrasonic waves are generated from each transducer, and an ultrasound beam is formed from a combined wave of these ultrasonic waves.

The transmitted ultrasound beam is reflected by each part (for example, a blood vessel or other tissue) in the subject, an instrument disposed in the subject, or the like, thereby generating an ultrasound echo. The generated ultrasound echo propagates in the subject and is received by the plurality of transducers included in the transducer array <NUM>. Each transducer generates an electric signal corresponding to the received ultrasound echo. The electric signal generated in each transducer is output to the reception circuit <NUM>.

The reception circuit <NUM> generates a sound ray signal by performing processing on a signal (strictly speaking, an analog electric signal) output from the transducer array <NUM> in accordance with the control signal from the imaging controller <NUM> of the main body portion <NUM>. <FIG> is a block diagram showing an example of a configuration of the reception circuit <NUM> according to the present embodiment. As shown in <FIG>, the reception circuit <NUM> includes, for example, an amplification unit <NUM>, an analog digital (AD) conversion unit <NUM>, and a beam former <NUM>.

The amplification unit <NUM> amplifies the electric signal output from each of the plurality of transducers included in the transducer array <NUM>, and outputs the amplified electric signal to the AD conversion unit <NUM>. The AD conversion unit <NUM> converts the amplified electric signal into digital reception data, and outputs each piece of the converted reception data to the beam former <NUM>. The beam former <NUM> performs reception focus processing by giving and adding delay with respect to each piece of the reception data converted by the AD conversion unit <NUM>, in accordance with a sound velocity or a sound velocity distribution set based on a reception delay pattern selected in response to the control signal from the imaging controller <NUM> of the main body portion <NUM>. Through the reception focusing processing, a sound ray signal in which each piece of the reception data converted by the AD conversion unit <NUM> is phased and added and the focus of the ultrasound echo is narrowed is generated. The generated sound ray signal is output to the image generation unit <NUM> via a communication interface (I/F) unit <NUM> of the main body portion <NUM>.

On the other hand, the main body portion <NUM> comprises a processor <NUM>, a memory <NUM>, a storage unit <NUM>, the communication I/F unit <NUM>, an input I/F unit <NUM>, a display unit <NUM>, and the image generation unit <NUM>. The processor <NUM>, the memory <NUM>, the storage unit <NUM>, the communication I/F unit <NUM>, the input I/F unit <NUM>, the display unit <NUM>, and the image generation unit <NUM> are connected to each other via a bus <NUM> such as a system bus or a control bus such that various kinds of information can be exchanged.

The processor <NUM> reads out various programs, which include an imaging control program <NUM> stored in the storage unit <NUM>, to the memory <NUM> and executes processing according to the read-out program. Accordingly, the processor <NUM> controls capturing of an ultrasound image, and image processing on the ultrasound image. The memory <NUM> is a work memory that is used for the processor <NUM> to execute processing.

The storage unit <NUM> stores image data of the ultrasound image generated by the image generation unit <NUM>, posture information P acquired from the detector <NUM>, the imaging control program <NUM>, marker feature information <NUM> to be described in detail below, and various other kinds of information. Specific examples of the storage unit <NUM> include a hard disk drive (HDD), a solid state drive (SSD), and a secure digital (SD) card.

The communication I/F unit <NUM> performs communication of various kinds of information with an external device of the ultrasound probe <NUM>, the detector <NUM>, and the main body portion <NUM> through wireless communication such as WiFi (registered trademark) or Bluetooth (registered trademark) or wired communication. A control signal for capturing the ultrasound image is output from the main body portion <NUM> to the ultrasound probe <NUM> via the communication I/F unit <NUM>. In addition, a sound ray signal is input from the ultrasound probe <NUM> to the main body portion <NUM> via the communication I/F unit <NUM>. In addition, the posture information P is input from the detector <NUM> to the main body portion <NUM> via the communication I/F unit <NUM>.

The input I/F unit <NUM> and the display unit <NUM> function as a user interface. The display unit <NUM> provides a user with various kinds of information regarding the capturing of the ultrasound image. The display unit <NUM> is not particularly limited, and examples of the display unit <NUM> include a liquid crystal monitor and a light emitting diode (LED) monitor. In addition, the input I/F unit <NUM> is operated by the user in order to input various instructions regarding the capturing of the ultrasound image or the like. The input I/F unit <NUM> is not particularly limited, and examples of the input I/F unit <NUM> include a keyboard, a touch pen, and a mouse. A touch panel display in which the input I/F unit <NUM> and the display unit <NUM> are integrated may be adopted.

The image generation unit <NUM> has a function of generating the ultrasound image based on the sound ray signal input from the reception circuit <NUM> of the ultrasound probe <NUM>. <FIG> shows a block diagram showing an example of a configuration of the image generation unit <NUM> according to the present embodiment. As shown in <FIG>, the image generation unit <NUM> includes, for example, a signal processing unit <NUM>, a digital scan converter (DSC) <NUM>, and an image processing unit <NUM>. The signal processing unit <NUM> generates a B-mode image signal representing an ultrasound image U by performing, on the sound ray signal generated by the reception circuit <NUM>, attenuation correction due to a distance according to a depth of a reflection position of the ultrasonic wave and then performing envelope detection processing. The DSC <NUM> converts the B-mode image signal generated by the signal processing unit <NUM> into an image signal according to a normal television signal scanning method by raster conversion or the like. The image processing unit <NUM> performs required various image processing such as gradation processing on the B-mode image signal input from the DSC <NUM>, and then outputs the B-mode image signal. The B-mode image signal output from the image generation unit <NUM> corresponds to the ultrasound image U.

<FIG> shows an example of the ultrasound image U generated by the image generation unit <NUM>. The ultrasound image U shown in <FIG> shows a cross section of a blood vessel B. Here, the cross section of the blood vessel B means a cut surface orthogonal to an extension direction of the blood vessel B. In the present embodiment, as shown in <FIG>, in the ultrasound image U, each portion of the blood vessel B in the ultrasound image U, in which a direction connecting a body surface S and an inside of the subject is called a depth direction D, is displayed in the depth direction D at a position corresponding to a distance from the body surface S of the subject with which the ultrasound probe <NUM> is in contact, that is, a depth.

On the other hand, the detector <NUM> is a detector that detects a posture of a palm of the subject. In the present embodiment, as shown in <FIG>, as the detector <NUM>, an optical camera in which a region including a palm W1 of a subject W is set as an imaging range R1 is used as a sensor for detecting the posture information P. Therefore, an optical camera image captured by the optical camera of the detector <NUM> is an image corresponding to the imaging range R1. An imaging range of the ultrasound image U in the present embodiment is an imaging range R2 shown in <FIG>, and the ultrasound image U is captured with an arm W2 of the subject W as an imaging part.

In a case in which a detection start instruction is input from the main body portion <NUM> of the ultrasonography apparatus <NUM>, the optical camera of the detector <NUM> according to the present embodiment continuously captures optical camera images at a set frame rate until a detection end instruction is input from the main body portion <NUM>. The detector <NUM> estimates the posture information P of the palm W1 using the optical camera image of each frame as a sensor value, and sequentially outputs the estimated posture information P to the ultrasonography apparatus <NUM>.

Next, a functional configuration of the main body portion <NUM> of the ultrasonography apparatus <NUM> will be described. <FIG> shows a functional block diagram showing an example of a configuration related to a function of the main body portion <NUM> of the ultrasonography apparatus <NUM> according to the present embodiment. As shown in <FIG>, the main body portion <NUM> comprises a first acquisition unit <NUM>, a difference derivation unit <NUM>, a second acquisition unit <NUM>, a display controller <NUM>, and an imaging controller <NUM>. For example, in the main body portion <NUM> according to the present embodiment, the processor <NUM> executes the imaging control program <NUM> stored in the storage unit <NUM>, so that the processor <NUM> functions as the first acquisition unit <NUM>, the difference derivation unit <NUM>, the second acquisition unit <NUM>, the display controller <NUM>, and the imaging controller <NUM>.

The imaging controller <NUM> has a function of outputting the control signal to the transmission/reception circuit <NUM> of the ultrasound probe <NUM> as described above in a case of capturing the ultrasound image U. In a case in which the control signal output from the imaging controller <NUM> is input to the transmission circuit <NUM> and the reception circuit <NUM> of the ultrasound probe <NUM>, the sound ray signal is output from the reception circuit <NUM> of the ultrasound probe <NUM> to the image generation unit <NUM> of the main body portion <NUM> as described above. Under the control of the imaging controller <NUM>, the transmission/reception circuit <NUM> of the ultrasound probe <NUM> and the image generation unit <NUM> of the main body portion <NUM> continuously acquire the ultrasound image a plurality of times at a fixed frame rate during a capturing period of the ultrasound image.

The first acquisition unit <NUM> has a function of acquiring the posture information P. As an example, the first acquisition unit <NUM> according to the present embodiment acquires the posture information P from the detector <NUM> as described above. As will be described below, the posture information P includes initial posture information P0 imaged by the detector <NUM> at the start of the imaging and imaging posture information P1 imaged by the detector <NUM> after the start of the imaging and during the imaging.

The difference derivation unit <NUM> derives a difference between the initial posture information P0 and the imaging posture information P1. In addition, the difference derivation unit <NUM> determines whether or not the derived difference is equal to or greater than a threshold value. A method of deriving, via the difference derivation unit <NUM>, a difference between a posture of the palm W1 in the initial posture information P0 and a posture of the palm W1 in the imaging posture information P1, and a method of determining whether or not the difference is equal to or greater than a threshold value are not limited. For example, the difference derivation unit <NUM> may detect the difference from an outline of the palm W1 in each of the initial posture information P0 and the imaging posture information P1. In this case, the difference derivation unit <NUM> determines whether or not an amount of change between the outline in the initial posture information P0 and the outline in the imaging posture information P1 is equal to or greater than a threshold value. Examples of the threshold value in this case include a threshold value based on an amount of movement. In addition, specific examples of the threshold value include at least one of a case in which the outline is changed by <NUM> pixels or more or a case in which positions of three or more fingers are changed regardless of the amount of movement. The difference derivation unit <NUM> outputs a determination result to the display controller <NUM>.

The second acquisition unit <NUM> has a function of acquiring the ultrasound image U. As an example, the second acquisition unit <NUM> according to the present embodiment acquires the ultrasound image U from the storage unit <NUM>. The ultrasound image U acquired by the second acquisition unit <NUM> is output to the display controller <NUM>.

The display controller <NUM> has a function of displaying the ultrasound image U on the display unit <NUM> of the main body portion <NUM>. In addition, the display controller <NUM> has a function of displaying information representing a warning together with the ultrasound image U in a case in which the determination result input from the difference derivation unit <NUM> represents that the difference is equal to or greater than the threshold value.

Next, an operation of the main body portion <NUM> according to the present embodiment will be described with reference to the drawings. <FIG> shows a flowchart showing an example of a flow of the imaging control processing executed in the main body portion <NUM> according to the present embodiment. As an example, in the main body portion <NUM> according to the present embodiment, in a case in which power is supplied to the main body portion <NUM>, the processor <NUM> executes the imaging control program <NUM> stored in the storage unit <NUM> to execute the image processing shown in <FIG> as an example.

In step S100 of <FIG>, the imaging controller <NUM> determines whether or not to start the capturing of the ultrasound image U. For example, in a case in which an imaging start instruction input by a technician from the input I/F unit <NUM> is received, the imaging controller <NUM> determines to start the capturing of the ultrasound image U. In a case in which the capturing of the ultrasound image U is not started, a determination result in step S100 is NO. On the other hand, in a case in which the capturing of the ultrasound image U is started, a determination result in step S100 is YES, and the process proceeds to step S102.

In step S102, the first acquisition unit <NUM> outputs a start instruction for starting acquisition of the posture information P to the detector <NUM>. As described above, in a case in which the start instruction is input, the detector <NUM> causes the optical camera to start the imaging of the palm W1 of the subject W at a set frame rate. In addition, the detector <NUM> estimates the posture information P from the optical camera image of each frame, and outputs the posture information P to the main body portion <NUM>.

In next step S104, the first acquisition unit <NUM> acquires the initial posture information P0. That is, the posture information P at the start of the capturing of the ultrasound image U is acquired. The main body portion <NUM> may control the ultrasound probe <NUM> and the detector <NUM> such that transmission of an ultrasonic wave or reception of an ultrasound echo corresponding to the first ultrasound image U by the ultrasound probe <NUM> is synchronized with the imaging of the initial posture information P0 by the optical camera of the detector <NUM>.

In next step S106, the imaging controller <NUM> outputs the imaging start instruction of the ultrasound image to the ultrasound probe <NUM>. The ultrasound probe <NUM> outputs an ultrasound beam to the subject W in response to the imaging start instruction, and outputs the obtained sound ray signal to the main body portion <NUM>. As described above, the image generation unit <NUM> of the main body portion <NUM> generates the ultrasound image U from the sound ray signal.

In next step S108, the second acquisition unit <NUM> starts acquisition of the ultrasound image U, and the display controller <NUM> starts display of the ultrasound image U. After that, until the imaging is ended, the ultrasound image U is captured at a predetermined frame rate, and the generated ultrasound images U are sequentially displayed on the display unit <NUM>.

In next step S110, the second acquisition unit <NUM> acquires the imaging posture information P1 from the detector <NUM>. That is, the second acquisition unit <NUM> acquires, as the imaging posture information P1, the posture information P estimated from the optical camera image captured by the optical camera of the detector <NUM> during the capturing of the ultrasound image U, from the detector <NUM>.

In next step S112, as described above, the difference derivation unit <NUM> derives the difference between the initial posture information P0 and the imaging posture information P1.

In next step S114, as described above, the difference derivation unit <NUM> determines whether or not the difference derived by the difference derivation unit <NUM> is equal to or greater than the threshold value. In a case in which the difference is not equal to or greater than the threshold value, in other words, in a case in which the difference is smaller than the threshold value, a determination result in step S114 is NO, and the process proceeds to step S120.

On the other hand, in a case in which the difference is equal to or greater than the threshold value, a determination result in step S114 is YES, and the process proceeds to step S116. In step S116, as described above, the display controller <NUM> causes the display unit <NUM> to display the information representing the warning.

In next step S118, the display controller <NUM> adds the information representing the warning to the ultrasound image U and stores the resultant image. Specifically, the information representing the warning is added to the ultrasound image U captured at the same timing as a timing at which the imaging posture information P1 used for deriving the difference that is a source of the warning is imaged.

In next step S <NUM>, the imaging controller <NUM> determines whether or not to end the imaging. For example, in a case in which an imaging end instruction input by the technician from the input I/F unit <NUM> is received, the imaging controller <NUM> determines to end the capturing of the ultrasound image U. In a case in which the imaging is not to be ended, a determination result in step S120 is NO, the process returns to step S110, and the processes of steps S110 to S <NUM> are repeated. On the other hand, in a case in which the imaging is to be ended, a determination result in step S <NUM> is YES, and, in step S <NUM>, the imaging controller <NUM> outputs the end instruction to the detector <NUM> so that the first acquisition unit <NUM> ends the acquisition of the posture information P. As described above, in a case in which the end instruction is input, the detector <NUM> ends the imaging of the palm W1 of the subject W by the optical camera and the estimation of the posture information P from the optical camera image. In a case in which the process in step S <NUM> is ended, the imaging control processing shown in <FIG> is ended.

As described above, according to the main body portion <NUM> according to the present embodiment, in a case in which the posture of the palm W1 of the subject W changes by a threshold value or more from the start of the capturing of the ultrasound image U during the capturing of the ultrasound image U, a warning is displayed. In a case in which the posture of the palm W1 changes by a threshold value or more, a position of a blood vessel in the arm W2 of the subject W may change even though the posture of the arm W2 does not change. In a case in which the position of the blood vessel in the arm W2 changes, a state of the blood vessel B reflected in the ultrasound image U changes. The ultrasound image U in which the state of the blood vessel B has changed in this way may not be preferable for use in diagnosis. For example, in a case in which a series of the ultrasound images U are connected to generate a blood vessel structure image showing a structure of the blood vessel B, the change in the image of the blood vessel B changed from the imaging start may result in noise.

With respect to this, in the present embodiment, as described above, in a case in which the posture of the palm W1 of the subject W changes by the threshold value or more, a warning is displayed, so that the technician who notices the warning tries to return the posture of the palm W1 to the initial state. Accordingly, a state of a blood vessel of the arm W2 of the subject W can be returned to the same state as before the change, that is, at the start of the capturing of the ultrasound image U. Therefore, according to the main body portion <NUM> according to the present embodiment, it is possible to obtain an ultrasound image suitable for diagnosis.

Next, generation of the blood vessel structure image using a series of the ultrasound images U captured by the ultrasonography apparatus <NUM> will be described. As an example, in the present embodiment, after a series of the ultrasound images U are captured by the ultrasonography apparatus <NUM>, the blood vessel structure image is generated by an image processing apparatus <NUM> provided outside the ultrasonography apparatus <NUM>.

<FIG> shows a configuration diagram showing a hardware configuration of an example of the image processing apparatus <NUM> according to the present embodiment. As shown in <FIG>, the image processing apparatus <NUM> according to the present embodiment comprises a processor <NUM>, a memory <NUM>, a storage unit <NUM>, a communication I/F unit <NUM>, an input I/F unit <NUM>, and a display unit <NUM>. The processor <NUM>, the memory <NUM>, the storage unit <NUM>, the communication I/F unit <NUM>, the input I/F unit <NUM>, and the display unit <NUM> are connected to each other via a bus <NUM> such as a system bus or a control bus such that various kinds of information can be exchanged.

The processor <NUM> reads out various programs, which include an image generation program <NUM> stored in the storage unit <NUM>, to the memory <NUM> and executes processing according to the read-out program. Accordingly, the processor <NUM> controls the generation of the blood vessel structure image (object-of-interest image). The memory <NUM> is a work memory that is used for the processor <NUM> to execute processing.

The storage unit <NUM> stores the acquired ultrasound image, the image generation program <NUM>, various other kinds of information, and the like. Specific examples of the storage unit <NUM> include an HDD, an SSD, and an SD card.

The communication I/F unit <NUM> performs communication of various kinds of information with the main body portion <NUM> of the ultrasonography apparatus <NUM> through wireless communication such as WiFi (registered trademark) or Bluetooth (registered trademark) or wired communication.

The input I/F unit <NUM> and the display unit <NUM> function as a user interface. The display unit <NUM> provides the user with various kinds of information regarding the generation of the blood vessel image (object-of-interest image). The display unit <NUM> is not particularly limited, and examples thereof include a liquid crystal monitor and an LED monitor. In addition, the input I/F unit <NUM> is operated by the user in order to input various instructions regarding the generation of the blood vessel image (object-of-interest image) and the like. The input I/F unit <NUM> is not particularly limited, and examples of the input I/F unit <NUM> include a keyboard, a touch pen, and a mouse. A touch panel display in which the input I/F unit <NUM> and the display unit <NUM> are integrated may be adopted.

Next, a functional configuration of the image processing apparatus <NUM> will be described. <FIG> is a functional block diagram showing an example of a configuration related to a function of the image processing apparatus <NUM> according to the present embodiment. As shown in <FIG>, the image processing apparatus <NUM> comprises an acquisition unit <NUM>, an image generation unit <NUM>, and a display controller <NUM>. For example, in the image processing apparatus <NUM> according to the present embodiment, the processor <NUM> executes the image generation program <NUM> stored in the storage unit <NUM>, so that the processor <NUM> functions as the acquisition unit <NUM>, the image generation unit <NUM>, and the display controller <NUM>.

The acquisition unit <NUM> has a function of acquiring an ultrasound image group from the main body portion <NUM> of the ultrasonography apparatus <NUM>. In a case in which a warning is issued by the imaging control processing of the main body portion <NUM>, the ultrasound images U in a state where information representing the warning is added to the corresponding ultrasound image U are acquired. That is, in a case in which a warning is issued by the imaging control processing of the main body portion <NUM>, the acquisition unit <NUM> acquires an ultrasound image group including the ultrasound image U to which the warning is added and the ultrasound image U to which the warning is not added. The acquisition unit <NUM> outputs the acquired ultrasound image group to the image generation unit <NUM>.

The image generation unit <NUM> generates an object-of-interest image representing an object of interest from each ultrasound image U of the ultrasound image group. In the present embodiment, since a blood vessel is an object of interest, the image generation unit <NUM> generates a blood vessel structure image showing a structure of the blood vessel B. The blood vessel structure image (object-of-interest image) generated by the image generation unit <NUM> from the ultrasound image group may be a two-dimensional image configured of pixel data, may be a three-dimensional image configured of voxel data, or may be both the two-dimensional image and the three-dimensional image. A method of generating, via the image generation unit <NUM>, the blood vessel structure image (object-of-interest image) from the ultrasound image group is not particularly limited. A known method can be applied to both a case in which the blood vessel structure image (object-of-interest image) is a two-dimensional image and a case in which the blood vessel structure image (object-of-interest image) is a three-dimensional image. The blood vessel structure image generated by the image generation unit <NUM> is output to the display controller <NUM>.

The display controller <NUM> has a function of causing the display unit <NUM> to display the blood vessel structure image.

Next, an action of the image processing apparatus <NUM> according to the present embodiment will be described with reference to the drawings. <FIG> shows a flowchart showing an example of a flow of image generation processing executed in the image processing apparatus <NUM> according to the present embodiment. As an example, in the main body portion <NUM> according to the present embodiment, in a case in which the user gives a start instruction for the image generation by the input I/F unit <NUM>, the processor <NUM> executes the image generation program <NUM> stored in the storage unit <NUM> to execute the image generation processing shown in <FIG> as an example.

In step S200 of <FIG>, as described above, the acquisition unit <NUM> acquires the ultrasound image group including a plurality of the ultrasound images U from the main body portion <NUM> of the ultrasonography apparatus <NUM>.

In next step S202, the image generation unit <NUM> determines whether or not the acquired ultrasound image group includes the ultrasound image U to which the warning is added. In a case in which the ultrasound image U to which the warning is added is not included, a determination result in step S202 is NO, and the process proceeds to step S206. On the other hand, in a case in which the ultrasound image U to which the warning is added is included, a determination result in step S202 is YES, and the process proceeds to step S204.

In step S204, the image generation unit <NUM> excludes the ultrasound image U to which the warning is added, from the ultrasound image group acquired in step S200 described above.

In next step S206, the image generation unit <NUM> generates the blood vessel structure image from the ultrasound image group as described above. Since, in a case in which the ultrasound image group acquired in step S200 described above includes the ultrasound image U to which the warning is added, the ultrasound image U to which the warning is added is excluded in step S204 described above, the image generation unit <NUM> generates the blood vessel structure image from the ultrasound image U to which the warning is not added. In this case, for example, the image generation unit <NUM> may generate a blood vessel structure image in which a portion corresponding to the ultrasound image U to which the warning is added is omitted. In addition, for example, the image generation unit <NUM> may generate a portion corresponding to the ultrasound image U to which the warning is added, that is, a pseudo ultrasound image corresponding to the excluded ultrasound image U by performing complementation from the ultrasound images U before and after the exclusion, and generate the blood vessel structure image also using the generated pseudo ultrasound image.

In next step S208, the display controller <NUM> causes the display unit <NUM> to display the blood vessel structure image generated in step S206 described above. In a case in which the process in step S208 is ended, the image generation processing shown in <FIG> is ended.

As described above, according to the image processing shown in <FIG>, the image generation unit <NUM> generates the blood vessel structure image without using the ultrasound image U to which the warning is added. Accordingly, the accuracy of the blood vessel structure image can be improved.

Further, a modification example of the image generation processing will be described.

<FIG> shows a flowchart showing an example of a flow of image generation processing executed in the image processing apparatus <NUM> according to the present modification example. The image generation processing according to the present modification example shown in <FIG> is different from the image generation processing shown in <FIG> in that the processes of steps S202 and S204 are not provided and after acquiring the ultrasound image group in step S200, the process proceeds to step S206. Therefore, the image generation unit <NUM> according to the present modification example generates the blood vessel structure image also using the ultrasound image U to which the warning is added. The blood vessel structure image generated in the present modification example is preferably a three-dimensional image.

In addition, the image generation processing according to the present modification example is different from the image generation processing shown in <FIG> in that the processes of steps S207A and S207B are included after the process of step S206.

In step S207A in <FIG>, the image generation unit <NUM> determines whether or not the ultrasound image group acquired in step S200 described above includes the ultrasound image U to which the warning is added. In a case in which the ultrasound image U to which the warning is added is not included in the ultrasound image group, a determination result in step S207A is NO, and the process proceeds to step S208 described above. On the other hand, in a case in which the ultrasound image U to which the warning is added is included in the ultrasound image group, a determination result in step S207A is YES, and the process proceeds to step S207B.

In step S207B, the display controller <NUM> causes the display unit <NUM> to display the blood vessel structure image in which a region generated by the ultrasound image U to which the warning is added and a region generated by the other ultrasound image U, that is, the ultrasound image U to which the warning is not added are made to be identifiable from each other. For example, the display controller <NUM> displays the blood vessel structure image such that the region generated by the ultrasound image U to which the warning is added is more conspicuous than the region generated by the other ultrasound image U. In addition, for example, the display controller <NUM> displays the blood vessel structure image by making a color of the region generated by the ultrasound image U to which the warning is added different from a color of the region generated by the other ultrasound image U. In a case in which the process in step S207B is ended, the image processing shown in <FIG> is ended.

In this way, by displaying the blood vessel structure image in which the region generated by the ultrasound image U to which the warning is added and the region generated by the other ultrasound image U are made to be identifiable from each other, the user can visually recognize a region where the accuracy of generation may have decreased.

The blood vessel structure image generated in the present modification example is preferably a three-dimensional image as in Modification Example <NUM>.

<FIG> shows a flowchart showing an example of a flow of image generation processing executed in the image processing apparatus <NUM> according to the present modification example. The image generation processing according to the present modification example shown in <FIG> is different from the image generation processing shown in <FIG> in that the process of step S204 is not provided and in a case in which a determination result in step S202 is NO, that is, in a case in which the ultrasound image U to which the warning is added is not included in the ultrasound image group, the process proceeds to step S206.

In addition, the image generation processing according to the present modification example is different from the image generation processing shown in <FIG> in that in a case in which a determination result in step S202 is YES, the process proceeds to step S205.

In step S205 of <FIG>, the image generation unit <NUM> generates the blood vessel structure image by making a weighting of the ultrasound image U to which the warning is added relatively lower than that of the ultrasound image U to which the warning is not added. For example, the image generation unit <NUM> may make the weighting of the ultrasound image U to which the warning is added relatively lower than that of the ultrasound image U to which the warning is not added, at a boundary portion between the ultrasound image U to which the warning is added and the ultrasound image U to which the warning is not added. In addition, for example, the image generation unit <NUM> may make the weighting of the ultrasound image U to which the warning is added relatively lower than that of the ultrasound image U to which the warning is not added, in the whole ultrasound image U to which the warning is added. In a case in which the process in step S205 is ended, the process proceeds to step S208.

In this way, in a case in which the blood vessel structure image is generated by making the weighting of the ultrasound image U to which the warning is added lower than that of the ultrasound image U to which the warning is not added, it is possible to suppress the decrease in accuracy of the blood vessel structure image by generating the blood vessel structure image using the ultrasound image U to which the warning is added.

As described above, according to the image processing apparatus <NUM> according to the present embodiment, it is possible to obtain a region-of-interest image (blood vessel structure image) suitable for a diagnosis.

The technique of the present disclosure is not limited to each of the above-described embodiments, and various modifications can be made.

For example, the detector <NUM> that detects the posture information P is not limited to the above-described form. For example, the detector <NUM> may be a detector <NUM> that estimates the posture information P from a sensor value output from a sensor as with the detector <NUM> that estimates the posture information P from the optical camera image captured by the optical camera described above. In addition, for example, the detector <NUM> may be a detector that uses the sensor value itself output from the sensor as the posture information P. That is, whether or not the detector <NUM> needs to estimate the posture information P is different depending on the used sensor.

Specifically, the detector <NUM> may be, for example, a combination of a tracking marker worn on a finger of the subject W and an optical camera. In addition, the detector <NUM> may be capable of detecting any one of a finger spacing, a finger angle, or a three-dimensional structure of a skeleton of the finger of the subject W. For example, the detector <NUM> may be a ring type sensor that is worn on the finger of the subject W or an electrostatic sensor that is worn on the palm W1 and that detects a change in static electricity. In addition, for example, the detector <NUM> may be a magnetic sensor that detects a change in magnetism. In addition, for example, the detector <NUM> may be a pressure sensor that detects a contact state of the finger or the palm W1. In addition, for example, the detector <NUM> may be a conductive glove worn on the palm W1 of the subject W.

In addition, for example, the detector <NUM> may be a distance measurement device using a laser, visible light, or the like. As the distance measurement device, light detection and ranging (LiDAR) that is a distance-measuring sensor which transmits a detection wave (laser) to the palm W1 of the subject W and receives a reflected wave from the palm W1 to measure a detection distance to the palm W1 may be used. In addition, as the distance measurement device, a camera that comprises a depth sensor and obtains an image configured of pixels having distance information to the palm W1, or a depth camera that comprises two optical cameras and can measure a distance from the parallax to the palm W1 may be used. In addition, as the distance measurement device, a communication device using WiFi (registered trademark), which is a general-purpose communication device having a wave transmitting part with a center frequency of <NUM>/<NUM> and a wave receiving part with a plurality of receiving channels such as a bandwidth of <NUM>/<NUM>/<NUM>/<NUM>, may be used. In addition, as the distance measurement device, a radar device using a frequency-modulated continuous wave and having a wave transmitting part that transmits a wideband (<NUM>) signal and a wave receiving part comprising a T-type antenna array may be used. In a case in which the distance measurement device uses LiDAR, WiFi, or a radar, the posture information P can be restored by analyzing the reflected wave with a deep learning model. In addition, in a case in which the distance measurement device uses a LiDAR comprising a camera or a depth camera, a position of a joint of the subject W can be grasped by image processing, and a distance from the camera at the corresponding pixel can be measured, so that the posture information P can be obtained.

In the above description, an aspect in which, in the imaging control processing for capturing the ultrasound image U, the main body portion <NUM> of the ultrasonography apparatus <NUM> performs a control of synchronizing the timing at which the capturing of the ultrasound image U is started with the timing at which the posture information P is acquired by the detector <NUM> has been described. However, the method of synchronizing both timings is not limited to the present aspect. For example, an aspect may be used in which each of the main body portion <NUM> or the ultrasound probe <NUM> and the detector <NUM> comprises a timepiece with the same time point, and starts the capturing of the ultrasound image U and starts the detection of the posture information P at a timing when each timepiece indicates a set time point.

In addition, for example, in capturing the ultrasound image U of the subject W, pre-capturing of the ultrasound image U may be performed while changing the posture of the palm W1, and, in a case in which the image of the blood vessel B in the ultrasound image U is in a preferable state, for example, in a case in which the image has a preferable shape, the subject W may be instructed, to maintain the posture of the palm W1 in that state.

In addition, in the above-described aspect, for example, as a hardware structure of a processing unit that executes various types of processing such as the first acquisition unit <NUM>, the difference derivation unit <NUM>, the second acquisition unit <NUM>, the display controller <NUM>, and the imaging controller <NUM>, or such as the acquisition unit <NUM>, the image generation unit <NUM>, and the display controller <NUM>, various processors shown below can be used. As described above, the various processors include, in addition to a central processing unit (CPU) which is a general-purpose processor that executes software (program) to function as various processing units, a programmable logic device (PLD) which is a processor whose circuit configuration can be changed after manufacturing such as a field programmable gate array (FPGA), and a dedicated circuitry which is a processor having a circuit configuration specifically designed to execute specific processing such as an application specific integrated circuit (ASIC).

One processing unit may be configured of one of the various processors, or configured of a combination of the same or different kinds of two or more processors (for example, a combination of a plurality of FPGAs or a combination of the CPU and the FPGA). In addition, a plurality of processing units may be configured of one processor.

As an example of configuring a plurality of processing units with one processor, first, there is a form in which, as typified by computers such as a client and a server, one processor is configured by combining one or more CPUs and software, and the processor functions as a plurality of processing units. Second, there is a form in which, as typified by a system on chip (SoC) and the like, in which a processor that implements functions of an entire system including a plurality of processing units with one integrated circuit (IC) chip is used. As described above, the various processing units are configured by using one or more of the various processors described above as a hardware structure.

Further, as the hardware structure of these various processors, more specifically, an electric circuit (circuitry) in which circuit elements such as semiconductor elements are combined can be used.

Claim 1:
An ultrasonography apparatus comprising:
at least one processor (<NUM>),
wherein the processor is configured to:
acquire an ultrasound image of an arm (W2) of a subject (W) captured by an ultrasound probe (<NUM>),
acquire, from a detector (<NUM>) configured to detect a posture of a palm of the subject:
information representing a posture of the palm at a first time point as a first detection result, and
information representing a posture of the palm at a second time point, which is after the first time point and is a time point during capturing of the ultrasound image, as a second detection result,
derive a difference between the posture of the palm at the first time point and the posture of the palm at the second time point based on the first detection result and the second detection result, and
perform notification in a case in which the derived difference is equal to or greater than a threshold value,
characterised in that
the processor is configured to add information representing a warning to the ultrasound image captured at the second time point in a case in which the derived difference is equal to or greater than the threshold value.