Patent Description:
PTL <NUM> to PTL <NUM> disclose a technique of generating a three-dimensional image of a cardiac cavity or a blood vessel using a US image system. The term "US" is an abbreviation of ultrasound.

Treatment using IVUS is widely performed on a cardiac cavity, a cardiac blood vessel, a lower limb artery region, and the like. The term "IVUS" is an abbreviation of intravascular ultrasound. The IVUS is a device or a method for providing a two-dimensional image of a plane perpendicular to a long axis of a catheter.

At present, an operator needs to perform treatment while reconstructing a three-dimensional structure by stacking two-dimensional images of IVUS in his/her brain, which is a barrier particularly to young doctors or inexperienced doctors. In order to remove such a barrier, it is conceivable to automatically generate a three-dimensional image representing a structure of biological tissue such as a cardiac cavity or a blood vessel from the two-dimensional images of IVUS and display the generated three-dimensional image toward the operator.

When, however, the operator can see only an outer wall of the biological tissue in the three-dimensional image, the operator cannot perform treatment inside the biological tissue.

An object of the present disclosure is to enable a user to see an inside of a biological tissue with the three-dimensional image.

The above object is achieved by a diagnostic assistance device according to independent claim <NUM>. The dependent claims relate to advantageous embodiments.

According to the present disclosure, a user can see an inside of a biological tissue with a three-dimensional image.

In the drawings, the same or corresponding parts are denoted by the same reference numerals. In the description of the present embodiment, the description of the same or corresponding parts will be omitted or simplified as appropriate.

An outline of the present embodiment will be described with reference to <FIG>, <FIG>, and <FIG>.

A diagnostic assistance device <NUM> according to the present embodiment generates three-dimensional data <NUM> of a biological tissue <NUM> based on tomographic data <NUM> of the biological tissue <NUM>. The diagnostic assistance device <NUM> displays the generated three-dimensional data <NUM> as a three-dimensional image <NUM> on a display <NUM>. The diagnostic assistance device <NUM> forms, in the three-dimensional data <NUM>, an opening <NUM> that exposes an inner wall surface <NUM> of the biological tissue <NUM> to an outside of the biological tissue <NUM> in the three-dimensional image <NUM>. The diagnostic assistance device <NUM> adjusts a viewpoint when displaying the three-dimensional image <NUM> on the display <NUM> according to a position of the formed opening <NUM>. The term "viewpoint" refers to a position of a virtual camera <NUM> disposed in a three-dimensional space.

According to the present embodiment, a user can see an inside of the biological tissue <NUM> with the three-dimensional image <NUM>. For example, when the user is an operator, it is easy to perform treatment on the inside of the biological tissue <NUM>.

The biological tissue <NUM> is, for example, an organ such as a blood vessel or a heart.

A configuration of a diagnostic assistance system <NUM> according to the present embodiment will be described with reference to <FIG>.

The diagnostic assistance system <NUM> includes the diagnostic assistance device <NUM>, a cable <NUM>, a drive unit <NUM>, a keyboard <NUM>, a mouse <NUM>, and the display <NUM>.

The diagnostic assistance device <NUM> is a dedicated computer specialized for image diagnosing in the present embodiment, but may also be a general-purpose computer such as a PC. The term "PC" is an abbreviation of personal computer.

The cable <NUM> is used to connect the diagnostic assistance device <NUM> and the drive unit <NUM>.

The drive unit <NUM> is a device to be used by connecting to a probe <NUM> shown in <FIG> to drive the probe <NUM>. The drive unit <NUM> is also referred to as an MDU. The term "MDU" is an abbreviation of motor drive unit. The probe <NUM> is applied to IVUS. The probe <NUM> is also referred to as an IVUS catheter or an image diagnostic catheter.

The keyboard <NUM>, the mouse <NUM>, and the display <NUM> are connected to the diagnostic assistance device <NUM> via any cable or wirelessly. The display <NUM> is, for example, an LCD, an organic EL display, or an HMD. The term "LCD" is an abbreviation of liquid crystal display. The term "EL" is an abbreviation of electro luminescence. The term "HMD" is an abbreviation of head-mounted display.

The diagnostic assistance system <NUM> optionally further includes a connecting terminal <NUM> and a cart unit <NUM>.

The connecting terminal <NUM> is used to connect the diagnostic assistance device <NUM> and an external device. The connecting terminal <NUM> is, for example, a USB terminal. The term "USB" is an abbreviation of universal serial bus. The external device is, for example, a recording medium such as a magnetic disc drive, a magneto-optical disc drive, or an optical disc drive.

The cart unit <NUM> is a cart equipped with casters for movement. The diagnostic assistance device <NUM>, the cable <NUM>, and the drive unit <NUM> are disposed on a cart body of the cart unit <NUM>. The keyboard <NUM>, the mouse <NUM>, and the display <NUM> are disposed on an uppermost table of the cart unit <NUM>.

A configuration of the probe <NUM> and the drive unit <NUM> according to the present embodiment will be described with reference to <FIG>.

The probe <NUM> includes a drive shaft <NUM>, a hub <NUM>, a sheath <NUM>, an outer tube <NUM>, an ultrasound transducer <NUM>, and a relay connector <NUM>.

The drive shaft <NUM> passes through the sheath <NUM> to be inserted into a body cavity of a living body and the outer tube <NUM> connected to a proximal end of the sheath <NUM>, and extends to an inside of the hub <NUM> provided at a proximal end of the probe <NUM>. The drive shaft <NUM> is provided with the ultrasound transducer <NUM>, which transmits and receives signals, at a distal end thereof, and is rotatably provided in the sheath <NUM> and the outer tube <NUM>. The relay connector <NUM> connects the sheath <NUM> and the outer tube <NUM>.

The hub <NUM>, the drive shaft <NUM>, and the ultrasound transducer <NUM> are connected to each other so as to integrally move forward and backward in an axial direction. Therefore, for example, when the hub <NUM> is pressed toward a distal side, the drive shaft <NUM> and the ultrasound transducer <NUM> move inside the sheath <NUM> toward the distal side. For example, when the hub <NUM> is pulled toward a proximal side, the drive shaft <NUM> and the ultrasound transducer <NUM> move inside the sheath <NUM> toward the proximal side as indicated by arrows.

The drive unit <NUM> includes a scanner unit <NUM>, a slide unit <NUM>, and a bottom cover <NUM>.

The scanner unit <NUM> is connected to the diagnostic assistance device <NUM> via the cable <NUM>. The scanner unit <NUM> includes a probe connection unit <NUM> connected to the probe <NUM>, and a scanner motor <NUM> which is a drive source for rotating the drive shaft <NUM>.

The probe connection unit <NUM> is detachably connected to the probe <NUM> through an insertion port <NUM> of the hub <NUM> provided at the proximal end of the probe <NUM>. Inside the hub <NUM>, a proximal end of the drive shaft <NUM> is rotatably supported, and a rotational force of the scanner motor <NUM> is transmitted to the drive shaft <NUM>. A signal is transmitted and received between the drive shaft <NUM> and the diagnostic assistance device <NUM> via the cable <NUM>. In the diagnostic assistance device <NUM>, a tomographic image of a body lumen is generated and image processing is performed based on the signal transmitted from the drive shaft <NUM>.

The slide unit <NUM> is mounted with the scanner unit <NUM> in a manner capable of moving forward and backward, and is mechanically and electrically connected to the scanner unit <NUM>. The slide unit <NUM> includes a probe clamp unit <NUM>, a slide motor <NUM>, and a switch group <NUM>.

The probe clamp unit <NUM> is disposed coaxially with the probe connection unit <NUM> on a distal side of the probe connection unit <NUM>, and supports the probe <NUM> to be connected to the probe connection unit <NUM>.

The slide motor <NUM> is a drive source that generates a driving force in the axial direction. The scanner unit <NUM> moves forward and backward when driven by the slide motor <NUM>, and the drive shaft <NUM> moves forward and backward in the axial direction accordingly. The slide motor <NUM> is, for example, a servo motor.

The switch group <NUM> includes, for example, a forward switch and a pull-back switch that are pressed when the scanner unit <NUM> is to be moved forward or backward, and a scan switch that is pressed when image drawing is to be started or ended. Various switches may be included in the switch group <NUM> as necessary without being limited to the example here.

When the forward switch is pressed, the slide motor <NUM> rotates forward, and the scanner unit <NUM> moves forward. On the other hand, when the pull-back switch is pressed, the slide motor <NUM> rotates backward, and the scanner unit <NUM> moves backward.

When the scan switch is pressed, the image drawing is started, the scanner motor <NUM> is driven, and the slide motor <NUM> is driven to move the scanner unit <NUM> backward. A user such as an operator connects the probe <NUM> to the scanner unit <NUM> in advance, and rotates and moves the drive shaft <NUM> toward the proximal side in the axial direction upon the start of the image drawing. When the scan switch is pressed again, the scanner motor <NUM> and the slide motor <NUM> are stopped, and the image drawing is ended.

The bottom cover <NUM> covers a bottom and an entire circumference of a side surface on a bottom side of the slide unit <NUM>, and is capable of moving toward and away from the bottom of the slide unit <NUM>.

A configuration of the diagnostic assistance device <NUM> according to the present embodiment will be described with reference to <FIG>.

The diagnostic assistance device <NUM> includes a control unit <NUM>, a storage unit <NUM>, a communication unit <NUM>, an input unit <NUM>, and an output unit <NUM>.

The control unit <NUM> includes at least one processor, at least one dedicated circuit, or a combination thereof. The processor is a general-purpose processor such as a CPU or GPU, or a dedicated processor specialized for a specific process. The term "CPU" is an abbreviation of central processing unit. The term "GPU" is an abbreviation of graphics processing unit. As the dedicated circuit, for example, an FPGA or an ASIC can be used. The term "FPGA" is an abbreviation of field-programmable gate array. The term "ASIC" is an abbreviation of application specific integrated circuit. The control unit <NUM> executes processing related to an operation of the diagnostic assistance device <NUM> while controlling each unit of the diagnostic assistance system <NUM> including the diagnostic assistance device <NUM>.

The storage unit <NUM> includes at least one semiconductor memory, at least one magnetic memory, at least one optical memory, or a combination of at least two thereof. The semiconductor memory is, for example, a RAM or a ROM. The term "RAM" is an abbreviation of random access memory. The term "ROM" is an abbreviation of read only memory. The RAM is, for example, an SRAM or a DRAM. The term "SRAM" is an abbreviation of static random access memory. The term "DRAM" is an abbreviation of dynamic random access memory. The ROM is, for example, an EEPROM. The term "EEPROM" is an abbreviation of electrically erasable programmable read only memory. The storage unit <NUM> functions as, for example, a main storage device, an auxiliary storage device, or a cache memory. The storage unit <NUM> stores data used for the operation of the diagnostic assistance device <NUM>, such as the tomographic data <NUM>, and data obtained by the operation of the diagnostic assistance device <NUM>, such as the three-dimensional data <NUM> and the three-dimensional image <NUM>.

The communication unit <NUM> includes at least one communication interface. The communication interface is a wired LAN interface, a wireless LAN interface, or an image diagnostic interface for receiving IVUS signals and performing A/D conversion on the IVUS signals. The term "LAN" is an abbreviation of local area network. The term "A/D" is an abbreviation of analog to digital. The communication unit <NUM> receives data used for the operation of the diagnostic assistance device <NUM> and transmits data obtained by the operation of the diagnostic assistance device <NUM>. In the present embodiment, the drive unit <NUM> is connected to the image diagnostic interface included in the communication unit <NUM>.

The input unit <NUM> includes at least one input interface. The input interface is, for example, a USB interface, an HDMI (registered trademark) interface, or an interface compatible with short-range wireless communication such as Bluetooth (registered trademark). The term "HDMI" is an abbreviation of high-definition multimedia interface. The input unit <NUM> receives an operation of inputting data used for the operation of the diagnostic assistance device <NUM>. In the present embodiment, the keyboard <NUM> and the mouse <NUM> are connected to the USB interface or the interface corresponding to short-range wireless communication included in the input unit <NUM>. When a touch screen is provided integrally with the display <NUM>, the display <NUM> may be connected to the USB interface or the HDMI (registered trademark) interface included in the input unit <NUM>.

The output unit <NUM> includes at least one output interface. The output interface is, for example, a USB interface, an HDMI (registered trademark) interface, or an interface compatible with short-range wireless communication such as Bluetooth (registered trademark). The output unit <NUM> outputs the data obtained by the operation of the diagnostic assistance device <NUM>. In the present embodiment, the display <NUM> is connected to the USB interface or the HDMI (registered trademark) interface included in the output unit <NUM>.

A function of the diagnostic assistance device <NUM> is implemented by executing a diagnostic assistance program according to the present embodiment by the processor included in the control unit <NUM>. That is, the function of the diagnostic assistance device <NUM> is implemented by software. The diagnostic assistance program is a program for causing a computer to execute the processing of steps included in the operation of the diagnostic assistance device <NUM> to implement a function corresponding to the processing of the steps. That is, the diagnostic assistance program is a program for causing the computer to function as the diagnostic assistance device <NUM>.

The program can be recorded in a computer-readable recording medium. The computer-readable recording medium is, for example, a magnetic recording device, an optical disc, a magneto-optical recording medium, or a semiconductor memory. The program is distributed by, for example, selling, transferring, or lending a portable recording medium such as a DVD or a CD-ROM on which the program is recorded. The term "DVD" is an abbreviation of digital versatile disc. The term "CD-ROM" is an abbreviation of compact read only memory. The program may be distributed by storing the program in a storage of a server and transferring the program from the server to another computer via a network. The program may be provided as a program product.

For example, the computer temporarily stores the program recorded in the portable recording medium or the program transferred from the server in the main storage device. The computer causes the processor to read the program stored in the main storage device, and causes the processor to execute processing according to the read program. The computer may read the program directly from the portable recording medium and execute the processing according to the program. Each time the program is transferred from the server to the computer, the computer may sequentially execute processing according to the received program. The processing may be executed by a so-called ASP type service in which the function is implemented only by execution instruction and result acquisition without transferring the program from the server to the computer. The term "ASP" is an abbreviation of application service provider. The program includes information provided for processing by an electronic computer and conforming to the program. For example, data that is not a direct command to the computer but has a property that defines the processing of the computer corresponds to the "information conforming to the program".

The functions of the diagnostic assistance device <NUM> may be partially or entirely implemented by the dedicated circuit included in the control unit <NUM>. That is, the functions of the diagnostic assistance device <NUM> may be partially or entirely implemented by hardware.

An operation of the diagnostic assistance system <NUM> according to the present embodiment will be described with reference to <FIG>. The operation of the diagnostic assistance system <NUM> corresponds to a diagnostic assistance method.

Before the start of a flow in <FIG>, the probe <NUM> is primed by the user. Thereafter, the probe <NUM> is fitted into the probe connection unit <NUM> and the probe clamp unit <NUM> of the drive unit <NUM>, and is connected and fixed to the drive unit <NUM>. The probe <NUM> is inserted to a target site in the biological tissue <NUM> such as the blood vessel or the heart.

In step S1, the scan switch included in the switch group <NUM> is pressed, and a so-called pull-back operation is performed by pressing the pull-back switch included in the switch group <NUM>. The probe <NUM> transmits ultrasound inside the biological tissue <NUM> by the ultrasound transducer <NUM> that moves backward in the axial direction by the pullback operation. The ultrasound transducer <NUM> radially transmits the ultrasound while moving inside the biological tissue <NUM>. The ultrasonic transducer <NUM> receives a reflected wave of the transmitted ultrasound. The probe <NUM> inputs a signal of the reflected wave received by the ultrasound transducer <NUM> to the diagnostic assistance device <NUM>. The control unit <NUM> of the diagnostic assistance device <NUM> processes the input signal to sequentially generate cross-sectional images of the biological tissue <NUM>, thereby acquiring the tomographic data <NUM>, which includes a plurality of cross-sectional images.

Specifically, the probe <NUM> transmits ultrasonic waves in a plurality of directions from a rotation center to an outside by the ultrasound transducer <NUM> while causing the ultrasound transducer <NUM> to rotate in a circumferential direction and to move in the axial direction inside the biological tissue <NUM>. The probe <NUM> receives the reflected wave from a reflecting object existing in each of the plurality of directions inside the biological tissue <NUM> by the ultrasound transducer <NUM>. The probe <NUM> transmits the signal of the received reflected wave to the diagnostic assistance device <NUM> via the drive unit <NUM> and the cable <NUM>. The communication unit <NUM> of the diagnostic assistance device <NUM> receives the signal transmitted from the probe <NUM>. The communication unit <NUM> performs A/D conversion on the received signal. The communication unit <NUM> inputs the A/D-converted signal to the control unit <NUM>. The control unit <NUM> processes the input signal to calculate an intensity value distribution of the reflected wave from the reflecting object existing in a transmission direction of the ultrasound of the ultrasound transducer <NUM>. The control unit <NUM> sequentially generates two-dimensional images having a luminance value distribution corresponding to the calculated intensity value distribution as the cross-sectional images of the biological tissue <NUM>, thereby acquiring the tomographic data <NUM> which is a data set of the cross-sectional images. The control unit <NUM> stores the acquired tomographic data <NUM> in the storage unit <NUM>.

In the present embodiment, the signal of the reflected wave received by the ultrasound transducer <NUM> corresponds to raw data of the tomographic data <NUM>, and the cross-sectional images generated by processing the signal of the reflected wave by the diagnostic assistance device <NUM> correspond to processed data of the tomographic data <NUM>.

In a modification of the present embodiment, the control unit <NUM> of the diagnostic assistance device <NUM> may store the signal input from the probe <NUM> as it is in the storage unit <NUM> as the tomographic data <NUM>. Alternatively, the control unit <NUM> may store data indicating the intensity value distribution of the reflected wave calculated by processing the signal input from the probe <NUM> in the storage unit <NUM> as the tomographic data <NUM>. That is, the tomographic data <NUM> is not limited to the data set of the cross-sectional images of the biological tissue <NUM>, and may be data representing a cross section of the biological tissue <NUM> at each moving position of the ultrasound transducer <NUM> in some format.

In a modification of the present embodiment, an ultrasound transducer that transmits ultrasound in a plurality of directions without rotating may be used instead of the ultrasound transducer <NUM> that transmits ultrasound in the plurality of directions while rotating in the circumferential direction.

In a modification of the present embodiment, the tomographic data <NUM> may be acquired using OFDI or OCT instead of being acquired by using the IVUS. The term "OFDI" is an abbreviation of optical frequency domain imaging. The term "OCT" is an abbreviation of optical coherence tomography. When OFDI or OCT is used, as a sensor that acquires the tomographic data <NUM> while moving in the biological tissue <NUM>, a sensor that acquires the tomographic data <NUM> by emitting light in the biological tissue <NUM> is used instead of the ultrasound transducer <NUM> that acquires the tomographic data <NUM> by transmitting the ultrasound in the biological tissue <NUM>.

In a modification of the present embodiment, instead of the diagnostic assistance device <NUM> generating the data set of the cross-sectional images of the biological tissue <NUM>, another device may generate the same data set, and the diagnostic assistance device <NUM> may acquire the data set from the other device. That is, instead of the control unit <NUM> of the diagnostic assistance device <NUM> processing the IVUS signal to generate the cross-sectional image of the biological tissue <NUM>, another device may process the IVUS signal to generate the cross-sectional image of the biological tissue <NUM> and input the generated cross-sectional image to the diagnostic assistance device <NUM>.

In step S2, the control unit <NUM> of the diagnostic assistance device <NUM> generates the three-dimensional data <NUM> of the biological tissue <NUM> based on the tomographic data <NUM> acquired in step S1.

Specifically, the control unit <NUM> of the diagnostic assistance device <NUM> generates the three-dimensional data <NUM> of the biological tissue <NUM> by stacking the cross-sectional images of the biological tissue <NUM> included in the tomographic data <NUM> stored in the storage unit <NUM>, and converting the same into three-dimensional data. As a method of three-dimensional conversion, any process among a rendering method such as surface rendering or volume rendering, texture mapping accompanying the rendering method, such as environment mapping and bump mapping, or the like is used. The control unit <NUM> stores the generated three-dimensional data <NUM> in the storage unit <NUM>.

In step S3, the control unit <NUM> of the diagnostic assistance device <NUM> displays the three-dimensional data <NUM> generated in step S2 on the display <NUM> as the three-dimensional image <NUM>. At this point, the control unit <NUM> may arrange the viewpoint when displaying the three-dimensional image <NUM> on the display <NUM> at any position.

Specifically, the control unit <NUM> of the diagnostic assistance device <NUM> generates the three-dimensional image <NUM> from the three-dimensional data <NUM> stored in the storage unit <NUM>. The control unit <NUM> displays the generated three-dimensional image <NUM> on the display <NUM> via the output unit <NUM>.

In step S4, when operated by the user, the processing from step S5 to step S8 is performed. When not operated by the user, the processing from step S5 to step S8 is skipped.

In step S5, the control unit <NUM> of the diagnostic assistance device <NUM> receives, via the input unit <NUM>, an operation of setting a position of the opening <NUM> as shown in <FIG>. The position of the opening <NUM> is set to a position at which the inner wall surface <NUM> of the biological tissue <NUM> is exposed to the outside of the biological tissue <NUM> through the opening <NUM> in the three-dimensional image <NUM> displayed in step S3.

Specifically, the control unit <NUM> of the diagnostic assistance device <NUM> receives, via the input unit <NUM>, an operation of the user cutting off a portion of the biological tissue <NUM> using the keyboard <NUM>, the mouse <NUM>, or the touch screen provided integrally with the display <NUM> in the three-dimensional image <NUM> displayed on the display <NUM>. In the example of <FIG>, the control unit <NUM> receives an operation of cutting off a portion of the biological tissue <NUM> so that the inner wall surface <NUM> of the biological tissue <NUM> has an opened shape in the cross section of the biological tissue <NUM>. The term "cross section of the biological tissue <NUM>" refers to, for example, a tomographic cross section having two edges of the opening <NUM> facing each other and the inner wall surface <NUM> of the biological tissue <NUM> facing the opening <NUM>, but is not limited to this tomographic cross section, and may be a transverse cross section of the biological tissue <NUM>, a longitudinal cross section of the biological tissue <NUM>, or another cross section of the biological tissue <NUM>. The term "transverse cross section of the biological tissue <NUM>" refers to a cross section obtained by cutting the biological tissue <NUM> perpendicularly to a direction in which the ultrasound transducer <NUM> moves in the biological tissue <NUM>. The term "longitudinal cross section of the biological tissue <NUM>" refers to a cross section obtained by cutting the biological tissue <NUM> along a direction in which the ultrasound transducer <NUM> moves in the biological tissue <NUM>. The term "another cross section of the biological tissue <NUM>" refers to a cross section obtained by cutting the biological tissue <NUM> obliquely with respect to a direction in which the ultrasound transducer <NUM> moves in the biological tissue <NUM>. The term "opened shape" refers to, for example, a substantially C shape, a substantially U shape, a substantially "<NUM>" shape, or a shape in which any of these shapes is partially missing due to a hole originally opened in the biological tissue <NUM>, such as a bifurcated portion of the blood vessel or pulmonary vein ostia. In the example of <FIG>, a shape of the inner wall surface <NUM> of the biological tissue <NUM> is a substantially C shape.

In step S6, the control unit <NUM> of the diagnostic assistance device <NUM> determines the position set by the operation received in step S5 as the position of the opening <NUM>.

Specifically, the control unit <NUM> of the diagnostic assistance device <NUM> specifies, as three-dimensional coordinates of an edge of the opening <NUM>, three-dimensional coordinates of a boundary of a portion of the biological tissue <NUM> cut off by the operation of the user in the three-dimensional data <NUM> stored in the storage unit <NUM>. The control unit <NUM> stores the specified three-dimensional coordinates in the storage unit <NUM>.

In step S7, the control unit <NUM> of the diagnostic assistance device <NUM> forms, in the three-dimensional data <NUM>, the opening <NUM> that exposes the inner wall surface <NUM> of the biological tissue <NUM> to the outside of the biological tissue <NUM> in the three-dimensional image <NUM>.

Specifically, the control unit <NUM> of the diagnostic assistance device <NUM> sets a portion in the three-dimensional data <NUM> stored in the storage unit <NUM> that is specified by the three-dimensional coordinates stored in the storage unit <NUM> to be hidden or transparent when the three-dimensional image <NUM> is to be displayed on the display <NUM>.

In step S8, the control unit <NUM> of the diagnostic assistance device <NUM> adjusts the viewpoint when displaying the three-dimensional image <NUM> on the display <NUM> according to the position of the opening <NUM> formed in step S7. In the present embodiment, the control unit <NUM> arranges the viewpoint on a straight line extending from the inner wall surface <NUM> of the biological tissue <NUM> to the outside of the biological tissue <NUM> through the opening <NUM>. Therefore, the user can virtually observe the inner wall surface <NUM> of the biological tissue <NUM> by looking into the biological tissue <NUM> through the opening <NUM>.

Specifically, the control unit <NUM> of the diagnostic assistance device <NUM> arranges the virtual camera <NUM> at a position where the inner wall surface <NUM> of the biological tissue <NUM> can be seen through the portion set to be hidden or transparent in the three-dimensional image <NUM> displayed on the display <NUM>. In the example of <FIG>, the control unit <NUM> arranges the virtual camera <NUM> in a region AF sandwiched between a first straight line L1 and a second straight line L2 in the cross section of the biological tissue <NUM>. The first straight line L1 extends from the inner wall surface <NUM> of the biological tissue <NUM> to the outside of the biological tissue <NUM> through a first edge E1 of the opening <NUM>. The second straight line L2 extends from the inner wall surface <NUM> of the biological tissue <NUM> to the outside of the biological tissue <NUM> through a second edge E2 of the opening <NUM>. A point at which the first straight line L1 intersects the inner wall surface <NUM> of the biological tissue <NUM> is a point Pt identical to a point at which the second straight line L2 intersects the inner wall surface <NUM> of the biological tissue <NUM>. Therefore, the user can observe the point Pt on the inner wall surface <NUM> of the biological tissue <NUM> regardless of a position of the virtual camera <NUM> in the region AF.

In the example of <FIG>, the point Pt is identical to a point at which a fourth straight line L4 intersects the inner wall surface <NUM> of the biological tissue <NUM>. The fourth straight line L4 is drawn perpendicularly to a third straight line L3 from a midpoint Pc of the third straight line L3. The third straight line L3 connects the first edge E1 of the opening <NUM> and the second edge E2 of the opening <NUM>. Therefore, the user can easily observe the point Pt on the inner wall surface <NUM> of the biological tissue <NUM> through the opening <NUM>. In particular, when the virtual camera <NUM> is arranged on an extension line of the fourth straight line L4, the user can easily observe the point Pt on the inner wall surface <NUM> of the biological tissue <NUM>.

The position of the virtual camera <NUM> may be any position at which the inner wall surface <NUM> of the biological tissue <NUM> can be observed through the opening <NUM>, and is within a range facing the opening <NUM> in the present embodiment. The position of the virtual camera <NUM> is preferably set to an intermediate position facing a central portion of the opening <NUM>.

In the example of <FIG>, a minimum value Smin and a maximum value Smax are set for a ratio S of a distance Un from a center to one end of the three-dimensional image <NUM> displayed on a screen <NUM> of the display <NUM> to a distance Um from a center to one end of the screen <NUM> such that the centers of the screen <NUM> and the three-dimensional image <NUM> overlap with each other. For example, Smin is set to <NUM>/<NUM>, and Smax is set to <NUM>. In the example of <FIG>, a minimum distance Lmin from the point Pt to the position of the camera <NUM> may be set according to the minimum value Smin, and a maximum distance Lmax from the point Pt to the position of the virtual camera <NUM> may be set according to the maximum value Smax. Alternatively, the minimum distance Lmin from the point Pt to the position of the camera <NUM> may be set to such a distance that the camera <NUM> is not closer to the point Pt than the opening <NUM> regardless of the minimum value Smin. The maximum distance Lmax from the point Pt to the position of the virtual camera <NUM> may be set to such a distance that the camera <NUM> is not away from the point Pt more than such a distant that the user cannot observe the inner wall surface <NUM> of the biological tissue <NUM> regardless of the maximum value Smax.

In step S9, when the tomographic data <NUM> is updated, the processing of step S1 and the subsequent steps are performed again. When the tomographic data <NUM> is not updated, whether operated by the user is confirmed again in step S4.

In steps S5 to S8 for the second and subsequent times, when the position of the opening <NUM> is changed from a first position to a second position as shown in <FIG> and <FIG>, the control unit <NUM> of the diagnostic assistance device <NUM> moves the viewpoint from a third position corresponding to the first position to a fourth position corresponding to the second position. The control unit <NUM> moves virtual light sources <NUM> when the three-dimensional image <NUM> is to be displayed on the display <NUM>, in accordance with movement of the viewpoint from the third position to the fourth position.

As shown in <FIG> and <FIG>, the control unit <NUM> moves the virtual light sources <NUM> by using a rotation matrix R(θ) used for moving the virtual camera <NUM> when changing the position of the opening <NUM> in the circumferential direction in the cross section of the biological tissue <NUM>. The number and the relative positions of the light sources <NUM> are not limited to those illustrated in the drawings, and can be changed as appropriate.

The control unit <NUM> may instantly switch the viewpoint from the third position to the fourth position when changing the position of the opening <NUM> from the first position to the second position, but in the present embodiment, a video in which the viewpoint gradually moves from the third position to the fourth position is displayed on the display <NUM> as the three-dimensional image <NUM>. Therefore, the movement of the viewpoint is easily introduced to the user.

In a modification of the present embodiment, in step S5, the control unit <NUM> of the diagnostic assistance device <NUM> may receive, via the input unit <NUM>, an operation of setting a position of a target point that the user wants to see and an operation of setting the position of the opening <NUM>.

Specifically, in the three-dimensional image <NUM> displayed on the display <NUM>, the control unit <NUM> of the diagnostic assistance device <NUM> may receive, via the input unit <NUM>, an operation of designating the position of the target point using the keyboard <NUM>, the mouse <NUM>, or the touch screen provided integrally with the display <NUM> by the user. In the example of <FIG>, the control unit <NUM> may receive, via the input unit <NUM>, an operation of setting a position of the point Pt as a position of the point at which the first straight line L1 and the second straight line L2 intersect the inner wall surface <NUM> of the biological tissue <NUM>.

In a modification of the present embodiment, in step S5, the control unit <NUM> of the diagnostic assistance device <NUM> may receive, via the input unit <NUM>, an operation of setting the position of the target point that the user wants to see, instead of the operation of setting the position of the opening <NUM>. In step S6, the control unit <NUM> may determine the position of the opening <NUM> according to the position set by the operation received in step S5.

Specifically, in the three-dimensional image <NUM> displayed on the display <NUM>, the control unit <NUM> of the diagnostic assistance device <NUM> may receive, via the input unit <NUM>, the operation of designating the position of the target point using the keyboard <NUM>, the mouse <NUM>, or the touch screen provided integrally with the display <NUM> by the user. The control unit <NUM> may determine the position of the opening <NUM> according to the position of the target point. In the example of <FIG>, the control unit <NUM> may receive, via the input unit <NUM>, an operation of setting the position of the point Pt as the position of the point at which the first straight line L1 and the second straight line L2 intersect the inner wall surface <NUM> of the biological tissue <NUM>. In the cross section of the biological tissue <NUM>, the control unit <NUM> may determine, as the region AF, a fan-shaped region centered on the point Pt and having a central angle that is preset or that is an angle α specified by the user. The control unit <NUM> may determine a position in the biological tissue <NUM> that overlaps with the region AF as the position of the opening <NUM>. The control unit <NUM> may determine a normal line of the inner wall surface <NUM> of the biological tissue <NUM> perpendicular to a tangent line passing through the point Pt as the fourth straight line L4.

The region AF may be set to be narrower than a width of the opening <NUM>. That is, the region AF may be set so as not to include at least one of the first edge E1 of the opening <NUM> and the second edge E2 of the opening <NUM>.

As shown in <FIG>, the point at which the first straight line L1 intersects the inner wall surface <NUM> of the biological tissue <NUM> may not be identical to the point at which the second straight line L2 intersects the inner wall surface <NUM> of the biological tissue <NUM>. In the example of <FIG>, a point P1 at which the first straight line L1 intersects the inner wall surface <NUM> of the biological tissue <NUM> and a point P2 at which the second straight line L2 intersects the inner wall surface <NUM> of the biological tissue <NUM> are on a circumference of a radius r centered on the point Pt. That is, the point P1 and the point P2 are substantially equidistant from the point Pt.

In the example of <FIG>, as in the example of <FIG>, the point Pt is identical to a point at which the fourth straight line L4 intersects the inner wall surface <NUM> of the biological tissue <NUM>. The fourth straight line L4 is drawn perpendicularly to the third straight line L3 from the midpoint Pc of the third straight line L3. The third straight line L3 connects the first edge E1 of the opening <NUM> and the second edge E2 of the opening <NUM>. Therefore, the user can easily observe the point Pt on the inner wall surface <NUM> of the biological tissue <NUM> and the periphery thereof through the opening <NUM>. In particular, when the virtual camera <NUM> is arranged on the extension line of the fourth straight line L4, the user can easily observe the point Pt on the inner wall surface <NUM> of the biological tissue <NUM> and the periphery thereof.

The point Pt as a target point of observation, and the radius r as a target range of observation may be set manually. For example, the point Pt may be set by clicking the three-dimensional image <NUM>, and the radius r may be set by a menu. A sphere including a zone observable by the user is defined by setting the point Pt and the radius r. Further, the angle α, a unit vector in a direction from the point Pt toward the front side in <FIG>, that is, a unit vector up in a direction orthogonal to the cross section of the biological tissue <NUM>, and a unit vector dir in a direction from the point Pt toward the midpoint Pc may be set manually. As a result, the point P1 and the point P2, which are both end points of the zone observable by the user, and a point Pf, which is a vertex of a cone inscribed in the sphere including the zone observable by the user, are automatically set as follows. <MAT> <MAT> <MAT>.

Here, cross() is outer product calculation, and normalize() is unit vector calculation.

In a modification of the present embodiment, in step S3, the control unit <NUM> of the diagnostic assistance device <NUM> may display a plurality of images on the display <NUM> as the three-dimensional image <NUM>. Specifically, as shown in <FIG>, the control unit <NUM> may display a first image 53a and a second image 53b on the display <NUM> side by side as the three-dimensional image <NUM>. In the modification, the control unit <NUM> forms a first opening as the opening <NUM>, and adjusts a first viewpoint as the viewpoint for the first image 53a. The control unit <NUM> forms, at a position obtained by rotating a position of the first opening in the circumferential direction, a second opening that exposes the inner wall surface <NUM> of the biological tissue <NUM> to the outside of the biological tissue <NUM> in the second image 53b. The control unit <NUM> adjusts a second viewpoint when the second image 53b is to be displayed on the display <NUM> according to the position of the second opening. The position of the second opening is, for example, a position obtained by rotating the position of the first opening by <NUM> degrees in the circumferential direction.

In the example of <FIG>, the biological tissue <NUM> is the heart. In the example, when the position of the first opening and the first viewpoint are set, the second opening obtained by rotating the position of the first opening by <NUM> degrees in the circumferential direction and the second viewpoint are set automatically. The first image 53a in which the inner wall surface <NUM> can be seen through the first opening from the first viewpoint and the second image 53b in which the inner wall surface <NUM> can be seen through the second opening from the second viewpoint are displayed side by side.

According to the example, since an image from the second viewpoint which captures a fossa ovalis <NUM> from the lateral side is displayed together with an image from the first viewpoint which captures the fossa ovalis <NUM> from the front, when the operator is to puncture the fossa ovalis <NUM> from a left atrium toward a right atrium, it is possible to visually grasp a distance from the fossa ovalis <NUM> to a right atrium wall, and to prevent the right atrium wall from being erroneously punctured.

In a modification of the present embodiment, in step S7 and step S8, the control unit <NUM> of the diagnostic assistance device <NUM> may form the opening <NUM> by cutting a portion of the biological tissue <NUM> in the three-dimensional data <NUM>, and may further display the position of the camera <NUM> and a cut line on the display <NUM> when the viewpoint is adjusted. Specifically, as shown in <FIG>, the control unit <NUM> may display the position of the camera <NUM> in the cross section of the biological tissue <NUM> and the first straight line L1 and the second straight line L2 corresponding to the cut line in a state where the biological tissue <NUM> is not cut. In the example of <FIG>, in order to clearly indicate which portion of the cross section is cut, a color of a region <NUM> corresponding to the cut portion is changed.

The region <NUM> corresponding to the cut portion may be displayed on the original cross-sectional image. For example, as shown in <FIG>, the control unit <NUM> of the diagnostic assistance device <NUM> may display, on the display <NUM>, a two-dimensional image <NUM> that represents the region <NUM> corresponding to the cut portion on the cross-sectional image of the biological tissue <NUM>. In the example of <FIG>, the position of the camera <NUM> is also represented by the two-dimensional image <NUM>. As the cross-sectional image, an image corresponding to a current position of the sensor may be used, but an image corresponding to a position other than the current position of the sensor may also be used.

As described above, in the present embodiment, the control unit <NUM> of the diagnostic assistance device <NUM> generates the three-dimensional data <NUM> of the biological tissue <NUM> based on the tomographic data <NUM> of the biological tissue <NUM>. The control unit <NUM> displays the generated three-dimensional data <NUM> as the three-dimensional image <NUM> on the display <NUM>. The control unit <NUM> forms, in the three-dimensional data <NUM>, the opening <NUM> that exposes the inner wall surface <NUM> of the biological tissue <NUM> to the outside of the biological tissue <NUM> in the three-dimensional image <NUM>. The control unit <NUM> adjusts the viewpoint when displaying the three-dimensional image <NUM> on the display <NUM> according to the position of the formed opening <NUM>.

According to the present embodiment, the user can see the inside of the biological tissue <NUM> with the three-dimensional image <NUM>. For example, when the user is an operator, it is easy to perform treatment on the inside of the biological tissue <NUM>.

In the present embodiment, once the position of the opening <NUM> is determined, the positions of the camera <NUM> and the light sources <NUM> move such that the inside of the biological tissue <NUM> can be seen from the opening <NUM>. Therefore, when the position of the opening <NUM> is changed to another position, it is possible to avoid a situation that only the outer wall surface of the biological tissue <NUM> can be seen and an object of interest cannot be confirmed.

Claim 1:
A diagnostic assistance device (<NUM>) configured to generate three-dimensional data (<NUM>) of a biological tissue (<NUM>) based on tomographic data (<NUM>) of the biological tissue (<NUM>), and display (<NUM>) the generated three-dimensional data (<NUM>) as a three-dimensional image (<NUM>) on a display (<NUM>), the diagnostic assistance device (<NUM>) comprising:
a control unit (<NUM>) configured to form, in the three-dimensional data (<NUM>), an opening (<NUM>) that exposes an inner wall surface (<NUM>) of the biological tissue (<NUM>) to an outside of the biological tissue (<NUM>) in the three-dimensional image (<NUM>), and adjust a viewpoint when displaying the three-dimensional image (<NUM>) on the display (<NUM>) according to a position of the formed opening (<NUM>), and, when
the position of the opening (<NUM>) is to be changed from a first position to a second position, to move the viewpoint from a third position corresponding to the first position to a fourth position corresponding to the second position ,
characterized by the control unit (<NUM>) being further configured to display, on the display (<NUM>), a video in which the viewpoint gradually moves from the third position to the fourth position as the three-dimensional image (<NUM>).