Patent Description:
In the related art, in an examination of the inside of the body of a subject by an endoscope (hereinafter, referred to as an "endoscopic examination"), the endoscopic examination may be supported by detecting the shape of an insertion part of an endoscope inserted into the body of the subject and displaying a shape image showing the shape of the insertion part on a display unit.

For example, <CIT> discloses a technique for combining an endoscopic image obtained by an endoscope and a shape image and displaying the combined image on a display unit. In addition, for example, <CIT> discloses a technique for displaying the shape of an insertion part of an endoscope in a case of being observed in a predetermined viewpoint direction, and the shape of the insertion part in a viewpoint direction different by <NUM> degrees from the viewpoint direction on a display unit. An image processing device according to the preamble of claim <NUM> is known from <CIT>.

However, in the techniques disclosed in <CIT> and <CIT>, although the shape image is displayed, it may be insufficient as a display for observing a predetermined shape as the shape of the insertion part, for example, a so-called loop shape in which an examiner wants to observe the state. Therefore, it may be difficult for the examiner to grasp the shape of the insertion part of the endoscope.

The present disclosure has been made in view of the above circumstances, and an object of the present disclosure is to provide an image processing device, an image processing method, and an image processing program capable of easily grasping the shape of an insertion part of an endoscope.

In order to achieve the above object, there is provided an image processing device according to a first aspect of the present disclosure. The image processing device comprises: an acquisition unit that acquires a shape of an insertion part to be inserted into a subject in an endoscope; a storage unit in which a predetermined shape is stored; a determination unit that determines whether or not the shape of the insertion part includes the predetermined shape stored in the storage unit; and a display controller that controls a display unit to display a first shape image in a case where determination is made that the shape of the insertion part does not include the predetermined shape, and controls the display unit to display the first shape image and a second shape image in a case where determination is made that the shape of the insertion part includes the predetermined shape, the first shape image representing the shape of the insertion part in a first viewpoint direction, the second shape image including a portion corresponding to the predetermined shape of the insertion part in a second viewpoint direction different from the first viewpoint direction.

An image processing device according to a second aspect of the present disclosure, in the image processing device according to the first aspect, further comprises a generation unit that generates the first shape image in a case where the determination unit determines that the shape of the insertion part does not include the predetermined shape, and generates the first shape image and the second shape image in a case where the determination unit determines that the shape of the insertion part includes the predetermined shape, the first shape image representing the shape of the insertion part in the first viewpoint direction, the second shape image including a portion corresponding to the predetermined shape of the insertion part in the second viewpoint direction different from the first viewpoint direction.

In an image processing device according to a third aspect of the present disclosure, in the image processing device according to the first or second aspect, the predetermined shape is a loop shape.

In an image processing device according to a fourth aspect of the present disclosure, in the image processing device according to the third aspect, the second viewpoint direction is a direction of connection from an intersection point of a loop-shaped portion of the insertion part to a center point of the loop-shaped portion.

In an image processing device according to a fifth aspect of the present disclosure, in the image processing device according to the third aspect, the second viewpoint direction is a predetermined direction as a viewpoint direction for grasping an intersecting state of a loop-shaped portion of the insertion part.

In an image processing device according to a sixth aspect of the present disclosure, in the image processing device according to any one of the first to fifth aspects, in a case where the determination unit determines that the shape of the insertion part includes the predetermined shape, the display controller controls the display unit to display the second shape image on a partial region in the first shape image.

In an image processing device according to a seventh aspect of the present disclosure, in the image processing device according to any one of the first to sixth aspects, in a case where the determination unit determines that the shape of the insertion part includes the predetermined shape, the display controller controls the display unit to further display information that is visually recognizable in the second viewpoint direction.

In an image processing device according to an eighth aspect of the present disclosure, in the image processing device according to any one of the first to seventh aspects, in a case where the determination unit determines that the shape of the insertion part includes the predetermined shape, the display controller performs control such that the second shape image in which the second viewpoint direction is changed is displayed for each changed second viewpoint direction.

In an image processing device according to a ninth aspect of the present disclosure, in the image processing device according to any one of the first to seventh aspects, in a case where the shape of the insertion part includes the predetermined shape, the display controller performs control such that the second shape image in which the portion corresponding to the predetermined shape of the insertion part is rotated is displayed for each rotated rotation position.

In an image processing device according to a tenth aspect of the present disclosure, in the image processing device according to any one of the first to ninth aspects, the first viewpoint direction is a predetermined direction as a viewpoint direction for grasping an overall shape of the insertion part.

In an image processing device according to an eleventh aspect of the present disclosure, in the image processing device according to any one of the first to tenth aspects, the display controller controls the display unit to further display an endoscopic image obtained by the endoscope.

An image processing device according to a twelfth aspect of the present disclosure, in the image processing device according to any one of the first to eleventh aspects, further comprises a detection unit that detects the shape of the insertion part to be inserted into the subject in the endoscope, and the acquisition unit acquires the shape of the insertion part from the detection unit.

There is provided an image processing method according to a thirteenth aspect of the present disclosure. The image processing method executed by a computer, comprises: acquiring a shape of an insertion part to be inserted into a subject in an endoscope; acquiring a predetermined shape stored in a storage unit; determining whether or not the shape of the insertion part includes the predetermined shape; and controlling a display unit to display a first shape image in a case where determination is made that the shape of the insertion part does not include the predetermined shape, and controlling the display unit to display the first shape image and a second shape image in a case where determination is made that the shape of the insertion part includes the predetermined shape, the first shape image representing the shape of the insertion part in a first viewpoint direction, the second shape image including a portion corresponding to the predetermined shape of the insertion part in a second viewpoint direction different from the first viewpoint direction.

There is provided an image processing program according to a fourteenth aspect of the present disclosure. The image processing program causes a computer to execute a process comprising: acquiring a shape of an insertion part to be inserted into a subject in an endoscope; acquiring a predetermined shape stored in a storage unit; determining whether or not the shape of the insertion part includes the predetermined shape; and controlling a display unit to display a first shape image in a case where the shape of the insertion part does not include the predetermined shape, and controlling the display unit to display the first shape image and a second shape image in a case where the shape of the insertion part includes the predetermined shape, the first shape image representing the shape of the insertion part in a first viewpoint direction, the second shape image including a portion corresponding to the predetermined shape of the insertion part in a second viewpoint direction different from the first viewpoint direction.

In addition, the image processing device according to the aspects of the present disclosure is an image processing device including a processor. The processor acquires a shape of an insertion part to be inserted into a subject in an endoscope, controls a display unit to display a first shape image in a case where the shape of the insertion part does not include a predetermined shape, and controls the display unit to display the first shape image and a second shape image in a case where the shape of the insertion part includes the predetermined shape, the first shape image representing the shape of the insertion part in a first viewpoint direction, the second shape image including a portion corresponding to the predetermined shape of the insertion part in a second viewpoint direction different from the first viewpoint direction.

According to the present disclosure, it is possible to easily grasp the shape of the insertion part of the endoscope.

Hereinafter, examples of an embodiment for implementing a technique of the present disclosure will be described in detail with reference to the drawings.

First, the overall configuration of an endoscope system <NUM> of the present embodiment will be described with reference to <FIG> is a configuration diagram showing an example of a configuration of the endoscope system <NUM> of the present embodiment.

The endoscope system <NUM> comprises an endoscope <NUM> that captures an image of the inside of the body of a subject W (hereinafter, referred to as an "endoscopic image"), an endoscopic examination device <NUM>, and a position detection device <NUM>.

The endoscope <NUM> comprises an insertion part 10A and an operation part 10B, and in a case where an endoscopic examination is performed, an examiner operates the operation part 10B to insert the insertion part 10A into the body of the subject W, and captures an endoscopic image of the inside of the body of the subject W. The endoscopic examination device <NUM> connected to the endoscope <NUM> by a cable <NUM> comprises a video processor <NUM>, an overall controller <NUM>, an image processing unit <NUM>, and a display unit <NUM> such as a liquid crystal display. The video processor <NUM> controls capturing of an endoscopic image by the endoscope <NUM>. The overall controller <NUM> controls the entire endoscope system <NUM>. The image processing unit <NUM> performs image processing on various images displayed on the display unit <NUM>. The image processing unit <NUM> of the present embodiment is an example of an image processing device of the present disclosure. On the other hand, the position detection device <NUM> comprises a transmission unit <NUM> (see <FIG>) provided in the endoscopic examination device <NUM> and a reception unit <NUM> (see <FIG>) provided inside the endoscope <NUM>, and detects a position of the insertion part 10A by receiving a magnetic field generated by the transmission unit <NUM> at the reception unit <NUM>.

Next, with reference to <FIG>, detailed configurations of the endoscope <NUM>, the endoscopic examination device <NUM>, and the position detection device <NUM> will be described. <FIG> is a block diagram showing an example of the configuration of the endoscope system <NUM> of the present embodiment.

As shown in <FIG>, the endoscope <NUM> comprises an image sensor <NUM> including an imaging element such as a charge coupled device (CCD) image sensor and a complementary metal-oxide-semiconductor (CMOS) image sensor. Under the control of the video processor <NUM>, the endoscope <NUM> transmits light emitted from a light source <NUM> through a transmission path (not shown), emits the light from an emitting portion (not shown) provided at the distal end of the insertion part 10A, and illuminates the inside of the body of the subject W with the emitted light. Reflected light from the subject W due to this illumination light forms an image on the image sensor <NUM> by an objective lens (not shown), an image signal corresponding to an endoscopic image, which is a formed optical image, is output to the video processor <NUM> of the endoscopic examination device <NUM> via the cable <NUM>. The video processor <NUM> performs predetermined image processing on the input image signal, and image data of the endoscopic image obtained by this image processing is output to the image processing unit <NUM>.

As shown in <FIG>, in the position detection device <NUM>, the transmission unit <NUM> provided inside the endoscopic examination device <NUM> comprises a transmission control unit <NUM> and a transmitting coil unit <NUM>. As shown in <FIG>, the transmitting coil unit <NUM> comprises a plurality (<NUM> in the present embodiment) of transmitting coils <NUM>, specifically, transmitting coils <NUM>1X, <NUM>1Y, <NUM>1Z, <NUM>2X, <NUM>2Y, <NUM>2Z, <NUM>3X, <NUM>3Y, <NUM>3Z, <NUM>4X, <NUM>4Y, and <NUM>4Z. In the present embodiment, in the case where the transmitting coils <NUM> are collectively referred to, the transmitting coils <NUM> are simply referred to as the "transmitting coil <NUM>", and in the case of distinguishing each, reference numerals (1X,. , 4Z) representing each are added after the "transmitting coil <NUM>".

As shown in <FIG>, the transmitting coil <NUM> of the present embodiment includes, as a set, three transmitting coils <NUM> whose axes are oriented in the directions of an X-axis, a Y-axis, and a Z-axis, respectively, and the transmitting coil unit <NUM> comprises four sets of transmitting coil groups. Specifically, the transmitting coil unit <NUM> comprises a set of the transmitting coil <NUM>1X oriented in the X-axis direction, the transmitting coil <NUM>1Y oriented in the Y-axis direction, and the transmitting coil <NUM>1Z oriented in the Z-axis direction, and a set of the transmitting coil <NUM>2X oriented in the X-axis direction, the transmitting coil <NUM>2Y oriented in the Y-axis direction, and the transmitting coil <NUM>2Z oriented in the Z-axis direction. In addition, the transmitting coil unit <NUM> comprises a set of the transmitting coil <NUM>3X oriented in the X-axis direction, the transmitting coil <NUM>3Y oriented in the Y-axis direction, and the transmitting coil <NUM>3Z oriented in the Z-axis direction, and a set of the transmitting coil <NUM>4X oriented in the X-axis direction, the transmitting coil <NUM>4Y oriented in the Y-axis direction, and the transmitting coil <NUM>4Z oriented in the Z-axis direction. As described above, the transmitting coil unit <NUM> of the present embodiment is equivalent to a state where four triaxial coils are provided as the transmitting coils <NUM>.

In addition, the transmission control unit <NUM> comprises a transmission controller <NUM> and a transmitting circuit <NUM> connected to the transmitting coil <NUM>, specifically, transmitting circuits <NUM>1X, <NUM>1Y, <NUM>3Z, <NUM>2X, <NUM>2Y, <NUM>2Z, <NUM>3X, <NUM>3Y, <NUM>3Z, <NUM>4X, <NUM>4Y, and <NUM>4Z. In the present embodiment, similarly to the transmitting coil <NUM>, in the case where the transmitting circuits <NUM> are collectively referred to, the transmitting circuits <NUM> are simply referred to as the "transmitting circuit <NUM>", and in the case of distinguishing each, reference numerals (1X,. , 4Z) representing each are added after the "transmitting circuit <NUM>".

The transmitting circuit <NUM> generates a drive signal for driving the transmitting coil <NUM> under the control of the transmission controller <NUM>, and outputs the drive signal to the transmitting coil <NUM> connected thereto. A drive signal is applied to each transmitting coil <NUM> such that the transmitting coil <NUM> radiates an electromagnetic wave accompanied by a magnetic field to the surroundings. The transmission controller <NUM> of the present embodiment causes each transmitting circuit <NUM> to generate a drive signal and drive each transmitting coil <NUM> sequentially at a predetermined time interval, for example, at an interval of several tens of milliseconds.

On the other hand, as shown in <FIG>, in the position detection device <NUM>, the reception unit <NUM> provided inside the endoscope <NUM> comprises a reception controller <NUM>, a receiving coil unit <NUM>, receiving circuits <NUM> (<NUM><NUM> to <NUM><NUM>), analog-to-digital converters (ADCs) <NUM> (<NUM><NUM> to <NUM><NUM>), and an interface (I/F) <NUM>. The reception controller <NUM> controls the entire reception unit <NUM> and controls the driving of the receiving coil unit <NUM>.

As shown in <FIG>, the receiving coil unit <NUM> comprises, for example, <NUM> (six shown in <FIG>) receiving coils <NUM>, specifically, receiving coils <NUM><NUM> to <NUM><NUM>. In the present embodiment, similarly to the transmitting coil <NUM>, in the case where the receiving coils <NUM>, the receiving circuits <NUM>, and the ADCs <NUM> are collectively referred to, the receiving coils <NUM>, the receiving circuits <NUM>, and the ADCs <NUM> are simply referred to as the "receiving coil <NUM>", the "receiving circuit <NUM>", and the "ADC <NUM>", respectively, and in the case of distinguishing each, reference numerals (<NUM>,. , <NUM>) representing each are added after the "receiving coil <NUM>", the "receiving circuit <NUM>", and the "ADC <NUM>".

Each of the receiving coils <NUM> of the receiving coil unit <NUM> is arranged in the insertion part 10A of the endoscope <NUM> along the direction of insertion into the subject W. The receiving coil <NUM> detects a magnetic field generated by each transmitting coil <NUM> of the transmitting coil unit <NUM>. Each receiving coil <NUM> is connected to the receiving circuit <NUM>, and outputs a detection signal corresponding to the detected magnetic field to the receiving circuit <NUM>. The receiving circuit <NUM> includes a low pass filter (LPF), an amplifier (both not shown), and the like, the disturbance noise is removed by the LPF, and the detection signal amplified by the amplifier is output to the ADC <NUM>. The ADC <NUM> converts the input analog detection signal into a digital detection signal and outputs the digital detection signal to the reception controller <NUM>. The reception controller <NUM> transmits the detection signal input from each ADC <NUM> to the endoscopic examination device <NUM> via the I/F <NUM>.

The detection signal input to the endoscopic examination device <NUM> is input to the image processing unit <NUM> via an I/F <NUM>.

The image processing unit <NUM> detects the position of each receiving coil <NUM> based on the input detection signal. That is, the image processing unit <NUM> of the present embodiment detects the magnetic field generated by each transmitting coil <NUM> with the receiving coil <NUM>, and detects the position and direction (orientation) of each receiving coil <NUM> based on the detection signal output from the receiving coil <NUM>. The method by which the image processing unit <NUM> detects the position of the receiving coil <NUM> based on the detection signal is not particularly limited, and, for example, a technique disclosed in <CIT> can be applied thereto. In the technique disclosed in <CIT>, from a measured value of the magnetic field generated by each transmitting coil <NUM> and an estimated value in the direction of the receiving coil <NUM>, an estimated value of the distance from the specific transmitting coil <NUM> to the receiving coil <NUM> is calculated. Next, an estimated value of the position of the receiving coil <NUM> is calculated from the estimated value of the distance from each transmitting coil <NUM> to the receiving coil <NUM> and the known position of the transmitting coil <NUM>. Subsequently, a new estimated value in the direction of the receiving coil <NUM> is calculated from the estimated position of the receiving coil <NUM> and a measured value of the magnetic field of the receiving coil <NUM>. Then, the position and direction of the receiving coil <NUM> are derived by repeating the calculation of the estimated value of the distance from the transmitting coil <NUM> to the receiving coil <NUM> and the calculation of the estimated value of the position of the receiving coil <NUM> described above using the new estimated value in the direction of the receiving coil <NUM>.

Further, the image processing unit <NUM> of the present embodiment detects the shape of the insertion part 10A of the endoscope <NUM> based on the detected position and direction of each receiving coil <NUM>. <FIG> is a functional block diagram showing an example of the image processing unit <NUM> of the present embodiment.

As shown in <FIG>, the image processing unit <NUM> of the present embodiment comprises a detection unit <NUM>, an acquisition unit <NUM>, a generation unit <NUM>, a combination unit <NUM>, and a display controller <NUM>. The detection signal is input to the detection unit <NUM> from the position detection device <NUM>, specifically, from the reception controller <NUM> of the endoscope <NUM> via the I/F <NUM> and the I/F <NUM>. The detection unit <NUM> detects the position and direction of each receiving coil <NUM> based on the input detection signal. Further, the detection unit <NUM> detects the shape of the insertion part 10A based on the detected position and direction of the receiving coil <NUM>, and outputs information indicating the detected shape and the position and direction of each receiving coil <NUM> to the acquisition unit <NUM>. The acquisition unit <NUM> acquires information indicating the shape of the insertion part 10A and the position and direction of each receiving coil <NUM> from the detection unit <NUM>, and outputs the information to the generation unit <NUM>.

The generation unit <NUM> generates a first shape image (details described later) representing the shape of the insertion part 10A in a first viewpoint direction (details described later) based on the position and direction of each receiving coil <NUM>, and outputs image data of the first shape image to the combination unit <NUM>. In addition, based on the information indicating the shape of the insertion part 10A, the generation unit <NUM> further generates a second shape image (details described later) including a predetermined shape (details described later) of the insertion part 10A in a second viewpoint direction (details described later) different from the first viewpoint direction in a case where the shape of the insertion part 10A includes the predetermined shape, and outputs image data of the second shape image to the combination unit <NUM>.

Image data of an endoscopic image is input from the video processor <NUM> to the combination unit <NUM>. In addition, the image data of the first shape image, or the image data of the first shape image and the image data of the second shape image are input from the generation unit <NUM>. The combination unit <NUM> generates image data of a combined image obtained by combining at least one of the image data of the first shape image or the image data of the second shape image, which is input, with the image data of the endoscopic image, and outputs the generated image data to the display controller <NUM>.

The display controller <NUM> controls the display unit <NUM> to display an image represented by the image data output from the combination unit <NUM>. That is, in a case where the shape of the insertion part 10A does not include the predetermined shape, the display controller <NUM> of the present embodiment causes the display unit <NUM> to display the first shape image representing the shape of the insertion part 10A in the first viewpoint direction. Further, in a case where the shape of the insertion part 10A includes the predetermined shape, the display controller <NUM> controls the display unit <NUM> to display the first shape image and the second shape image.

As an example, the image processing unit <NUM> of the present embodiment is realized by a microcomputer or the like including the hardware shown in <FIG>. As shown in <FIG>, the image processing unit <NUM> comprises a central processing unit (CPU) <NUM>, a read only memory (ROM) <NUM>, a random access memory (RAM) <NUM>, and a nonvolatile storage unit <NUM> such as a hard disk drive (HDD), a solid state drive (SSD), and a flash memory. The CPU <NUM>, the ROM <NUM>, the RAM <NUM>, and the storage unit <NUM> are connected to a bus <NUM> such that they can communicate with each other. The storage unit <NUM> stores an image processing program <NUM> for performing image processing whose details will be described later. The CPU <NUM> reads out the image processing program <NUM> from the storage unit <NUM>, loads the read-out program in the RAM <NUM>, and executes the loaded image processing program <NUM>. The CPU <NUM> executes the image processing program <NUM>, and thus, the image processing unit <NUM> functions as each of the detection unit <NUM>, the acquisition unit <NUM>, the generation unit <NUM>, the combination unit <NUM>, and the display controller <NUM>.

In addition, in the endoscope system <NUM> of the present embodiment, the reception controller <NUM>, the overall controller <NUM>, and the transmission controller <NUM> are also realized by the same hardware as the image processing unit <NUM> (see <FIG>).

Next, the operation of the image processing unit <NUM> of the present embodiment will be described. <FIG> is a flowchart showing an example of a flow of image processing performed by the CPU <NUM> of the image processing unit <NUM>. As an example, in the endoscope system <NUM> of the present embodiment, in a case where an instruction to start the endoscopic examination is received from an examiner by operating an operation button or the like (not shown), the CPU <NUM> executes the image processing program <NUM>, and thus the image processing shown in <FIG> is performed.

In step S100, the detection unit <NUM> analyzes the shape of the insertion part 10A. In the present embodiment, the detection unit <NUM> first detects the position and direction of each receiving coil <NUM> based on the detection signal input from the position detection device <NUM> as described above. Then, the shape of the insertion part 10A is analyzed based on the detected position and direction of the receiving coil <NUM>, and in the shape of the insertion part 10A, a loop shape is detected as an example of a predetermined shape. In the present embodiment, so-called α loop shape and inverse α loop shape are detected as an example of the loop shape. The method by which the detection unit <NUM> detects the loop shape is not particularly limited, and, for example, a technique disclosed in <CIT> or a technique disclosed in <CIT> can be applied thereto.

The acquisition unit <NUM> acquires information indicating the shape of the insertion part 10A and the position and direction of each receiving coil <NUM> from the detection unit <NUM>, and outputs the information to the generation unit <NUM>.

In the next step S102, the generation unit <NUM> determines whether or not the detection unit <NUM> has detected a loop shape, in other words, whether or not the shape of the insertion part 10A includes a loop shape. In a case where the loop shape is not detected, the determination in step S102 is a negative determination, and the process proceeds to step S104. The generation unit <NUM> of the present embodiment is an example of a determination unit of the present disclosure. The generation unit <NUM> can acquire the predetermined shape from the storage unit <NUM> that stores the predetermined shape such as the loop shape, and determine whether or not the shape detected by the detection unit <NUM> includes the predetermined shape. Further, the detection unit <NUM> may be an example of the determination unit. In this case, the detection unit <NUM> may acquire the predetermined shape from the storage unit <NUM> and detect the predetermined shape from the shape of the insertion part 10A to determine whether or not the shape of the insertion part 10A includes the predetermined shape.

In step S104, as described above, the generation unit <NUM> generates the first shape image representing the shape of the insertion part 10A in the first viewpoint direction based on the position and direction of each receiving coil <NUM>, and outputs image data of the first shape image to the combination unit <NUM>. As an example, the first viewpoint direction in the present embodiment is a predetermined direction as a viewpoint direction for grasping the overall shape of the insertion part 10A. A specific example of such a first viewpoint direction includes the X-axis direction in the transmitting coil unit <NUM> shown in <FIG>, and in the present embodiment, a direction in which the examiner looks at the subject W in front (a direction in which the face-side surface is visually recognized) can be included. In this case, first shape images 90A and 90B representing the shape of the insertion part 10A in the first viewpoint direction are two-dimensional images represented by the Y-axis and the Z-axis, as shown in <FIG>. The first shape image 90A shown in <FIG> is the first shape image in the case where the loop shape is not included, and the first shape image 90B shown in <FIG> is the first shape image in the case where the loop shape is included. In the following, in a case where the first shape image 90A and the first shape image 90B are collectively referred to, the first shape image 90A and the first shape image 90B are simply referred to as the "first shape image <NUM>", and in the case of distinguishing each, reference numerals (A, B) representing each are added after the "first shape image <NUM>". In addition, another specific example of the first viewpoint direction includes a direction in which it is possible to visually recognize that the insertion part is annular (forms a closed curve) in the case where the insertion part has a loop shape.

In the next step S106, the combination unit <NUM> generates an image in which the first shape image 90A and the endoscopic image are combined, and outputs image data of the combined image including the first shape image 90A and the endoscopic image to the display controller <NUM>. In the next step S114, the display controller <NUM> causes the display unit <NUM> to display the combined image. <FIG> shows an example of a combined image <NUM> displayed on the display unit <NUM>. The combined image <NUM> shown in <FIG> includes the first shape image 90A and the endoscopic image <NUM>. Note that, in the present embodiment, an image combined in a state where the first shape image 90A and the endoscopic image <NUM> are arranged side by side is used as the combined image <NUM>. However, the method of combining the first shape image 90A and the endoscopic image <NUM> is not limited to the present embodiment. The degree of superimposition of the first shape image 90A and the endoscopic image <NUM> in the combined image <NUM>, the size of each image, and the like may be in a state where the image desired by the examiner is appropriately displayed in the endoscopic examination. For example, the combined image <NUM> may be an image combined in a state where the first shape image 90A and the endoscopic image <NUM> are at least partially superimposed. Further, the combined image <NUM> may be in a form in which the degree of superimposition of the first shape image 90A and the endoscopic image <NUM> and the size of each image are controlled according to the size of the display unit <NUM> and the like.

On the other hand, in a case where the loop shape is detected, the determination in step S102 is a positive determination, and the process proceeds to step S108. In step S108, the generation unit <NUM> generates the first shape image 90B in the same manner as the generation of the first shape image 90A in step S104.

In the next step S110, the generation unit <NUM> generates the second shape image representing the shape of the insertion part 10A in the second viewpoint direction based on the position and direction of each receiving coil <NUM>. As an example, the second viewpoint direction in the present embodiment is a predetermined direction as a viewpoint direction for grasping the intersecting state of the loop-shaped portion. As a specific example of such a direction, in the present embodiment, as shown in <FIG>, a direction of connection from a loop-shaped intersection point <NUM> to a loop-shaped center point <NUM> is defined as a second viewpoint direction <NUM>. In this case, a second shape image <NUM> representing the shape of the insertion part 10A in the second viewpoint direction <NUM> is a two-dimensional image represented by the Y-axis and the X-axis, as shown in <FIG>. By setting the second viewpoint direction to the second viewpoint direction <NUM> in this way, it is possible to easily grasp the shape (state) of the loop shape in the depth direction.

As an example, in the present embodiment, the second shape image <NUM> is not an image representing the overall shape of the insertion part 10A, but is an image mainly representing only the loop-shaped portion as shown in <FIG>. Note that, an image is not limited to the second shape image <NUM> shown in <FIG>, and may be an image representing the overall shape of the insertion part 10A similarly to the first shape image <NUM>. However, by using an image mainly representing only the loop-shaped portion as in the present embodiment, it is easy to grasp the state of the loop shape, particularly the intersecting portion of the loop shape. In addition, the second shape image <NUM> of the present embodiment is set as an image in which the size of the loop shape is larger than the size of the loop shape in the first shape image <NUM>. For example, the size of the loop-shaped image in the second shape image <NUM> is set to be four times the size of the loop-shaped image in the first shape image <NUM> (each side of the rectangle is twice). In this way, by setting the second shape image <NUM> as an image mainly representing only the loop shape, it is possible to use the image in a state where the loop shape can be easily grasped while suppressing an increase in the overall size of the second shape image <NUM>.

The image data of the first shape image 90B and the image data of the second shape image <NUM> which are generated in this way are output to the combination unit <NUM>.

In the next step S <NUM>, the combination unit <NUM> generates an image in which the first shape image 90B, the second shape image <NUM>, and the endoscopic image are combined, and outputs image data of the combined image <NUM> including the first shape image 90B, the second shape image <NUM>, and the endoscopic image to the display controller <NUM>. In the next step S <NUM>, the display controller <NUM> causes the display unit <NUM> to display the combined image.

<FIG> shows an example of the combined image <NUM> displayed on the display unit <NUM>. The combined image <NUM> shown in <FIG> includes the first shape image 90B, the second shape image <NUM>, and the endoscopic image <NUM>. Note that, in the present embodiment, an image which is combined in a state where the second shape image <NUM> is superimposed on a partial region in the first shape image 90B, and then, further combined in a state where the combined image and the endoscopic image <NUM> are arranged side by side is used as the combined image <NUM>. However, the method of combining the first shape image 90B, the second shape image <NUM>, and the endoscopic image <NUM> is not limited to the present embodiment. The degree of superimposition of the first shape image 90B, the second shape image <NUM>, and the endoscopic image <NUM> in the combined image <NUM>, the size of each image, and the like may be in a state where the image desired by the examiner is appropriately displayed in the endoscopic examination. For example, the combined image <NUM> may be an image combined in a state where the first shape image 90B and the second shape image <NUM> are arranged side by side. Further, for example, the combined image <NUM> may be an image combined in a state where the first shape image 90B, the second shape image <NUM>, and the endoscopic image <NUM> are at least partially superimposed. Further, the combined image <NUM> may be in a form in which the degree of superimposition of the first shape image 90B, the second shape image <NUM>, and the endoscopic image <NUM> and the size of each image are controlled according to the size of the display unit <NUM> and the like.

Note that, as shown in <FIG>, it is preferable that the generation unit <NUM> also generate viewpoint information <NUM> as information that can be visually recognized in the second viewpoint direction <NUM>, and the display controller <NUM> cause the display unit <NUM> to display the viewpoint information <NUM>. Regarding the display of the viewpoint information <NUM>, the display position, the shape, the size, and the like are not particularly limited. However, as in the example shown in <FIG>, the viewpoint direction can be more easily grasped by superimposing the viewpoint information <NUM> having the shape of an arrow indicating the viewpoint direction on the first shape image 90B and displaying it.

Further, the generation unit <NUM> may generate, as the second shape image <NUM>, the second shape image <NUM> in which the second viewpoint direction is changed for each changed second viewpoint direction, and the display controller <NUM> may cause the display unit <NUM> to display the generated second shape image <NUM>. In addition, for example, the second shape image <NUM> may be the second shape image <NUM> in a state where the loop shape is rotated. In this case, the generation unit <NUM> may set the rotation speed per rotation to <NUM> seconds to <NUM> seconds every predetermined time, for example, clockwise or counterclockwise and generate a predetermined number of second shape images <NUM> such as <NUM> images to <NUM> images per rotation for each rotation position, and the display controller <NUM> may cause the display unit <NUM> to display the second shape image <NUM>. <FIG> shows an example of the combined image <NUM> displayed on the display unit <NUM> in this case. In this case, the second shape image <NUM> displayed on the display unit <NUM> changes with time, for example, from the second shape image <NUM> shown in <FIG> to the second shape image <NUM> shown in <FIG>. Note that, the storage unit <NUM> may store rotation display information such as the orientation of rotation (clockwise or counterclockwise), the rotation speed (time required for one rotation), and the number of images per rotation of the second shape image for each predetermined shape, the generation unit <NUM> may generate the second shape image <NUM> based on the rotation display information acquired from the storage unit <NUM>, and the display controller <NUM> may cause the display unit <NUM> to display the generated second shape image <NUM>.

In the next step S <NUM>, the display controller <NUM> determines whether to end the endoscopic examination. In the endoscope system <NUM> of the present embodiment, by operating an operation button or the like (not shown), the determination in step S <NUM> is a negative determination until an instruction to end the endoscopic examination is received from an examiner by operating an operation button or the like (not shown), the process returns to step S <NUM>, and the respective processes of steps S <NUM> to S <NUM> are repeated. On the other hand, in a case where the instruction to end the endoscopic examination is received, the determination in step S <NUM> is a positive determination, and the present image processing ends.

As described above, the image processing unit <NUM> of the present embodiment comprises the acquisition unit <NUM> and the display controller <NUM>. The acquisition unit <NUM> acquires the shape of the insertion part 10A to be inserted into the subject W in the endoscope <NUM>. In a case where the shape of the insertion part 10A does not include a loop shape that is a predetermined shape, the display controller <NUM> controls the display unit <NUM> to display the first shape image 90A representing the shape of the insertion part 10A in the first viewpoint direction. Further, in a case where the shape of the insertion part 10A includes the predetermined shape, the display controller <NUM> controls the display unit <NUM> to display the first shape image 90B and the second shape image <NUM> including a portion corresponding to the predetermined shape of the insertion part 10A in the second viewpoint direction different from the first viewpoint direction.

In this way, the image processing unit <NUM> of the present embodiment displays the first shape image 90B and the second shape image <NUM> in a case where the loop shape is included. Since the second shape image <NUM> is the shape of the insertion part 10A in the second viewpoint direction, which is different from the first viewpoint direction, it is possible to easily grasp the loop shape, particularly in the present embodiment, the overlapping (intersection) state of the loop. Further, according to the image processing unit <NUM> of the present embodiment, in a case where the loop shape is not included, since only the first shape image 90A is displayed, the shape of the insertion part 10A can be more easily grasped by the first shape image 90A.

Therefore, according to the image processing unit <NUM> of the present embodiment, it is possible to easily grasp the shape of the insertion part 10A of the endoscope <NUM>.

Further, according to the image processing unit <NUM> of the present embodiment, even in a case where the endoscopic image <NUM>, the first shape image 90B, and the second shape image <NUM> are displayed on the display unit <NUM>, the loop shape of the insertion part 10A can be easily grasped by the second shape image <NUM>. Moreover, according to the image processing unit <NUM> of the present embodiment, since the loop-shaped image in the second shape image <NUM> is larger than the loop-shaped image included in the first shape image 90B, the loop shape can be more easily grasped.

In the present embodiment, the loop shape, particularly the α loop shape and the inverse α loop shape are used as an example of the predetermined shape; however, the predetermined shape is not limited to these shapes, and the shape may be any shape that the examiner desires to check in detail the state of the shape of the insertion part 10A. For example, it may be another loop shape such as an N loop shape or a γ loop shape, or may be a shape according to individual settings of the examiner.

In addition, although the position detection device <NUM> of the present embodiment has the configuration in which the transmission unit <NUM> including the transmitting coil unit <NUM> that generates a magnetic field is arranged in the endoscopic examination device <NUM>, and the reception unit <NUM> including the receiving coil unit <NUM> that detects the magnetic field is arranged in the endoscope <NUM>, the position detection device <NUM> is not limited to the configuration of the present embodiment. For example, an element that generates a magnetic field other than the transmitting coil unit <NUM> (transmitting coil <NUM>) such as a spin torque oscillator may be used. Further, for example, an element that detects a magnetic field other than the receiving coil unit <NUM> (receiving coil <NUM>) such as a Hall element or a magneto resistive (MR) element may be used.

The position detection device <NUM> may have a configuration in which the reception unit <NUM> is arranged in the endoscopic examination device <NUM> and the transmission unit <NUM> is arranged in the endoscope <NUM>. Further, the position detection device <NUM> may be a device that detects the shape of the insertion part 10A using a magnetic field other than the magnetic field. For example, the position detection device <NUM> may be a device that uses an optical cable or the like in which a fiber bragg grating (FBG) is arranged and that detects the shape of the insertion part 10A by the FBG.

Further, in the present embodiment, the configuration in which the image processing unit <NUM> has the functions of the detection unit <NUM>, the acquisition unit <NUM>, the generation unit <NUM>, the combination unit <NUM>, and the display controller <NUM> has been described; however, some of these functions may be provided by another device or a plurality of devices. For example, one of the detection unit <NUM> and the generation unit <NUM> may be provided in a device external to the image processing unit <NUM>.

Further, in the present embodiment, the form in which the combination unit <NUM> generates the combined image <NUM> obtained by combining the first shape image <NUM> and the endoscopic image <NUM> has been described; however, the present disclosure is not limited to the present embodiment, and the first shape image <NUM> and the endoscopic image <NUM> may be displayed on the display unit <NUM> as separate images without being combined. In addition, each of the endoscopic image <NUM>, the first shape image <NUM>, and the second shape image <NUM> is not limited to the same display unit <NUM>, and may be displayed on different display devices, for example. Alternatively, the display device that displays the endoscopic image <NUM> and the display device that displays the first shape image <NUM> and the second shape image <NUM> may have different forms.

In the present embodiment, as hardware structures of processing units that execute various kinds of processing, such as the detection unit <NUM>, the acquisition unit <NUM>, the generation unit <NUM>, the combination unit <NUM>, and the display controller <NUM>, various processors shown below can be used.

As described above, the various processors include a programmable logic device (PLD) as a processor of which the circuit configuration can be changed after manufacture, such as a field programmable gate array (FPGA), a dedicated electrical circuit as a processor having a dedicated circuit configuration for executing specific processing such as an application specific integrated circuit (ASIC), and the like, in addition to the CPU as a general-purpose processor that functions as various processing units by executing software (program).

One processing unit may be configured by one of the various processors, or configured by a combination of the same or different kinds of two or more processors (for example, a combination of a plurality of FPGAs or a combination of the CPU and the FPGA). In addition, a plurality of processing units may be configured by one processor.

As an example where a plurality of processing units are configured by one processor, first, there is a form in which one processor is configured by a combination of one or more CPUs and software as typified by a computer, such as a client or a server, and this processor functions as a plurality of processing units. Second, there is a form in which a processor for realizing the function of the entire system including a plurality of processing units by one integrated circuit (IC) chip as typified by a system on chip (SoC) or the like is used. As described above, various processing units are configured by using one or more of the above-described various processors as hardware structures.

Furthermore, as the hardware structure of the various processors, more specifically, an electrical circuit (circuitry) in which circuit elements such as semiconductor elements are combined can be used.

Claim 1:
An image processing device comprising:
an acquisition unit (<NUM>) that acquires a shape of an insertion part (10A) to be inserted into a subject in an endoscope (<NUM>);
a storage unit (<NUM>) in which a predetermined shape is stored;
a determination unit that determines whether or not the shape of the insertion part (10A) includes the predetermined shape stored in the storage unit (<NUM>); and characterised by
a display controller (<NUM>) that controls a display unit (<NUM>) to display a first shape image in a case where the determination unit determines that the shape of the insertion part (10A) does not include the predetermined shape, and controls the display unit (<NUM>) to display the first shape image and a second shape image in a case where the determination unit determines that the shape of the insertion part (10A) includes the predetermined shape, the first shape image representing the shape of the insertion part (10A) in a first viewpoint direction, the second shape image including a portion corresponding to the predetermined shape of the insertion part (10A) in a second viewpoint direction different from the first viewpoint direction.