ENDOSCOPE SYSTEM

Provided is an endoscope system wherein, when a trigger signal generating unit of an endoscope (10) generates a zoom-in or zoom-out trigger signal, a control device (40) identifies the fixation point of an operator in a display image of a display device (30) on the basis of a relationship between a detection signal from a sensor in polarized glasses (50) and a detection signal from a sensor in the display device (30) at the time of generation of the trigger signal, and zooms in or zooms out the peripheries of the fixation point.

BACKGROUND

Technical Field

The invention relates to an endoscope system using an 8K high-resolution endoscope.

Description of Related Art

Various techniques, related to a flexible endoscope for inserting an elongated insertion part into a body cavity and photographing the inside of the body cavity to perform minimally invasive surgery, have been proposed. Patent Document 1 is a document disclosing an invention related to this type of endoscope.

RELATED ART

Patent Document

SUMMARY

Problems to be Solved

With the development of image processing technology and optical technology, high-resolution imaging technologies called 4K and 8K have been put to practical use. The evolution of imaging technologies from 2K to 4K to 8K is also causing technological innovation in the field of medical equipment using endoscope and the field of minimally invasive surgery. When the 8K high-resolution imaging technology is applied to endoscope, for example, it becomes easy to recognize a fine thread for surgery, a fine affected part of an organ, and a boundary between organs and tissues, and it is also possible to observe at the cell level. As a result, the reliability and certainty of surgery are improved, and it is expected to see further progress in medical technology. That is, the discriminability of the affected part of an organ is enhanced, and the possibility of unexpectedly damaging parts other than the affected part is reduced. In addition, the field of view for surgery can be expanded, and surgery can be easily performed even when the surgery range is wide. It is also convenient for confirming the position of surgical equipment and avoiding interference between surgical equipment. Furthermore, it is possible for observation through a large screen so that all members involved in the surgery can share the same image to achieve smooth communication. Thus, the use of 4K and 8K high-resolution imaging technologies has great potential.

However, the conventional high-resolution endoscope system has room for improvement in terms of specifying the zoom position in the display image of the endoscope.

In view of such problems, the invention further improves the convenience of an endoscope system including an endoscope.

Means for Solving the Problems

In order to solve the above problems, the invention provides an endoscope system that includes: an endoscope photographing a subject in a body cavity of a patient and outputting an image signal of a predetermined number of pixels; a control device performing a predetermined 3D process on an output signal of the endoscope and outputting a 3D image signal obtained by the 3D process to a display device as a moving image signal of a predetermined frame rate; polarized glasses worn by an operator performing a surgical operation on the patient; sensors provided in the display device and the polarized glasses respectively; and a trigger signal generating unit generating a trigger signal for instructing zoom of a display image of the display device. When the trigger signal generating unit generates the trigger signal, the control device identifies a fixation point of the operator in the display image of the display device based on a relationship between a detection signal of the sensor in the polarized glasses and a detection signal of the sensor in the display device at a time of generation of the trigger signal, and zooms in a periphery of the fixation point.

In the endoscope system, the sensors provided in the polarized glasses include a first sensor detecting a potential of a left nose pad of the polarized glasses, a second sensor detecting a potential of a right nose pad of the polarized glasses, a third sensor detecting a potential of a bridge of the polarized glasses, a fourth sensor detecting a position of the polarized glasses, and a fifth sensor detecting an orientation of lenses of the polarized glasses. The control device obtains a line-of-sight position corresponding to a line of sight of the operator on the lenses of the polarized glasses based on a potential waveform indicated by a detection signal of the first sensor, a potential waveform indicated by a detection signal of the second sensor, and a potential waveform indicated by a detection signal of the third sensor, and identifies the fixation point of the operator in the display image based on the line-of-sight position, a relationship between a position where the display device is placed and a position detected by the fourth sensor, and a relationship between an orientation of the display device and an orientation detected by the fifth sensor.

In the endoscope system, the endoscope is an 8K endoscope, including: a housing; a solid-state imaging element housed in the housing and including pixels, a number of which corresponds to 8K and which each include a photoelectric conversion element, arranged in a matrix; and an insertion part extending with the housing as a base end, and the insertion part being inserted into the body cavity of the patient and guiding light from the subject in the body cavity to the solid-state imaging element. A pitch between adjacent pixels in the solid-state imaging element may be larger than a longest wavelength of wavelengths of light in illumination that illuminates the subject.

Further, the housing includes a mount part having a large cross-sectional area orthogonal to an optical axis of light passing through the insertion part, and a grip part having a smaller cross-sectional area than the mount part, and the solid-state imaging element may be housed in the mount part.

In addition, the insertion part includes a hollow rigid lens barrel, and a plurality of lenses including an objective lens may be provided in the rigid lens barrel.

Further, the endoscope system includes: an air supply pipe and an air exhaust pipe connected to the housing; an air supply and exhaust device forcibly supplying air into the housing via the air supply pipe and forcibly exhausting air from the housing via the air exhaust pipe; and an air cooling device cooling air flowing through the air supply pipe. The housing, the air supply pipe, and the air exhaust pipe are connected to form one closed space. In the housing, a first heat sink provided on the solid-state imaging element; a FPGA for image processing, a second heat sink provided on the FPGA, and a cover member covering the second heat sink and connected to the air exhaust pipe are provided. In the housing, a first airflow for cooling the first heat sink and a second airflow for cooling the second heat sink are generated. The first airflow may be formed by blowing cooling air supplied from the air supply pipe to the first heat sink to diverge around the first heat sink, and the second airflow may be formed to flow from around the second heat sink to the air exhaust pipe via the cover member.

In addition, the endoscope includes: a housing; a solid-state imaging element housed in the housing and including pixels, which each include a photoelectric conversion element, arranged in a matrix; and a hollow flexible lens barrel. In the flexible lens barrel, an objective lens, a multi-core fiber, and one or more mirrors for reflecting light from the subject one or more times and guiding the light to the objective lens are provided. At least one mirror of the one or more mirrors is tiltable around two axes, a first axis having a tilt with respect to an optical axis direction of light passing through each core of the multi-core fiber, and a second axis orthogonal to the first axis. The control device generates divided area images of different portions of the subject by periodically switching a tilt angle of the mirror at a time interval shorter than a frame switching time interval of the frame rate, and generates a moving image for one frame by combining the generated divided area images.

DESCRIPTION OF THE EMBODIMENTS

First Embodiment

FIG. 1is a diagram showing a configuration of an endoscope system including an endoscope according to the first embodiment of the invention. The endoscope system includes a rigid endoscope10, an illumination device20, a display device30, a control device40, polarized glasses50, an air intake and exhaust device60, and an air cooling device70.

FIG. 2is a diagram showing single-incision laparoscopic surgery, which is an example of a surgical operation performed with assistance of the endoscope system of the present embodiment. In single-incision laparoscopic surgery, an incision of about 2 cm is made in the abdomen of the patient (in most cases, the position of the navel) and a port (frame made of resin) is mounted to the incision, and carbon dioxide gas is introduced to expand the body cavity. The operator inserts an endoscope, a scalpel, and forceps (a surgical tool for pinching and pulling) into the body cavity from the port and treats the affected part in the body cavity while observing the image photographed by the endoscope, which is displayed on the display device30.

In the present embodiment, the vein of the patient is injected with a near-infrared excitation drug prior to surgery. When this drug spreads over the body of the patient, the blood vessel illuminated by excitation light of a certain wavelength emits near-infrared light. In addition, the operator wears the polarized glasses50for stereoscopically viewing a 3D image. Then, as shown inFIG. 3, the control device40acquires from the rigid endoscope10a visible light image of the affected part in the body cavity of the patient and an infrared light image from the capillary in the depth, and displays these images as 3D moving images on the display device30. Further, the control device40identifies a fixation point FP ahead of a line of sight of the operator in the display image of the display device30based on a relationship between a detection signal of a sensor in the polarized glasses50and a detection signal of a sensor in the display device30, and zooms in the periphery of the fixation point FP.

InFIG. 1, the rigid endoscope10is a device that serves to photograph the body cavity of the patient. The rigid endoscope10has a camera body130, an insertion part110, and an eyepiece mount part120. A housing131of the camera body130has a shape that the thickness of the cross section of the front part of the cylindrical body is increased.

FIG. 4is a diagram of the housing131ofFIG. 1when viewed from the direction of the arrow A. The housing131includes a mount part1131on the front side and a grip part1132on the rear side. A perfect circular opening is provided on the front surface of the mount part1131. The area of the cross section of the mount part1131of the housing131is larger than the area of the grip part1132of the housing131. An annular frame is fitted into the opening on the front surface of the mount part1131of the housing131. At the position, facing the frame, on the front surface in the housing131, a solid-state imaging element1311and an A/D conversion unit1319are provided. The solid-state imaging element1311is a CMOS (Complementary Metal Oxide Semiconductor) image sensor. The solid-state imaging element1311has pixels PXij(i=1 to 4320, j=1 to 7680), the number of which corresponds to 8K, arranged in a matrix. Here, i is the index of the pixel row, and j is the index of the pixel column. Each of the pixels PXij(i=1 to 4320, j=1 to 7680) of the solid-state imaging element1311has a photoelectric conversion element EL and an amplifier AMP that amplifies signal charge obtained by photoelectric conversion of the photoelectric conversion element EL.

InFIG. 4, the pixels PXij(i=1 to 4320, j=1 to 7680) in 4320 rows and 7680 columns in the solid-state imaging element1311form blocks each having four pixels PXijin 2 rows and 2 columns. Red, green, blue, and near-infrared filters are affixed to the four pixels PXijof each block. Specifically, the filter of the pixel PXijat the upper left of the block is a red filter (a filter that transmits only the red wavelength). The filter of the pixel PXijat the lower left of the block is a green filter (a filter that transmits only the green wavelength). The filter of the pixel PXijat the upper right of the block is a blue filter (a filter that transmits only the blue wavelength). The filter of the pixel PXijat the lower right of the block is a near-infrared filter (a filter that transmits only the near-infrared wavelength).

In addition, a pitch between adjacent pixels PXijin the solid-state imaging element1311(more specifically, a distance D between the centers of the light receiving areas of the photoelectric conversion elements EL of adjacent pixels PXij) is larger than the longest wavelength of the wavelengths of the light that illuminates the subject. If the light illuminating the subject contains only visible light, the pitch of the pixels PXijis preferably 2.8 μm to 3.8 μm. If the light illuminating the subject contains visible light and near-infrared light, the pitch of the pixels PXijis preferably 3.8 μm or more.

The A/D conversion unit1319dot-sequentially A/D converts the signal charges of the pixels PXij(i=1 to 4320, j=1 to 7680) that have been amplified by the amplifier AMP, and outputs the data obtained by the A/D conversion as an image signal SD. In the housing131, in addition to the solid-state imaging element1311and the A/D conversion unit1319, a heat sink1316, a duct1318, an image processing FPGA1331, a heat sink1336, a cover member1338, a duct166B, etc. are housed (see (A) and (B) ofFIG. 14). Details of these parts will be described later.

Buttons1391N and139OUT are provided on the rear portion of a side surface of the housing131. The buttons1391N and139OUT serve as a trigger generating unit. When the button139IN is pressed shortly once, a zoom-in trigger signal that instructs zoom-in of the image displayed on the display device30is transmitted from the rigid endoscope10to the control device40. When the button139OUT is pressed shortly once, a zoom-out trigger signal that instructs zoom-out of the image displayed on the display device30is transmitted from the rigid endoscope10to the control device40.

InFIG. 1, the eyepiece mount part120has a shape that a part of the outer periphery of the cylindrical body is recessed inward. An eyepiece1201is fitted into the eyepiece mount part120. The insertion part110is a part inserted into the body cavity of the patient. The insertion part110has a rigid lens barrel111and an eyepiece112. A connector113for the illumination device is provided at a position on the outer periphery of the insertion part110on the tip side of the eyepiece112.

FIG. 5is a cross-sectional diagram of the insertion part110ofFIG. 1taken along the line B-B′. As shown inFIG. 5, in the insertion part110, a hollow light guide area1112is provided. The hollow light guide area1112is a cavity having a diameter slightly smaller than the diameter of the insertion part110. Hundreds to thousands of optical fibers1901are embedded in the outer shell surrounding the hollow light guide area1112in the insertion part110.FIG. 5shows only 16 optical fibers1901for simplicity. A diffusion lens (not shown) is provided in front of the tips of the optical fibers in the insertion part110. An objective lens1111is fitted at a position slightly inward of the tip in the hollow light guide area1112of the insertion part110. A relay lens1113is fitted between the objective lens1111in the hollow light guide area1112and the eyepiece112. The eyepiece112of the insertion part110is connected to the eyepiece mount part120. The eyepiece mount part120is connected to the frame on the front surface of the housing131.

InFIG. 1, the illumination device20includes a light-emitting element that emits light having a wavelength of visible light and a wavelength of near-infrared light, and a driver circuit that drives the light-emitting element. The illumination device20is connected to the connector113of the insertion part110. Illumination light (light having wavelengths of visible light and near-infrared light) of the illumination device20passes through the optical fibers1901of the insertion part110and is emitted into the body cavity via the diffusion lens ahead of it.

InFIG. 1, the display device30is a liquid crystal display having display pixels corresponding to 8K (display pixels of 4320 rows and 7680 columns). The display device30is provided with a position detection sensor36and an orientation detection sensor37. The position detection sensor36detects the position of the display device30. Specifically, the position detection sensor36outputs coordinate signals SDX, SDY, and SDZindicating the position of the display device30on the X axis, the position of the display device30on the Y axis, and the position of the display device30on the Z axis when setting the direction parallel to the earth axis direction as the Z-axis direction, one direction orthogonal to the Z-axis direction as the Y-axis direction, a direction orthogonal to both the Z-axis direction and the Y-axis direction as the X-axis direction, and a reference point in the room for surgery (for example, the center of the room) as the origin (0.0.0).

The orientation detection sensor37detects the orientation of the display device30. Specifically, the orientation detection sensor37sets a plane parallel to the X-axis direction and the Y-axis direction as the reference plane, sets a tilt of the display screen of the display device30in a direction around the X axis with respect to the reference plane as an elevation angle of the display screen, and outputs an angle signal SDθXindicating the elevation angle of the display screen. Further, the orientation detection sensor37sets a tilt of the display screen of the display device30in a direction around the Z axis with respect to the reference plane as the direction of the display screen, and outputs an angle signal SDθZindicating the direction of the display screen.

InFIG. 1, the polarized glasses50are passive (circularly polarized filter type) polarized glasses. As shown inFIG. 6, a left lens55L and a right lens55R are fitted into a frame54of the polarized glasses50. The left lens55L guides a left-eye image of the 3D image to the left-eye retina of the operator. The right lens55R guides a right-eye image of the 3D image to the right-eye retina of the operator.

A position detection sensor56is embedded at the upper left of the left lens55L in the frame54of the polarized glasses50. The position detection sensor56detects the position of the polarized glasses50. More specifically, the position detection sensor56outputs coordinate signals SGX, SGY, and SGZindicating the positions of the polarized glasses50on the X axis, the Y axis, and the Z axis when setting the reference point (the center of the room) as the origin (0.0.0).

An orientation detection sensor57is embedded at the upper right of the right lens55R in the frame54of the polarized glasses50. The orientation detection sensor57detects the orientations of the lenses55L and55R of the polarized glasses50. Specifically, the orientation detection sensor57sets a tilt of the lenses55L and55R of the polarized glasses50in the direction around the X axis with respect to the reference plane (a plane parallel to the X-axis direction and the Y-axis direction) as an elevation angle of the lenses55L and55, and outputs an angle signal SGθXindicating the elevation angle of the lenses55L and55R. In addition, the orientation detection sensor57sets a tilt in the direction around the Z axis with respect to the reference plane as the direction of the lenses55L and55R, and outputs an angle signal SGθZindicating the direction of the lenses55L and55R.

A first potential sensor51is embedded in the left nose pad of the frame54of the polarized glasses50. A second potential sensor52is embedded in the right nose pad of the frame54of the polarized glasses50. A third potential sensor53is embedded in the middle bridge of the frame54of the polarized glasses50. The potential sensor51detects a potential of a portion of the face of the operator with which the left nose pad is in contact, and outputs a left potential signal SGV1indicating the detected potential. The potential sensor52detects a potential of a portion of the face of the operator with which the right nose pad is in contact, and outputs a right potential signal SGV2indicating the detected potential. The potential sensor53detects a potential of a portion of the face of the operator with which the bridge is in contact, and outputs an upper potential signal SGV3indicating the detected potential.

A wireless communication unit58is embedded in the right temple of the frame54of the polarized glasses50. The wireless communication unit58modulates a carrier by the output signals SGX, SGY, and SGZof the position detection sensor56, the output signals SGθXand SGθZof the orientation detection sensor57, the output signal SGV1of the potential sensor51, the output signal SGV2of the potential sensor52, and the output signal SGV3of the potential sensor53, and transmits a radio signal SG′ obtained by the modulation.

InFIG. 1, the control device40is a device that serves as the control center of the endoscope system. As shown inFIG. 7, the control device40includes a wireless communication unit41, an operation unit42, an input/output interface43, an image processing unit44, a storage unit45, and a control unit46. The wireless communication unit41receives the radio signal SG′ and supplies signals SGX, SGY, SGZ, SGθX, SGθZ, SGV1, SGV2, and SGV3obtained by demodulating the signal SG′ to the control unit46.

The operation unit42is a device that performs various operations such as a keyboard, a mouse, a button, and a touch panel. The input/output interface43mediates transmission and reception of data between the display device30and the rigid endoscope10and the control device40. The image processing unit44is an image processor. The storage unit45has both a volatile memory such as a RAM (Random Access Memory) and a non-volatile memory such as an EEPROM (Electrically Erasable Programmable Read Only Memory). The storage unit45stores an operation program PRG of the control unit46or the image processing unit44. In addition, the storage unit45provides the control unit46and the image processing unit44with storage areas and work areas such as a reception buffer45S, a left-eye image buffer45L, a right-eye image buffer45R, a drawing frame buffer45D, and a display frame buffer45E.

The control unit46includes a CPU. The control unit46executes an illumination driving process, an imaging element driving process, a display control process, and a zoom control process by running of the operation program PRG in the storage unit45. The illumination driving process is a process of supplying a drive signal for driving a driver in the illumination device20to the illumination device20via the input/output interface43. The imaging element driving process is a process of supplying a drive signal for driving the solid-state imaging element1311in the rigid endoscope10to the endoscope10via the input/output interface43.

The display control process is a process of applying a 3D process on the image signal SD transmitted from the rigid endoscope10and outputting the 3D image obtained by the 3D process to the display device30as an image signal SD3Dof a moving image having a frame rate of 59.94 frames per second.

The zoom control process is a process of obtaining a line-of-sight position corresponding to the line of sight of the operator on the lenses55L and55R of the polarized glasses50based on the potential waveform indicated by the detection signal SGV1of the potential sensor51, the potential waveform indicated by the detection signal SGV2of the potential sensor52, and the potential waveform indicated by the detection signal SGV3of the potential sensor53of the polarized glasses50at the time of generation of a zoom-in or zoom-out trigger signal when the trigger signal is generated, identifying the fixation point FP of the operator based on the line-of-sight position, the relationship between the position of the display device30and the position of the polarized glasses50, and the relationship between the orientation of the display device30and the orientation of the polarized glasses50, and enlarging or reducing the image of the periphery of the fixation point FP.

More specifically, as shown inFIG. 8, in the display control process, the control unit46stores the image signal SD (the image of visible light and the image of near-infrared light) in the reception buffer45S of the storage unit45as photographed image data. Next, the control unit46generates left-eye image data and right-eye image data having binocular parallax from the photographed image data in the reception buffer45S, and stores the left-eye image data and the right-eye image data in the left-eye image buffer45L and the right-eye image buffer45R. Then, the control unit46combines the left-eye image data and the right-eye image data, and stores the combined image data in the drawing frame buffer45D of the storage unit45. The control unit46replaces the drawing frame buffer45D and the display frame buffer45E of the storage unit45every 1/59.94 seconds (≈0.17 seconds), and outputs the image data in the display frame buffer45E to the display device30as the image signal SD3D.

As shown inFIG. 9, in the zoom control process, the control unit46obtains, from the output signal SGV1of the potential sensor51, the output signal SGV2of the potential sensor52, and the output signal SGV3of the potential sensor53of the polarized glasses50, the potential of the potential sensor51(an absolute value of the amplitude and a positive/negative sign) when the potential of the potential sensor53is set as the reference potential and the potential of the potential sensor52(an absolute value of the amplitude and a positive/negative sign) when the potential of the potential sensor53is set as the reference potential.

Then, the control unit46identifies the X coordinate value and Y coordinate value of the left-eye line-of-sight position of the operator on the lens55L of the polarized glasses50(X coordinate value and Y coordinate value of the intersection position of the XY plane, which is parallel to the lens55L and takes the position of the potential sensor53as the origin (0.0), and the line of sight of the left eye) with reference to a left-eye line-of-sight position identification table of the storage unit45. Further, the control unit46identifies the X coordinate value and Y coordinate value of the right-eye line-of-sight position of the operator on the lens55R of the polarized glasses50(X coordinate value and Y coordinate value of the intersection position of the XY plane, which is parallel to the lens55R and takes the position of the potential sensor53as the origin (0.0), and the line of sight of the right eye) with reference to a right-eye line-of-sight position identification table of the storage unit45.

Here, for human eyes, the cornea side is positively charged and the retina side is negatively charged. Therefore, as indicated by the waveform ofFIG. 10, when the operator turns the line of sight upward from the front, the potential (left-eye potential) of the potential sensor51taking the potential of the potential sensor53as reference and the potential (right-eye potential) of the potential sensor52taking the potential of the potential sensor53as reference at the time when the line of sight is turned upward (time to ofFIG. 10) both become negative.

As shown inFIG. 11, when the operator turns the line of sight downward from the front, the potential (left-eye potential) of the potential sensor51taking the potential of the potential sensor53as reference and the potential (right-eye potential) of the potential sensor52taking the potential of the potential sensor53as reference at the time when the line of sight is turned downward (time tDofFIG. 11) both become positive.

As shown inFIG. 12, when the operator turns the line of sight from the front to the left, the potential (left-eye potential) of the potential sensor51taking the potential of the potential sensor53as reference at the time when the line of sight is turned to the left (time tLofFIG. 12) becomes positive, and the potential (right-eye potential) of the potential sensor52taking the potential of the potential sensor53as reference becomes negative.

As shown inFIG. 13, when the operator turns the line of sight from the front to the right, the potential (left-eye potential) of the potential sensor51taking the potential of the potential sensor53as reference at the time when the line of sight is turned to the left (time tRofFIG. 13) becomes negative, and the potential (right-eye potential) of the potential sensor52taking the potential of the potential sensor53as reference becomes positive.

In the left-eye line-of-sight position identification table, the measured values of the potentials of the first potential sensor51, the second potential sensor52, and the third potential sensor53when the line of sight of the subject is directed to each point on the left lens55L of the polarized glasses50are recorded in association with the X coordinate and Y coordinate of each point. In the right-eye line-of-sight position identification table, the measured values of the potentials of the first potential sensor51, the second potential sensor52, and the third potential sensor53when the line of sight of the subject wearing the polarized glasses50is directed to each point on the right lens55R are recorded in association with the X coordinate and Y coordinate of each point. Therefore, by referring to the tables based on the output signals SGV1, SGV2, and SGV3of the sensors of the polarized glasses50of the operator, the line-of-sight position of the operator on the lenses of the polarized glasses50can be identified.

InFIG. 9, the control unit46respectively obtains the difference SDX-SGXbetween the output signal SDXof the position detection sensor36of the display device30and the output signal SGXof the position detection sensor56of the polarized glasses50, the difference SDY-SGYbetween the output signal SDYof the position detection sensor36of the display device30and the output signal SGYof the position detection sensor56of the polarized glasses50, the difference SDZ-SGZbetween the output signal SDZof the position detection sensor36of the display device30and the output signal SGZof the position detection sensor56of the polarized glasses50, the difference SDθX-SGθXbetween the output signal SDθXof the orientation detection sensor37of the display device30and the output signal SGθXof the orientation detection sensor57of the polarized glasses50, and the difference SDθZ-SGθZbetween the output signal SDθZof the orientation detection sensor37of the display device30and the output signal SGθZof the orientation detection sensor57of the polarized glasses50.

From the differences SDX-SGX, SDY-SGY, SDZ-SGZ, SDθX-SGθX, and SDθZ-SGθZ, the control unit46generates a transformation matrix for transforming the X coordinate values and Y coordinate values of the line-of-sight positions of the left and right eyes into the X coordinate value and Y coordinate value of the fixation point FP on the display screen of the display device30, and obtains the X coordinate value and Y coordinate value of the fixation point FP by applying the transformation matrix to the X coordinate values and Y coordinate values of the line-of-sight positions of the left and right eyes. The control unit46supplies the X coordinate value and Y coordinate value of the fixation point FP to the image processing unit44as fixation point data. If a zoom-in trigger signal is generated, when receiving the fixation point data, the image processing unit44performs a process of rewriting the image in the drawing frame buffer45D to an enlarged image of a predetermined rectangular area centered on the fixation point FP. In addition, if a zoom-out trigger signal is generated, when receiving the fixation point data, the image processing unit44performs a process of rewriting the image in the drawing frame buffer45D to a reduced image of the predetermined rectangular area centered on the fixation point FP.

InFIG. 1, a cable165, an air supply pipe164A, and one end of an air exhaust pipe164B are connected to the rear end of the grip part1132of the housing131of the endoscope. The cable165is connected to the control device40. The air supply pipe164A is connected to the air intake and exhaust device60via the air cooling device70. The air exhaust pipe164B is connected to the air intake and exhaust device60. The air intake and exhaust device60is a device that serves to forcibly supply air into the housing131via the air supply pipe164A as well as forcibly exhaust air from inside the housing131via the air exhaust pipe. The air cooling device70is a device that serves to cool the air flowing through the air supply pipe164A.

The housing131of the rigid endoscope10, the air supply pipe164A, and the air exhaust pipe164B form one closed space, and a flow of air for cooling the inside of the housing131is generated in the closed space. The flow of air will be described. (A) ofFIG. 14is a diagram showing the details of the configuration inside the housing131. (B) ofFIG. 14is a diagram showing the details of the configuration of the air intake and exhaust device60, the air cooling device70, the air supply pipe164A, and the air exhaust pipe164B.

As shown in (A) ofFIG. 14, the solid-state imaging element1311and a substrate1312supporting the solid-state imaging element1311are provided at a position facing the eyepiece1201in the front of the housing131. Electronic components such as the A/D conversion unit1319are mounted on the substrate1312.FIG. 15is a diagram enlarging the area of the solid-state imaging element1311and the substrate1312of (A) ofFIG. 14. An anti-reflection glass1315is attached to the solid-state imaging element1311. A plurality of ball grids1313are interposed between the solid-state imaging element1311and the substrate1312.

A heat sink1316is provided behind the substrate1312. The heat sink1316is of a so-called needle type in which a plurality of fins1316B are erected from a flat plate1316A. An opening having substantially the same size as the flat plate1316A of the heat sink1316is provided at the center of the substrate1312. The flat plate1316A of the heat sink1316is fitted into the opening. The flat plate1316A of the heat sink1316is bonded to the solid-state imaging element1311via a heat conductive adhesive1314. A duct1318is provided at a position on the inner side of the grip part1132in the housing131. One end of the duct1318is directed to the fins1316B of the heat sink1316. The other end of the duct1318is connected to the air supply pipe164A.

A structure composed of the image processing FPGA (Field Programmable Gate Array)1331, a substrate1332, a heat sink1336, a cover member1338, and a duct166B is provided below the duct1318on the inner side of the grip part1132of the housing131.FIG. 16is a diagram enlarging the area of the image processing FPGA1331, the substrate1332, the cover member1338, the heat sink1336, and the duct166B of (A) ofFIG. 14. A plurality of ball grids1333are interposed between the image processing FPGA1331and the substrate1332.

The heat sink1336is provided above the substrate1332. The heat sink1336is of a so-called needle type in which a plurality of fins1336B are erected from a flat plate1336A. The size of the flat plate1336A of the heat sink1336is substantially the same as the size of the image processing FPGA1331. The flat plate1336A of the heat sink1336is bonded to the image processing FPGA1331via a heat conductive adhesive1334.

The cover member1338is provided above the heat sink1336. The cover member1338has a shape that opens the lower surface of a thin box1338A, protrudes a cylinder1338B from the center of a surface opposite to the opened side, and mildly bends the cylinder1338B in a direction orthogonal to the base end surface of the cylinder. The opening of the cover member1338covers the heat sink1336. The tip of the cylinder1338B of the cover member1338is connected to the duct166B. The other end of the duct166B is connected to the air exhaust pipe164B.

In the above configuration in the housing131, by operating the air supply device160A and the air exhaust device160B in the air intake and exhaust device60and the air cooling device70, the air supply device160A generates a positive pressure of +10 hPa to +20 hPa, and sends out the air sucked in from the outside to the air supply pipe164A by this pressure. The air exhaust device160B generates a negative pressure of −10 hPa to −20 hPa, and sends out the air sucked in from the air exhaust pipe164B164B to the outside by this pressure.

The air sent from the air supply device160A to the air supply pipe164A is cooled by the air cooling device70when passing through the air cooling device70. The cooled air passes through the air supply pipe164A and the duct1318, and is blown to the heat sink1316as a first airflow from the opening at the tip of the duct1318. The first airflow passes through the heat sink1316and flows to the side part thereof, and circulates in housing131. The first airflow takes away the heat as it passes through the heat sink1316. The air that passes through the heat sink1316and flows to the lower part thereof is blown to the lower heat sink1336in the housing131as a second airflow. After passing through the heat sink1336, the second airflow is sucked into the opening of the cover member1338. The second airflow takes away the heat as it passes through the heat sink1336. The air that passes through the heat sink1336and is sucked into the cover member1338is exhausted to the outside by the air exhaust device160B via the duct166B and the air exhaust pipe164B.

The above are the details of the present embodiment. According to the present embodiment, the following effects can be obtained. First, in the present embodiment, the solid-state imaging element1311of the rigid endoscope10has the pixels PXij(i=1 to 4320, j=1 to 7680), the number of which corresponds to 8K. Here, for the conventional 2K or 4K endoscope, the endoscope must be brought close to the subject for photographing, and the surgical instruments such as a scalpel and forceps may interfere with the endoscope in the body cavity and cause the surgery to be delayed. However, the endoscope of the present embodiment can obtain a sufficiently fine photographed image even if the subject is photographed from a position about 8 cm to 12 cm away from the subject in the body cavity. Therefore, a wide field of view and a wide surgical space in the body cavity can be ensured, and the surgery can be realized more smoothly.

Second, in the present embodiment, a red filter, a green filter, a blue filter, and a near-infrared filter are attached to the pixels PXij(i=1 to 4320, j=1 to 7680) of the solid-state imaging element1311of the rigid endoscope10, and the display device30displays an image in which a RGB image of visible light and an image of near-infrared light are superimposed. Here, when an infrared excitation drug is injected into the vein of the patient, the drug binds to the protein in the blood. When excitation light is applied from outside the blood vessel, near-infrared light is emitted and the near-infrared light is reflected in the image. Thus, the operator can simultaneously grasp the state of the affected part itself, which is the target to be treated, and the state of distribution of blood vessel in the depth by viewing one image displayed on the display device30.

Third, in the present embodiment, the display device30displays the photographed image of the rigid endoscope10as a 3D moving image. Thus, the operator can accurately grasp the positional relationship and the distance between the target organ that is to be treated and the surgical instruments in the body cavity.

Fourth, in the present embodiment, the control device40identifies the fixation point FP of the operator in the display image of the display device30based on the relationship between the detection signal of the sensor in the polarized glasses50and the detection signal of the sensor in the display device30at the time of generation of the trigger signal, and enlarges and displays the periphery of the fixation point FP. The operator can specify the zoom-in or zoom-out range as intended without performing a troublesome input operation.

Fifth, in the present embodiment, the pitch between adjacent pixels PXijin the solid-state imaging element1311of the rigid endoscope10is larger than the longest wavelength of the wavelengths of the light in the illumination that illuminates the subject. Here, the 8K solid-state imaging element1311is an array of 4320 rows and 7680 columns of the pixels PXij(i=1 to 4320, j=1 to 7680). Therefore, if the degree of integration of the pixels PXij(i=1 to 4320, j=1 to 7680) is increased, it is difficult to make the size of the housing131easy to handle. On the other hand, if the degree of integration of the pixels PXij(i=1 to 4320, j=1 to 7680) of the solid-state imaging element1311is too high, the relationship between the pitch between adjacent pixels PXijin the solid-state imaging element1311and the wavelength of light becomes “pitch<wavelength”, and due to the diffraction effect of light, the photographed image may be blurred. By making the pitch between adjacent pixels PXijin the solid-state imaging element1311larger than the longest wavelength of the wavelengths of the light in the illumination that illuminates the subject, it is possible to provide an endoscope that achieves both a clear image and compactness.

Sixth, the housing131of the rigid endoscope10includes the mount part1131having a large cross-sectional area orthogonal to the optical axis of the light passing through the insertion part110, and the grip part1132having a smaller cross-sectional area than the mount part1131. Since the 8K solid-state imaging element1311has 4320 rows and 7680 columns of the pixels PXij(i=1 to 4320, j=1 to 7680), it is difficult to make the size the same as that of a 4K imaging element, and there are some limits to miniaturization. If the overall thickness of the housing of the 8K endoscope is adjusted to the vertical and horizontal dimensions of the solid-state imaging element1311, the endoscope cannot be held with one hand. In the present embodiment, the cross-sectional area of the grip part1132is smaller than the cross-sectional area of the mount part1131, so it is possible to provide an endoscope which has a high resolution and can be held with one hand to be handled properly.

Seventh, in the present embodiment, the housing131of the rigid endo scope10is provided with the first heat sink1316provided on the solid-state imaging element1311, the image processing FPGA1331, the second heat sink1336provided on the FPGA1331, and the cover member1338that covers the second heat sink1336and is connected to the air exhaust pipe164B. Further, in the present embodiment, the first airflow for cooling the first heat sink1316and the second airflow for cooling the second heat sink1336are generated. The first airflow is formed by blowing cooling air supplied from the air supply pipe164A to the first heat sink1316to diverge around the first heat sink1316, and the second airflow is formed to flow from around the second heat sink1336to the air exhaust pipe164B via the cover member1338. Therefore, the solid-state imaging element1311and the image processing FPGA1331, which are the main heat sources in the housing131of the endoscope, can be efficiently cooled.

Second Embodiment

FIG. 17is a diagram showing a configuration of an endoscope system including a flexible endoscope10′ according to the second embodiment of the invention. The endoscope system of the present embodiment supports surgery performed by inserting the flexible endoscope10′ from the mouth or anus for treatment on an organ in the body cavity. In the present embodiment, the rigid endoscope10of the endoscope system of the first embodiment is replaced with the flexible endoscope10′. The solid-state imaging element in the housing131of the flexible endoscope10′ has a smaller number of pixels (for example, a solid-state imaging element having 300,000 pixels) than that of the solid-state imaging element in the housing131of the rigid endoscope10of the first embodiment.

InFIG. 17, elements the same as those ofFIG. 1are denoted by the same reference numerals, and elements different from those ofFIG. 1are denoted by different reference numerals. (A) ofFIG. 18is a diagram of an insertion part110′ ofFIG. 1when viewed from the direction of the arrow A, and (B) ofFIG. 18is a cross-sectional diagram of the insertion part110′ ofFIG. 1taken along the line B-B′. The insertion part110′ of the flexible endoscope10′ has a flexible lens barrel111′ and an eyepiece112. The flexible lens barrel111′ is made of a flexible material.

A hollow light guide area1112′ is provided in the flexible lens barrel111′ of the insertion part110′. The hollow light guide area1112′ is a cavity having a diameter that is about half the diameter of the insertion part110′. The center of the cross section of the hollow light guide area1112′ is shifted from the center of the cross section of the insertion part110′. The portion of the outer shell surrounding the hollow light guide area1112′ of the insertion part110′ on the side close to the center of the hollow light guide area1112is thinner, and the portion on the side close to the center of the hollow light guide area1112′ is thicker. One optical fiber1901is embedded in the thick portion of the outer shell of the insertion part110′. A diffusion lens (not shown) is embedded in front of the tip of the optical fiber1901in the insertion part110′.

An objective lens1111is fitted at a position that is farther from the tip in the hollow light guide area1112′ of the insertion part110. A multi-core fiber1116is housed between the objective lens1111and the eyepiece112in the hollow light guide area1112′. The light guiding direction of each core CR of the multi-core fiber1116is parallel to the direction in which the insertion part110′ extends.

The cross section of the portion on the tip side of the position where the objective lens1111is fitted in the hollow light guide area1112′ of the insertion part110is wider than the cross section of the portion where the objective lens1111and the multi-core fiber1116are provided. A movable mirror1114is supported at a position facing the objective lens1111in the portion on the tip side of the hollow light guide area1112′. The movable mirror1114is a MEMS (Micro Electro Mechanical Systems) mirror. The movable mirror1114is supported to be swingable around two axes, a first axis φ1and a second axis φ2. The first axis φ1is an axis that intersects the optical axis of the light passing through each core of the multi-core fiber1116with a tilt with respect to the optical axis (the optical axis of the objective lens1111). The second axis φ2is an axis orthogonal to both the optical axis of the objective lens1111and the first axis φ1. A fixed mirror1115is fixed between the movable mirror1114and the optical fiber1901in the portion on the tip side of the hollow light guide area1112′. A reflective surface of the movable mirror1114faces the objective lens1111and the fixed mirror1115. A reflective surface of the fixed mirror1115faces the movable mirror1114and the outside of the opening1801at the tip of the insertion part110′.

In the present embodiment, the control unit46of the control device40performs an illumination driving process, an imaging element driving process, a display control process, a zoom control process, and a mirror driving process by running of the operation program PRG in the storage unit45. The mirror driving process is a process of supplying a drive signal for driving the movable mirror1114to the movable mirror1114via the input/output interface43. The control unit46generates divided area images of different portions of the subject in the body cavity by periodically switching the tilt angles of the movable mirror1114around the axis φ1and the axis φ2at intervals of a time T (for example, T= 1/120 second) shorter than the frame switching of the frames of the moving image through supply of a drive signal to the movable mirror1114, and generates the image of a frame by combining the generated divided area images.

More specifically, as shown inFIG. 19, the control unit46divides the photographing range of the flexible endoscope10′ in the body cavity into M (M is a natural number of 2 or more, and M=9 in the example ofFIG. 19), and performs the following processing in accordance with the time t1, time t2, . . . time tM(=9)within the time T.

At the time t1, the control unit46controls the tilt of the movable mirror1114so that the light of the first area AR-1of the M areas AR-k (k=1 to 9) obtained by dividing the entire photographing range into M is guided from the opening1801at the tip of the insertion part110′ to the multi-core fiber1116via the fixed mirror1115and the movable mirror1114. The light of the area AR-1reaches the solid-state imaging element1311through the multi-core fiber1116. After photoelectric conversion in the solid-state imaging element1311, the light of the area AR-1is stored in the reception buffer45S of the storage unit45as image data. When the image data of the area AR-1is stored in the reception buffer45S, the control unit46stores the image data in the storage area corresponding to the area AR-1in the drawing frame buffer45D.

At the time t2, the control unit46controls the tilt of the movable mirror1114so that the light of the second area AR-2of the M areas AR-k (k=1 to 9) is guided from the opening1801at the tip of the insertion part110′ to the multi-core fiber1116through the fixed mirror1115and the movable mirror1114. The light of the area AR-2reaches the solid-state imaging element1311through the multi-core fiber1116. After photoelectric conversion in the solid-state imaging element1311, the light of the area AR-2is stored in the reception buffer45S of the storage unit45as image data. When the image data of the area AR-2is stored in the reception buffer45S, the control unit46stores the image data in the storage area corresponding to the area AR-2in the drawing frame buffer45D.

The control unit46performs the same processing at the time t3, time t4, time t5, time t6, time t7, time t8, and time t9, and stores the image data generated for each of the area AR-3, area AR-4, area AR-5, area AR-6, area AR-7, area AR-8, and area AR-9in a separate storage area of the drawing frame buffer45D. According to the time of the next frame switching, the drawing frame buffer45D is replaced with the display frame buffer45E, and the image data of the areas AR-1, AR-2, AR-3, AR-4, AR-5, AR-6, AR-7, AR-8, and AR-9in the display frame buffer45E is output to the display device30as a moving image signal SC3Dof one frame.

The above are the details of the present embodiment. Here, the 8K solid-state imaging element can be housed in the housing131of the flexible endoscope10′ of the insertion part110′, but if the number of cores of the multi-core fiber1116is increased to be the same as the number of pixels of the solid-state imaging element, the flexibility may be impaired. The upper limit of the number of cores that can maintain the flexibility is about 10,000.

Regarding this, in the present embodiment, the movable mirror1114and the fixed mirror1115are provided in the flexible lens barrel111′ of the insertion part110′, and the movable mirror1114is tiltable around two axes, that is, the first axis φ1having a tilt with respect to the optical axis direction of the light passing through each core CR of the multi-core fiber1116, and the second axis φ2orthogonal to the first axis φ1. Further, the control device40generates divided area images of different portions of the subject by periodically switching the tilt angles of the movable mirror1114at intervals of the time T shorter than the frame switching time interval of the frame rate of a moving image, and generates a moving image for one frame by combining the generated divided area images. Therefore, according to the present embodiment, an 8K photographed image can be obtained while the multi-core fiber1116is kept as thin as the conventional fiberscope (less than 2K). Thus, according to the present embodiment, the flexible endoscope10′ having an 8K resolution can be realized using a solid-state imaging element of less than 2K.

The first and second embodiments of the invention have been described above. Nevertheless, the following modifications may be added to the present embodiments.

(1) In the first and second embodiments, as shown in (A) ofFIG. 20, the focal distance (distance between the solid-state imaging element1311and the optical system) may be set short so that the image circle of the optical system (objective lens1111, eyepiece1201, relay lens1113, etc.) in the insertion part110of the endoscope circumscribes the light receiving area of the solid-state imaging element1311, and as shown in (B) ofFIG. 20, the focal distance (distance between the solid-state imaging element1311and the optical system) may be set long so that the image circle of the optical system (objective lens1111, eyepiece1201, relay lens1113, etc.) in the insertion part110of the endoscope inscribes the light receiving area of the solid-state imaging element1311.

(2) In the first and second embodiments, the position detection sensor56and the orientation detection sensor57are embedded in the polarized glasses50, and the position detection sensor36and the orientation detection sensor375are also embedded in the display device30. However, the display device30may not include the position detection sensor36and the orientation detection sensor37. In that case, the control unit46may use fixed values of the X coordinate value, Y coordinate value, Z coordinate value, direction, and elevation angle of the position of the display device30, and the detection signals of the detection signals of the position detection sensor56and the orientation detection sensor57of the polarized glasses50to generate the transformation matrix.

(3) In the first embodiment, 4320 rows and 7680 columns of the pixels PXijin the solid-state imaging element1311form blocks each having four pixels PXijin 2 rows and 2 columns, and red, green, blue, and near-infrared filters are affixed to the four pixels PXijof each block. However, red, green, blue, and near-infrared filters may not be affixed to each block of four as described above. For example, 4320 rows may divided into blocks every 4 rows, a red filter may be affixed to the pixels PXijin all columns in the first row of the blocks, a green filter may be affixed to the pixels PXijin all columns in the second row, a blue filter may be affixed to the pixels PXijin all columns in the third row, and near-infrared light may be affixed to the pixels PXijin all columns in the fourth row.

(4) In the first and second embodiments, the housing131has the buttons139IN and139OUT, and a trigger signal is generated when the buttons139IN and139OUT are pressed shortly once. However, the generation of the trigger signal may be triggered in other ways. For example, a microphone may be mounted on the housing131, and the generation of the trigger signal may be triggered when the operator says the word “zoom”. In addition, a zoom-in trigger signal that instructs zoom-in of the image displayed on the display device30may be generated when the button139IN is pressed shortly once, and a release trigger signal that instructs to release the zoom-in may be generated and a zoom-out trigger signal may not be generated when the button139IN is pressed long once.

(5) In the first and second embodiments, the solid-state imaging element1311is a CMOS image sensor. However, the solid-state imaging element1311may be constituted by a CCD (Charge Coupled Device) image sensor.

(6) In the second embodiment, one multi-core fiber1116is housed in the insertion part110′. However, a plurality of multi-core fibers1116may be housed.

(7) In the second embodiment, the insertion part110′ has two mirrors, the fixed mirror1115and the movable mirror1114. However, the number of mirrors may be one or three or more. Further, one or more of them may be movable mirrors. In short, the light needs to be guided from the subject to the multi-core fiber1116via one or a plurality of mirrors to make the scanning area variable.

(8) In the second embodiment, the first axis φ1only needs to be tilted with respect to the optical axis of the light passing through each core of the multi-core fiber1116, and does not necessarily intersect the optical axis of the light passing through each core of the multi-core fiber1116. In this case, preferably the tilt of the movable mirror1114around the second axis φ2is controlled so that the tilt in the first axis φ1with respect to the optical axis direction of the light passing through each core CR of the multi-core fiber1116is 45 degrees±a predetermined angle. In addition, preferably the tilt of the movable mirror1114about the first axis φ1is controlled so that the tilt in the second axis φ2with respect to the light passing through each core CR of the multi-core fiber1116is 90 degrees±a predetermined angle.

DESCRIPTIONS OF REFERENCE NUMERALS