SURGICAL VIDEO CREATION SYSTEM

Disclosed is a surgical video creation system, comprising: a surgical microscope comprising a light source; an image processing device configured so as to create a 3D video of surgical scenes and fluorescent images by means of the surgical scenes using the microscope; an optical adapter provided so that the image processing unit is mounted on the microscope; and a display unit configured so as to display the 3D video, wherein the image processing unit is configured so as to recognize the boundary of tumorous tissue by means of the fluorescent images and display the boundary in the 3D video, and the fluorescent images are created by light emitted from a fluorescent material which is selectively accumulated only in the tumor.

TECHNICAL FIELD

The present invention relates to a surgical video creation system, and more particularly, to a system for creating a medical surgical stereoscopic video.

BACKGROUND ART

Generally, a medical surgical microscope is a surgical device that can magnify the inside of a human body, which cannot be easily checked, during surgery. Such surgical microscopes are equipped with an imaging system that allows an operating doctor (hereinafter referred to as an “operator”) to see a surgical procedure through a monitor. However, such an imaging system displays two-dimensional (2D) images, and it is difficult to accurately observe and check a site subject to surgery using the 2D image, and thus there is a problem that the operator cannot perform surgery through the imaging system.

Also, the white balance adjustment function of a conventional imaging device has a limited range of adjustment and was developed based on sunlight. Therefore, in an environment in which a narrow and deep site is imaged using strong light such as a surgical microscope light source, even if the white balance is adjusted, distortion in which the color of human tissues or blood is expressed as pink instead of red occurs, and this causes problems in medical judgements related to, e.g., bleeding and lesions.

Also, there was developed a surgical method for removing a tumor using a microscope during surgery after a patient takes a special fluorescent substance in order to distinguish the tumor. Such fluorescent substances react with a patient's cancer cells to produce a unique substance, and the produced substance emits a fluorescent substance at an excitation wavelength, thereby distinguishing between normal tissues and tumors. However, in this surgical method, the fluorescent substance absorbed by the tumor cells can be visually distinguished only when lighting of a specific wavelength must be applied to an affected part while all lights in an operating room are turned off. Therefore, it is not possible to check tumors during surgery at any time while a conventional surgical lighting is turned on.

Technical Problem

The present invention is directed to overcoming the above-described problems and to provide a stereoscopic video that can accurately show a surgical procedure.

The present invention is also directed to providing a stereoscopic video that can represent a red color without distortion in a surgical video using a medical surgical microscope.

Also, the present invention is also directed to providing a stereoscopic video that can distinguish normal tissues and tumors at any time without turning off the lighting of an operating room.

The technical objects to be achieved by the present invention are not limited to those mentioned above, and other technical objects, which are not mentioned herein, may be clearly understood by those skilled in the art from the following description.

Technical Solution

According to an embodiment, there is provided a surgical video creation system including a surgical microscope including a light source, an image processing device configured to create a stereoscopic video of a surgical scene and a fluorescent image by means of the surgical scene using the microscope, an optical adapter configured so that the image processing unit is mounted on the microscope, and a display unit configured to display the stereoscopic video. The image processing unit is configured to recognize a boundary of a tumor tissue using the fluorescent image and mark the boundary in the stereoscopic video, and the fluorescent image is formed of light emitted from a fluorescent material which is selectively accumulated only in the tumor tissue.

Also, the image processing unit includes a filter configured to pass light corresponding to a first wavelength. The emitted light is light of the first wavelength. The fluorescent image is represented in a first color and a second color corresponding to the first wavelength. The image processing unit is configured to recognize a first region corresponding to the first color as the tumor tissue, recognize a second region corresponding to the second color as a normal tissue, and create the stereoscopic video in which a boundary between the first region and the second region is marked.

Also, the first wavelength is 635 nm, the first color is a red fluorescent color, and the second color is a blue fluorescent color.

Also, the image processing device is configured to mark the boundary by applying at least one of Sobel, Prewitt, Roberts, Compass, Laplacian, Laplacian of Gaussian (LoG) or Canny to the fluorescent image.

Also, the image processing device includes a mirror assembly configured to divide the image of the surgery scene into a first image and a second image and an image processing unit configured to create the stereoscopic video using the first image. The second image passes through the filter.

Also, the image processing unit is configured to measure a color temperature of the light source and interpolate chrominance of a red color of the first image using reference chrominance corresponding to the light source subject to the measurement.

Also, the light source is a light source with a color temperature between 3000 K and 7000 K.

The image processing device includes a camera configured to capture the surgery scene, a first stage configured to move the focus of the camera to the right or the left, and a second stage configured to move the focus of the camera upward or downward.

The first stage includes a first moving part and a first fixed part. The second stage includes a second moving part and a second fixed part. The first moving part is configured to move along an arc with respect to the first fixed part. The second moving part is configured to move along an arc with respect to the second fixed part.

The first stage includes a first knob. The second stage includes a second knob. The focus is moved to the right or left in response to rotation of the first knob and is moved up or down in response to rotation of the second knob.

The camera is fixed to the first moving part and the first stage is fixed to the second moving part, and the camera and the first stage are moved up or down in response to rotation of the second knob.

A stator including a horizontal surface and a vertical surface is between the first stage and the second stage. The first fixed part is fixed to the vertical surface, and the second fixed part is fixed to the horizontal surface.

Advantageous Effects

According to an embodiment of the present invention, it is possible to provide a stereoscopic video that can accurately show a surgical procedure.

It is also possible to provide a stereoscopic video that can represent a red color without distortion in a surgical video using a medical surgical microscope.

It is also possible to provide a stereoscopic video that can distinguish normal tissues and tumors at any time without turning off the lighting of an operating room.

DETAILED DESCRIPTION

Hereinafter, embodiments disclosed herein will be described in detail with reference to the accompanying drawings, but the same or similar elements are assigned the same or similar reference numerals, and redundant descriptions thereof will be omitted. Also, the accompanying drawings are only for making it easier to understand the embodiments disclosed herein, and therefore, the technical spirit disclosed herein is not limited by the accompanying drawings. Also, it should be understood that all modifications, equivalents, and alternatives falling within the spirit and scope of the invention are encompassed.

It will be understood that when an element is referred to as being “connected” or “coupled” to another element, it can be directly connected or coupled to the other element or an intervening element may be present. In contrast, when an element is referred to as being “directly connected” or “directly coupled” to another element, there are no intervening elements present.

A surgical video creation system according to an embodiment will be described below with reference toFIG. 1.FIG. 1is a block diagram showing a configuration of a surgical video creation system according to an embodiment.

Referring toFIG. 1, a surgical video creation system1according to an embodiment may include a surgical microscope10, an optical adapter20, an image processing device30, a recorder40, and a display unit50and may create a stereoscopic video including a surgical site and adjacent sites and a stereoscopic video with the boundary of a tumor being marked and then may display the videos in the display unit50.

According to the present invention, prior to surgery, when a patient takes a fluorescent substance called 5-aminolevulinic acid (5-ALA; Gliolan), the active substance of the 5-ALA, protoporphyrin IX, is selectively accumulated only in tumor cells, and thus fluorescent light of a first wavelength (e.g., 635 nm) is emitted. This fluorescent light is brightest after a reference time (e.g., 2.5 hours after a patient takes 5-ALA).

Therefore, after the reference time, in a fluorescent image Fi seen through a filter unit33, a tumor site is viewed in a first color of a first wavelength (e.g., red florescent color of 635 nm), and a normal tissue is viewed as a second color (e.g., blue fluorescent color). According to the present invention, the surgery may be a surgery to remove such a tumor, but the embodiments are not limited thereto.

The surgical microscope10is a motorized mechanical optical device used in various surgical operations and includes a light source (light-emitting diode (LED), Xeon, Halogen, etc.). An image of a surgical site or an adjacent site may be enlarged and viewed using the light of the light source. The color temperature of such a light source may be between 3000 K and 7000 K, but the present invention is not limited thereto.

The optical adapter20is configured such that the image processing device30may be mounted on the surgical microscope10. The optical adapter20separates a surgical image i (hereinafter referred to as an image) input through the surgical microscope10into a plurality of images, and any one of the plurality of images is input to the image processing device30.

The image processing device30includes a mirror assembly31, a camera unit32, a filter unit33, a first image processing unit34, a second image processing unit35, and a third image processing unit36. The image processing device30converts an image into a right-eye image Ri and a left-eye image Li and outputs the images in order to generate a stereoscopic image. Also, the image processing device30recognizes a patient's tumor using the fluorescent image Fi and combines an image in which the boundary of the recognized tumor is marked with the stereoscopic video.

The mirror assembly31may divide the image i into a plurality of images. Specifically, the mirror assembly31includes a plurality of reflectors (not shown) that horizontally and/or vertically reflect the image i. The mirror assembly31may separate the image i into a first image i1, a second image i2, and a third image i3using the plurality of reflectors.

The camera unit32includes a first camera321a,a second camera321b,and a base plate322(seeFIG. 2). The camera unit32captures a surgery scene using the surgical microscope10. The camera unit32creates the first image it from the captured image and delivers the first image i1to the first image processing unit34. The camera unit32creates the second image i2and delivers the second image i2to the second image processing unit35.

The first camera321aincludes a first camera3211a,a first stage3212a,a first stator3213a,and a second stage3214a.

The first camera3211acaptures the first image i1and delivers the first image i1to the first image processing unit34.

The first stage3212aincludes a moving part3214am,a fixed part3212af,and a knob n1a,and the first camera3211ais fixed to the moving part3212am.The moving part3212ammoves along an arc to the right R or the left L according to the adjustment of the knob n1a,and the first camera3211amoves to the right R or the left L in response to the movement of the moving part3212am.Therefore, by adjusting the knob n1a,the focus of the first camera3211amay be moved to the right R or the left L.

The first stator3213ais in the shape of the letter “L.” The first stator3213ais vertically symmetrical with the second stator3213band is in contact with the second stator3213bon a symmetrical surface.

The second stage3214aincludes a moving part3214am,a fixed part3214af,and a knob n2a,and the bottom surface of the first stator3213ais fixed onto the moving part3214am.The moving part3214ammoves along an arc in one upward direction U1or another upward direction U2according to the adjustment of the knob n2a.Therefore, the first camera3211a,the first stage3212a,and the first stator3213amove vertically in response to the movement of the moving part3214am.That is, by manipulating the knob n2a,the focus of the first camera3211amay be moved up or down.

The second camera321bincludes a second camera3211b,a third stage3212b,a second stator3213b,and a fourth stage3214b.

The second camera3211bcaptures the second image i2and delivers the second image i2to the second image processing unit35.

The third stage3212bincludes a moving part3212bm,a fixed part3212bf,and a knob n1b,and the second camera3211bis fixed to the moving part3212bm.The moving part3212bmmoves along an arc to the right R or the left L according to the adjustment of the knob n1b,and the second camera3211bmoves to the right R or the left L in response to the movement of the moving part3212bm.Therefore, by adjusting the knob n1b,the focus of the second camera3211bmay be moved to the right R or the left L.

The second stator3213bis in the shape of the letter “L.” The second stator3213bis vertically symmetrical with the first stator3213aand is in contact with the first stator3213aon a symmetrical surface.

The fourth stage3214bincludes a moving part3214bm,a fixed part3214bf,and a knob n2b,and the bottom surface of the second stator3213bis fixed onto the moving part3214bm.The moving part3214bmmoves along an arc in one upward direction U1or another upward direction U2according to the adjustment of the knob n2b.Therefore, the first camera3211b,the third stage3212b,and the second stator3213bmove vertically in response to the movement of the moving part3214bm.That is, by manipulating the knob n2b,the focus of the second camera3211bmay be moved up or down.

The fixed part3214afand the fixed part3214bfare fixed onto the base plate322.

The filter unit33includes a band pass filter, and such a band pass filter passes light of a first wavelength in the third image i3that is input. The third image i3passes through the filter unit33and is converted into a fluorescent image Fi composed of light of the first wavelength, and the fluorescent image Fi is input to the third image processing unit36.

Specifically, when a patient takes 5-ALA, 5-ALA is absorbed only in the tumor (c) cell shown inFIG. 3and is converted into a fluorescent substance (protoporphyrin IX), and the fluorescent substance emits fluorescent light of the first wavelength. In this case, the fluorescent substance emits the brightest light after a reference time.

Therefore, the fluorescent image Fi is composed of a region of a first color and a region of a second color, and as shown inFIG. 4, the region of a tumor c, which is indicated by hatching, is expressed in the first color, and the region of a normal tissue other than the tumor c is expressed in the second color.

The first image processing unit34includes an image sensor341, a processor342, and an interpolation unit343. The first image processing unit34detects information of a subject to be captured by the first camera321a,generates an image signal, interpolates the generated image signal, and then overlaps a detection image Di of the third image processing unit36with the interpolated image to generate a left-eye image signal Li.

The image sensor341may be a charge-coupled device (CCD) or complementary metal-oxide-semiconductor (CMOS) sensor that detects information of a subject captured by the first camera321aand generates an electric signal. However, the embodiments are not limited thereto.

The processor342generates an image signal using the electric signal generated by the image sensor341. In this case, the processor342generates an image signal using the YCbCr color space composed of the luminance component Y and the chrominance components Cb and Cr. Image signals generated by the processor342are shown in Table 1 below.

The interpolation unit343measures the color temperature of the light source of the microscope10using the image created by the processor342, adjusts the white balance, and interpolates chrominance corresponding to red family colors of the image signal with preset reference chrominance using only the chrominance components Cb and CR rather than the luminance component Y to create a left-eye image Li. At this time, the reference chrominance is chrominance that corresponds to a predetermined light source color temperature and in which red family colors can be expressed without distortion.

For example, referring to Table 2 below, the interpolation unit343adjusts the white balance of the image created by the processor342and then interpolates image chrominance corresponding to red family colors of the image created by the processor342with reference chrominance components Br and Rr corresponding to color temperature T.

Therefore, by interpolating the color chrominance of the red family colors of the image signal with the reference chrominance corresponding to the color temperature of the light source, a left-eye image Li that represents red color may be created with a constant chrominance component Cr regardless of the luminance of the light source of the microscope10.

The second image processing unit35includes an image sensor351, a processor352, and an interpolation unit353. The second image processing unit35detects information of a subject to be captured by the second camera321b,generates an image signal, interpolates the generated image signal, and then overlaps a detection image Di of the third image processing unit36with the interpolated image to generate a right-eye image signal Ri.

The image sensor351, the processor352, and the interpolation unit353are substantially the same as the image sensor341, the processor342, and the interpolation unit343, respectively, and thus a detailed description thereof will be omitted.

The third image processing unit36includes an image sensor361and a processor362and creates a detection image Di including a boundary between a tumor and a normal tissue using the fluorescent image Fi.

As described with reference toFIG. 4, the image sensor361may be a CCD or CMOS sensor that detects a fluorescent image Fi, in which a tumor c shown by hatching is represented in the first color and a normal tissue other than the tumor c is represented in the second color, and that generates an electric signal. However, the embodiments are not limited thereto.

The processor362recognizes a region corresponding to the first color as a tumor c using the electric signal generated by the image sensor361, recognizes a region corresponding to the second color as a normal tissue, and creates a detection image Di including the boundary of the tumor.

Specifically, referring toFIG. 5, the processor362recognizes the boundary between the first color and the second color as the boundary of the tumor c and creates a detection image Di including the boundary cb of the tumor c.

Also, the processor362may apply a video analysis algorithm to the video, recognize the region corresponding to the first color as the tumor c, recognize the region corresponding to the second color as a normal tissue, and create the detection image Di. The video analysis algorithm, which is an example, may distinguish the tumor c and the normal tissue using at least one of the boundary (edge) of the tumor c, the color of the tumor c, and the change in surface color spectrum of the tumor c, recognize the boundary between the tumor c and the normal tissue, and create the detection image Di. Also, the processor362may recognize the tumor c and the normal tissue by applying a deep learning technology to the video, but the embodiments are not limited thereto.

Also, the processor362may use at least one of Sobel, Prewitt, Roberts, Compass, Laplacian, Laplacian of Gaussian (LoG) or Canny to recognize the boundary between the tissue c and the normal tissue and create the detection image Di.

The recorder40stores the left-eye image Li and the right-eye image Ri.

The display unit50includes a plurality of monitors51and52, and each of the plurality of monitors51and52displays a surgical image captured the left-eye image Li and the right-eye image Ri of the recorder40as a stereoscopic video.

In this way, a surgical site and even an adjacent site may be viewed as a stereoscopic video through the plurality of monitors51and52, and thus an assistant as well as an operator perform surgery through the monitors51and52without performing surgery through the surgical microscope10.

Also, conventionally, in order to check a fluorescent substance by selectively accumulating protoporphyrin IX in tumor tissues to emit light, it was possible to check tumors displayed as a fluorescent screen only by turning off the lighting of an operating room and using a microscope equipped with a filter. Also, such a fluorescent screen is composed of fluorescent substances and is represented in a color different from an original human tissue color.

However, according to embodiments, a stereoscopic video in which the boundary cb of a tumor is marked may be viewed through a plurality of monitors51and52, and thus it is possible to distinguish a normal tissue and a tumor through the plurality of monitors51and52at any time during surgery without turning off the lighting of an operating room. Also, the stereoscopic video in which the boundary cb of the tumor is marked is displayed in a unique human tissue color rather than being displayed in a fluorescent screen.

Although the embodiments of the present invention have been described in detail above, the scope of the present invention is not limited thereto but encompasses various modifications and improvements made by those skilled in the art using the basic concept of the present invention defined in the appended claims. Therefore, in all respects, the detailed description above should not be construed as restrictive and should be considered as illustrative. The scope of the present invention should be determined by reasonable interpretation of the appended claims, and all changes within the equivalent scope of the present invention are included in the scope of the present invention.

DESCRIPTION OF REFERENCE NUMERALS

1: Surgical video creation system

20: Optical adapter

30: Image processing device

50: Display unit