Image processing apparatus, image processing method, and medical system to correct blurring without removing a screen motion caused by a biological body motion

To correct blurring without removing a screen motion caused by a biological body motion. [Solution] An image processing apparatus includes: a blurring amount estimation unit configured to estimate a blurring amount of a medical image including a biological body motion of a subject; and a blurring correction processing unit configured to perform blurring correction processing so as not to remove a screen motion caused by the biological body motion, on a basis of the blurring amount. This configuration makes it possible to correct blurring without removing a screen motion caused by a biological body motion. Thus, an optimum image for observing a biological body motion can be obtained.

CROSS REFERENCE TO RELATED APPLICATIONS

This application is a U.S. National Phase of International Patent Application No. PCT/JP2017/004432 filed on Feb. 7, 2017, which claims priority benefit of Japanese Patent Application No. JP 2016-065219 filed in the Japan Patent Office on Mar. 29, 2016. Each of the above-referenced applications is hereby incorporated herein by reference in its entirety.

TECHNICAL FIELD

The present disclosure relates to an image processing apparatus, an image processing method, and a medical system.

BACKGROUND ART

Conventionally, for example, Patent Literature described below describes a technology that assumes setting the strength of a correction degree of a position shift on the basis of manipulation situation information indicating a situation of an endoscopic device, and presenting, to a user, a moving image with moderately-suppressed blurring.

CITATION LIST

Patent Literature

DISCLOSURE OF INVENTION

Technical Problem

When a medical image is acquired by manipulating a medical device such as an endoscope, image burring corresponding to the manipulation of an operator is generated in some cases. On the other hand, in a medical image, due to the characteristics thereof, a motion caused by a biological body motion is generated in a screen in some cases.

The technology described in Patent Literature described above suppresses blurring on the basis of the manipulation situation information, but does not consider a motion caused by a biological body motion. Thus, if the technology tries to suppress blurring in accordance with a manipulation situation, a motion caused by a biological body motion is removed. This leads to such a problem that it becomes impossible to accurately observe a biological body motion.

In view of the foregoing, it has been demanded to correct blurring without removing a screen motion caused by a biological body motion.

Solution to Problem

According to the present disclosure, there is provided an image processing apparatus including: a blurring amount estimation unit configured to estimate a blurring amount of a medical image including a biological body motion of a subject; and a blurring correction processing unit configured to perform blurring correction processing so as not to remove a screen motion caused by the biological body motion, on a basis of the blurring amount.

In addition, according to the present disclosure, there is provided an image processing method including: estimating a blurring amount of a medical image including a biological body motion of a subject; and performing blurring correction processing so as not to remove a screen motion caused by the biological body motion, on a basis of the blurring amount.

In addition, according to the present disclosure, there is provided a medical system including: an imaging apparatus configured to capture a medical image including a biological body motion of a subject; and an image processing apparatus including a blurring amount estimation unit configured to estimate a blurring amount of the medical image, and a blurring correction processing unit configured to perform blurring correction processing so as not to remove a screen motion caused by the biological body motion, on a basis of the blurring amount.

Advantageous Effects of Invention

According to the present disclosure, it becomes possible to correct blurring without removing a screen motion caused by a biological body motion.

MODE(S) FOR CARRYING OUT THE INVENTION

1. Configuration Example of Endoscopic Operation System

2. Configuration Example of Image Processing Apparatus

3. Example of Determining Operative Method from Input Image

4. Separation of Blurring That Is Based on Frequency Analysis

5. Decision of Correction Gain Suitable for Subject Position

6. Application to Microscopic Device

1. Configuration Example of Endoscopic Operation System

In recent years, in the field site of medical treatment, endoscopic operation is performed in place of conventional open abdominal operation. For example, in the case of performing operation of an abdominal part, an endoscopic operation system1disposed in an operating room as illustrated inFIG. 1is used. Instead of opening an abdominal part by cutting an abdominal wall as in the conventional operation, hole-opening tools referred to as trocars12aand12bare attached to several locations on an abdominal wall, and an abdominoscope (hereinafter, also referred to as an endoscope)2, an energy treatment tool3, forceps4, and the like are inserted into a body from holes provided in the trocars12aand12b. Then, while viewing in real time an image of an affected area (tumor, etc.)16that has been captured as a video by the endoscope2, treatment of cutting off the affected area16by the energy treatment tool3or the like, and the like are performed. The endoscope2, the energy treatment tool3, and the forceps4are held by an operator, an assistant, a scopist, a robot, or the like.

In an operating room in which such endoscopic operation is performed, a cart14on which devices for endoscopic operation are mounted, a patient bed13on which a patient lies, a foot switch15, and the like are disposed. On the cart14, for example, devices such as a camera control unit (CCU)5, a light source device6, a treatment tool device7, a pneumoperitoneum device8, a display device9, a recorder10, and a printer11are placed as medical devices.

An image signal of the affected area16imaged by an image sensor through an observation optical system of the endoscope2is transmitted to the CCU5via a camera cable, subject to signal processing in the CCU5, and then, output to the display device9, on which an endoscopic image of the affected area16is displayed. The CCU5may be wirelessly connected to the endoscope2in addition to being connected to the endoscope2via the camera cable.

The light source device6is connected to the endoscope2via a light guide cable, and can emit, to the affected area16, light with various wavelengths while performing switching.

The treatment tool device7is a high frequency output device that outputs high frequency current to the energy treatment tool3that cuts off the affected area16using electrical heat, for example.

The pneumoperitoneum device8includes an air supply means and an air suction means, and supplies air into an abdominal region inside the body of the patient, for example.

The foot switch15controls the CCU5, the treatment tool device7, and the like using a foot manipulation performed by an operator, an assistant, or the like, as a trigger signal.

2. Configuration Example of Image Processing Apparatus

The present embodiment relates to a blurring correction technology for an endoscope, and relates to a technology of obtaining good image quality by appropriately suppressing a frequency component of blurring in accordance with an operative method of each clinical department that uses an endoscope.FIG. 2is a schematic diagram illustrating a configuration of an image processing apparatus1000included in the CCU5, and illustrates a basic configuration for blurring correction. As illustrated inFIG. 2, the image processing apparatus1000includes a feature point extraction unit1010, a motion vector extraction unit (ME: Motion Estimation)1020, an image burring amount estimation unit1030, a smoothing unit1040, a blurring correction unit1050, a user interface (UI)1060, and a filter group1070. The filter group1070is stored in a database.

An input image imaged by an image sensor of the endoscope2is input to the feature point extraction unit1010. The image sensor includes a complementary metal-oxide semiconductor (CMOS) sensor or the like, for example. The feature point extraction unit1010performs feature point extraction from each pixel of the input image. The motion vector extraction unit1020performs motion vector detection for the feature points. The image burring amount estimation unit1030estimates a blurring amount of the entire screen from a motion vector detection result of each feature point that has been obtained by the motion vector extraction unit1020. The smoothing unit1040accumulates blurring amounts in a time direction, and performs smoothing by applying a filter to the accumulated blurring amounts. The blurring correction unit1050performs blurring correction on the smoothed blurring amounts.

The estimation of the blurring amount that is performed by the image burring amount estimation unit1030can be performed by a method such as Random Sample Consensus (RANSAC), for example. In this case, samples of motion vectors are extracted at random, and samples with little errors among all the samples are counted. Then, by employing a coefficient having the largest number of samples with little errors, and the samples, a least-square method is performed using only effective components in a state in which error components are excluded, and a coefficient is estimated. The blurring amount of the entire screen can be thereby estimated from a motion vector of each feature point.

When blurring amounts in the time direction are accumulated by the smoothing unit1040, blurring frequencies are obtained. By applying a smoothing filter to the blurring frequencies, a smoothing unit104leaves only a desired frequency component, and removes unnecessary frequency components. For example, a bandpass filter (BPF) is used as the smoothing filter. This can leave, as a blurring correction target, only a frequency for which blurring is desired to be suppressed, and can avoid setting a frequency for which blurring is undesired to be suppressed, as a blurring correction target.

On the basis of the smoothed blurring amounts, the blurring correction unit1050performs blurring correction on the input image by a publicly-known method such as affine transformation, for example. Note that, a blurring correction processing unit according to the present embodiment includes the smoothing unit1040and the blurring correction unit1050.

In the configuration example illustrated inFIG. 2, a user selects an operative method mode in accordance with an operation scene, via the user interface (UI)1060.FIG. 3is a schematic diagram illustrating smoothing filters suitable for operative method modes, and frequency properties of the smoothing filters. In accordance with an operative method mode selected by the user, a smoothing filter having a frequency property of optimally suppressing blurring is selected from among the filter group1070, in accordance with an operative method of each clinical department (otorhinolaryngology, digestive surgery, orthopedic surgery) as illustrated inFIG. 3. The smoothing unit1040performs smoothing by the selected smoothing filter.

The smoothing filters illustrated inFIG. 3include a low pass filter (LPF), for example. In the case of the otorhinolaryngology, by the low pass filter, blurring with a frequency of 3 to 5 [Hz] or more is suppressed, and blurring with a frequency of less than 3 to 5 [Hz] is left. In the case of the digestive surgery, by the low pass filter, blurring with a frequency of 8 to 10 [Hz] or more is suppressed, and blurring with a frequency of less than 8 to 10 [Hz] is left. In the case of the orthopedic surgery, blurring with a frequency of 4 to 6 [Hz] or more is suppressed, and blurring with a frequency of less than 4 to 6 [Hz] is left.

Because devices such as the endoscope2that are used in operation vary depending on a difference in clinical department, correction target blurring included in a capture image varies for each clinical department. In the present embodiment, because it becomes possible to suppress blurring in accordance with an operative method of each clinical department, an optimum image with corrected blurring can be obtained for each clinical department.

3. Example of Determining Operative Method from Input Image

InFIG. 2, it is assumed that the user selects an operative method mode. Alternatively, an operative method can also be determined from an input image.FIG. 4Aillustrates an example of performing frequency analysis from a screen blurring amount of an input image, automatically determining a clinical department or a scene of an operative method that corresponds to the input image, from a result of the frequency analysis, and selecting a smoothing filter suitable for the scene, from the filter group1070. Thus, in the example illustrated inFIG. 4A, a frequency analysis unit1071and a scene determination unit1100are further included in addition to the configuration inFIG. 2.

FIG. 5is a schematic diagram illustrating a configuration of the frequency analysis unit1071in detail. As illustrated inFIG. 5, the frequency analysis unit1071includes a buffer1072, a fast Fourier transformation unit (FFT)1074, a frequency component extraction unit1076, a normalization unit1078, a filter selection signal generation unit1080, an integration unit1082, gain adjustment units1084,1086, and1088, and a multiplication unit1090.

As a screen blurring amount of an input image, a shift amount of the image is input to the frequency analysis unit1071from a screen blurring amount estimation unit1030. The shift amount is accumulated into the buffer1072, and then, frequency transformation is performed by the fast Fourier transformation unit (FFT)1074. By extracting frequency components by the frequency component extraction unit1076, it is determined which frequency is largely included, and normalization is performed by the normalization unit1078. Then, on the basis of a result of the normalization, a filter selection signal for selecting a filter is generated by the filter selection signal generation unit1080, and the filter selection signal is output.

In addition, the frequency analysis unit1071performs gain adjustment by the gain adjustment units1084,1086, and1088using adjustment values A, B, C, D, and E. The integration unit1082integrates shift amounts with a frame number N2that are accumulated in the buffer1072, and the gain adjustment unit1084reduces a gain as a value of integral becomes larger, on the basis of the integrated shift amounts.

In addition, on the basis of the shift amount, the gain adjustment unit1086reduces a gain as the shift amount becomes larger. In addition, in accordance with reliability of a blurring amount (motion reliability as to whether a unit matrix has been obtained, etc.) that has been estimated by the screen blurring amount estimation unit1030, the gain adjustment unit1088adjusts a gain so as to reduce the gain as the reliability becomes lower. Particularly in a case where the reliability is a certain value or less, the gain adjustment unit1088sets the gain to 0, and avoids performing blurring correction.

The gains calculated by the gain adjustment units1084,1086, and1088are multiplied by the multiplication unit1090, and are output to the blurring correction unit1050. The blurring correction unit1050adjusts the strength of blurring correction on the basis of the gains. By performing exceptional processing by the gain adjustment units1084,1086, and1088, blurring correction can be inhibited from being excessively performed.

The filter selection signal output by the frequency analysis unit1071is input to the scene determination unit1100. The scene determination unit1100determines a scene of an operative method on the basis of the filter selection signal, and selects an optimum smoothing filter suitable for the scene of the operative method, from among the filter group1070. The smoothing unit1040performs smoothing by the selected smoothing filter.

FIG. 4Billustrates an example of performing extraction of a color distribution and a spatial frequency distribution from an input image, automatically determining a clinical department or a scene of an operative method that corresponds to the input image, from these pieces of information, and selecting a smoothing filter suitable for the scene, from the filter group1070. Thus, in the example illustrated inFIG. 4B, a color distribution extraction unit1110, a spatial frequency distribution extraction unit1112, and the scene determination unit1100are further included in addition to the configuration inFIG. 2. The color distribution extraction unit1110extracts a color distribution of the input image. The spatial frequency distribution extraction unit1112extracts a spatial frequency distribution of the input image. On the basis of the color distribution and the spatial frequency distribution of the input image, the scene determination unit1100determines a clinical department or a scene of an operative method that corresponds to the input image, and selects an optimum smoothing filter suitable for the scene of the operative method, from among the filter group1070. The smoothing unit1040performs smoothing by the selected smoothing filter.

FIG. 4Cillustrates an example of performing detection of a mask diameter from an input image, automatically determining a clinical department or a scene of an operative method that corresponds to the input image, from information regarding the mask diameter, and selecting a smoothing filter suitable for the scene, from the filter group1070. Thus, in the example illustrated inFIG. 4C, a mask diameter extraction unit1114and the scene determination unit1100are further included in addition to the configuration inFIG. 2. A mask diameter extraction unit114detects a mask diameter from the input image. On the basis of the mask diameter, the scene determination unit1100determines a clinical department or a scene of an operative method that corresponds to the input image, and selects an optimum smoothing filter suitable for the scene of the operative method, from among the filter group1070. The smoothing unit1040performs smoothing by the selected smoothing filter. On the basis of the mask diameter, the feature point extraction unit1010can extract a feature point from a narrower range in a mask, as the mask diameter becomes smaller.

FIG. 4Dillustrates an example of detecting the presence or absence of trocars from an input image, automatically determining a clinical department or a scene of an operative method that corresponds to the input image, from these pieces of information, and selecting a smoothing filter suitable for the scene, from the filter group1070. Thus, in the example illustrated inFIG. 4D, a trocar presence/absence detection unit1116and the scene determination unit1100are further included in addition to the configuration inFIG. 2. On the basis of a screen blurring amount estimated by the screen blurring amount estimation unit1030, the trocar presence/absence detection unit1116detects whether or not the trocars12aand12bare used (the presence or absence of the trocars). On the basis of the presence or absence of the trocars12aand12b, the scene determination unit1100determines a clinical department or a scene of an operative method that corresponds to the input image, and selects an optimum smoothing filter suitable for the scene of the operative method, from among the filter group1070. The smoothing unit1040performs smoothing by the selected smoothing filter.

A determination method of a clinical department or an operative method can be performed in accordance with information such as a motion frequency analysis result of a moving image, operative method/scene recognition, information regarding a mask diameter, and the presence or absence of the trocars12aand12b, as illustrated in Table 1. As for the frequency analysis of a motion, in accordance with a peak position of a frequency component, a clinical department can be decided to be orthopedic surgery if the peak position is 3 to 5 [Hz], to be an otorhinolaryngology if the peak position is 4 to 6 [Hz], or to be digestive surgery if the peak position is 8 to 10 [Hz]. According to the configuration example illustrated inFIG. 4A, in accordance with a result of frequency analysis, a clinical department or a scene of an operative method can be determined on the basis of Table 1.

As for operative method/scene recognition performed by the extraction of a color distribution and a spatial frequency distribution that is illustrated inFIG. 4B, a clinical department is determined to be otorhinolaryngology in a case where red regions are largely included in the input image, the image is flat, and a spatial frequency is low, or to be orthopedic surgery in a case where white regions are largely included, the image is flat, and a spatial frequency is low, as illustrated in Table 1. Alternatively, in a case where various color components are largely included in the input image, there are textures in blood vessels and fat, and a spatial frequency is high, a clinical department is determined to be digestive surgery.

In the determination that is based on information regarding a mask diameter that is illustrated inFIG. 4C, the mask diameter extraction unit1114detects a mask diameter from the input image, and a clinical department is determined to be otorhinolaryngology if the mask diameter is about 4.9 mm, to be orthopedic surgery if the mask diameter is about 5.5 mm, or to be digestive surgery if the mask diameter is about 10 mm, as illustrated in Table 1. In this manner, because a mask diameter varies depending on the clinical department or the scene of the operative method, by detecting a mask diameter, a clinical department or a scene of an operative method can be determined.

In addition, in a case where the trocars12aand12bare used, because the trocars12aand12bserve as fulcrum points, many of the motions of the endoscope2become presupposed decided motions. Thus, a frequency of a screen blurring amount varies between a case where the trocars12aand12bare used, and a case where the trocars12aand12bare not used. Thus, a trocar presence/absence detection unit116inFIG. 4Dcan determine whether or not the trocars12aand12bare used, on the basis of a frequency of a screen blurring amount. As illustrated in Table 1, in a case where the trocars12aand12bare used, a clinical department can be determined to be digestive surgery, and in a case where the trocars12aand12bare not used, a clinical department can be determined to be orthopedic surgery or otorhinolaryngology.

Table 2 shown below indicates “blurring” desired to be suppressed, and a frequency thereof, and “blurring” undesired to be suppressed, and a frequency thereof, in each clinical department. Using a determination result, an optimum smoothing filter is applied to each of the “blurring” desired to be suppressed and the “blurring” undesired to be suppressed. In any clinical department, a smoothing filter is selected so as not to remove a screen motion caused by a biological body motion, by blurring correction. By selecting a smoothing filter in accordance with each clinical department or operative method on the basis of Table 2, blurring caused by factors such as the shake of an endoscopic scope and the vibration of a building or an arm is set as a target of blurring correction to be performed by the blurring correction unit1050, and a motion intended by an operator or a motion of a biological main body is prevented from being set as a target of blurring correction to be performed by the blurring correction unit1050. Note that, when these pieces of operative method information are transmitted to the CCU5, not only a cable but also a Radio Frequency Identification (RFID) or the like can also be used.

4. Separation of Blurring That Is Based on Frequency Analysis

In the configuration examples illustrated inFIGS. 4A, 4B, 4C, and 4D, blurring caused by an operator or a scopist, and blurring caused by a patient (beat, etc.) can be separated from each other on the basis of frequency analysis.FIG. 6Ais a property diagram illustrating frequencies of blurring caused by a beat, and blurring caused by a scopist. In addition,FIG. 6Bis a property diagram illustrating a property of a bandpass filter (BPF) for suppressing blurring caused by a scopist. In this manner, an arbitrary blurring frequency can also be suppressed using the bandpass filter (BPF) instead of a low pass filter (LPF) as illustrated inFIG. 3. In this case, a bandpass filter is stored in the filter group1070, and a bandpass filter is selected on the basis of frequency analysis. As the configuration, a configuration similar to those ofFIGS. 4A, 4B, 4C, and 4Dcan be employed.

5. Decision of Correction Gain Suitable for Subject Position

FIG. 7is a schematic diagram illustrating a configuration of acquiring focus lens position information of the endoscope2, and deciding a correction gain. In addition to the configurations inFIGS. 4A and 4B, gain adjustment for blurring correction is performed in accordance with a distance between the endoscope2and a subject. Thus, in the configuration illustrated inFIG. 7, a focus lens position acquisition unit1120, a correction gain decision unit1122, and a gain adjustment unit1123are included in addition to the configuration inFIG. 4B.

In the configuration example illustrated inFIG. 7, focus lens position information of the endoscope2is acquired, and a correction gain is decided.FIG. 8is a schematic diagram illustrating a focus lens position. Because a relationship between a distance to a subject and a focus lens position is unambiguously defined, by adjusting a gain on the basis of the focus lens position information, it becomes possible to perform gain adjustment suitable for a distance to the subject.FIG. 9is a schematic diagram illustrating a relationship between a focus lens position and a correction gain (correction parameter). The focus lens position acquired by the focus lens position acquisition unit1120is transmitted to the correction gain decision unit1122, and the correction gain decision unit1122decides a correction gain on the basis of the property inFIG. 9.

InFIG. 9, it is assumed that a distance to the subject becomes larger as the focus lens position becomes larger. Thus, as a distance to the subject becomes larger, a value of a correction gain becomes larger. Because the distortion of the image that is caused by the shake of the endoscope2becomes larger as a distance to the subject becomes larger, the strength of blurring correction is enhanced by increasing a value of a correction gain as a distance to the subject becomes larger. It thereby becomes possible to optimally correct blurring in accordance with a distance to the subject. Note that, gain adjustment may be consecutively performed as indicated by a broken line inFIG. 9, or on/off may be controlled by discretely changing a correction gain as indicated by a solid line inFIG. 9.

FIG. 10is a schematic diagram illustrating a configuration example of acquiring a distance between the endoscope2and the subject on the basis of a spatial frequency of an image. Thus, in the configuration illustrated inFIG. 10, a spatial frequency analysis unit1124, a correction gain decision unit1126, and the gain adjustment unit1123are included in addition to the configuration inFIG. 4B. The spatial frequency analysis unit1124analyzes a spatial frequency of the image. In a case where the endoscope2comes close to the subject, a spatial frequency of an image to be obtained becomes low, and in a case where the endoscope2goes away from the subject, the spatial frequency becomes higher.FIG. 11is a schematic diagram illustrating a relationship between a spatial frequency and a correction gain (correction parameter). The spatial frequency analyzed by the spatial frequency analysis unit1124is transmitted to the correction gain decision unit1126, and the correction gain decision unit1126decides a correction gain on the basis of the property inFIG. 11. Note that, gain adjustment may be consecutively performed as indicated by a broken line inFIG. 11, or on/off may be controlled by discretely changing a correction gain as indicated by a solid line inFIG. 11.

As illustrated inFIG. 11, as a spatial frequency becomes lower, that is to say, as the subject comes close, a correction gain becomes lower. In addition, as a spatial frequency becomes higher, that is to say, as the subject goes away, a correction gain becomes higher. As described above, because the distortion of the image that is caused by the shake of the endoscope2becomes larger as a distance to the subject becomes larger, the strength of blurring correction is enhanced by increasing a value of a correction gain as a distance to the subject becomes larger. It thereby becomes possible to optimally correct blurring in accordance with a distance to the subject.

FIG. 12is a schematic diagram illustrating a configuration example of acquiring a distance between the endoscope2and the subject in a stereo view, in a case where the endoscope2performs two-lens stereo shooting. Thus, in the configuration illustrated inFIG. 12, a distance information detection unit1128, a correction gain decision unit1130, and the gain adjustment unit1123are included in addition to the configuration inFIG. 4B. The distance information detection unit1128detects a distance to the subject on the basis of disparity of a stereo image.FIG. 13is a schematic diagram illustrating a relationship between a distance to the subject, and a correction gain (correction parameter). A distance to the subject that has been detected by the distance information detection unit1128is transmitted to the correction gain decision unit1130, and the correction gain decision unit1130decides a correction gain on the basis of the property inFIG. 13. Note that, gain adjustment may be consecutively performed as indicated by a broken line inFIG. 13, or on/off may be controlled by discretely changing a correction gain as indicated by a solid line inFIG. 13.

Also inFIG. 13, as the subject comes close, a correction gain becomes lower. In addition, as the subject goes away, a correction gain becomes higher. As described above, because the distortion of the image that is caused by the shale of the endoscope2becomes larger as a distance to the subject becomes larger, the strength of blurring correction is enhanced by increasing a value of a correction gain as a distance to the subject becomes larger. It thereby becomes possible to optimally correct blurring in accordance with a distance to the subject.

FIG. 14is a schematic diagram illustrating a configuration example of a case of performing electronic zoom after blurring correction. In the configuration illustrated inFIG. 14, an electronic zoom1140, a correction gain decision unit1142, a user interface (UI)1144, and the gain adjustment unit1123are provided in addition to the configuration inFIG. 4B. The correction gain decision unit1142decides a correction gain in accordance with a zoom magnification of the electronic zoom1140.FIG. 15is a property diagram illustrating a relationship between a zoom magnification and a correction gain. The correction gain decision unit1142decides a correction gain on the basis ofFIG. 15. As illustrated inFIG. 15, as a zoom magnification becomes larger, a value of a correction gain increases.

Because the subject is magnified more as a zoom magnification becomes larger, the distortion of the image that is caused by the shake of the endoscope2becomes larger. Thus, by increasing a correction gain as a zoom magnification becomes larger, it becomes possible to optimally correct blurring. Note that, gain adjustment may be consecutively performed as indicated by a broken line inFIG. 15, or on/off may be controlled by discretely changing a correction gain as indicated by a solid line inFIG. 15. Note that, the focus lens position acquisition unit1120, the spatial frequency analysis unit1124, the distance information detection unit1128, and the electronic zoom1140that have been described above correspond to a distance-related parameter acquisition unit that acquires a distance-related parameter related to a distance to the subject.

6. Application to Microscopic Device

FIG. 16is a schematic diagram illustrating a configuration example of performing blurring correction of an image captured by a microscopic device (operative microscope). In the above-described examples, the description has been given of blurring correction of an image captured by the endoscope2. Alternatively, the blurring correction can be similarly applied to an image captured by a microscopic device. The configuration illustrated inFIG. 16is similar toFIG. 4Aexcept that an input image is a microscopic image. The microscopic device generally includes a microscopic portion for enlarging and observing an observation target (operated portion of a patient), an arm portion that supports the microscopic portion at a distal end, and a base portion that supports a proximal end of the arm portion. The microscopic portion is an electronic image-capturing-type microscopic portion (so-called video-type microscopic portion) that electrically captures a captured image by an imaging apparatus (image sensor). In the case of operation that uses the microscopic device, in some cases, difficulty is felt in seeing a video-shot image due to the shake of a building or an arm. Thus, for example, at the time of craniotomy operation in cranial nerve surgery, similarly to the case of the endoscope2, the feature point extraction unit1010extracts a feature point from an input image, the motion vector extraction unit1020performs motion vector detection for each feature point, and the image burring amount estimation unit1030estimates a blurring amount of the entire image. Then, the smoothing unit1040accumulates blurring amounts in the time direction, and performs smoothing by applying a filter. The blurring correction unit1050performs blurring correction on the smoothed blurring amounts. The correction of blurring caused by the vibration of a building or an arm can be thereby performed. Also in the operative microscope, “blurring” desired to be suppressed and “blurring” undesired to be suppressed are assumed. As indicated in Table 2, the “blurring” desired to be suppressed corresponds to a frequency of about 0 to 1 [Hz] and a frequency of about 8 to 9 [Hz]. In addition, the “blurring” undesired to be suppressed corresponds to a frequency of about 0 or 1 to 8, or 9 [Hz].

In blurring correction for the microscopic device, for performing blurring correction with higher accuracy, an arm vibration sensor1132is installed on an arm. Additionally, for detecting the vibration of a building, a floor vibration sensor1134is installed on a floor, and a ceiling vibration sensor1136is installed on a ceiling.FIG. 17is a schematic diagram illustrating an example in which the configuration inFIG. 16is provided with a frequency filter generation unit1138that generates a frequency filter for suppressing each eigen vibration component, using vibration information from each sensor, as an input. In addition,FIG. 18is a schematic diagram illustrating a property of a frequency filter1138(bandpass filter). By the smoothing unit104performing smoothing processing using the frequency filter1138illustrated inFIG. 18, it is possible to set eigen vibration components of arm vibration, floor vibration, and ceiling vibration that are respectively detected by the arm vibration sensor1132, the floor vibration sensor1134, and the ceiling vibration sensor1136, as targets of blurring correction to be performed by the blurring correction unit1050, and suppress blurring caused by these.

FIG. 19is a schematic diagram illustrating a configuration example of a case of obtaining a noise reduction (NR) effect on a moving image, by performing smoothing in a time direction filter according to the present embodiment. In the configuration example illustrated inFIG. 19, a frame position correction unit1145and a frame addition unit1146are added in addition to the configuration inFIG. 4B. Similarly toFIG. 4B, a color distribution and a spatial frequency distribution are extracted from an input image, a filter is selected after scene determination is performed, and smoothing is performed. By a frame position correction unit1144performing position correction on each frame, and the frame addition unit1146adding each frame, the noise reduction effect can be obtained on the moving image.

As described above, according to the present embodiment, because blurring correction is performed in accordance with a situation on the basis of a clinical department or a scene suitable for an operative method, blurring correction can be performed without removing a screen motion caused by a biological body motion. It therefore becomes possible to provide an optimum medical image suitable for a situation.

Additionally, the present technology may also be configured as below.(1)An image processing apparatus including:a blurring amount estimation unit configured to estimate a blurring amount of a medical image including a biological body motion of a subject; anda blurring correction processing unit configured to perform blurring correction processing so as not to remove a screen motion caused by the biological body motion, on a basis of the blurring amount.(2)The image processing apparatus according to (1),in which the blurring correction processing unit includesa smoothing unit configured to smooth the blurring amount by accumulating the blurring amount in a time direction, anda blurring correction unit configured to perform blurring correction on a basis of the smoothed blurring amount, andthe blurring correction processing unit performs the blurring correction processing in accordance with a situation, by changing a smoothing filter for the smoothing.(3)The image processing apparatus according to (2), in which the blurring correction processing unit performs the blurring correction processing in accordance with a situation in which operation is being performed, an operative method of the operation, or a clinical department.(4)The image processing apparatus according to (2), in which the blurring correction processing unit performs the blurring correction processing in accordance with a user instruction.(5)The image processing apparatus according to (2), includinga frequency analysis unit configured to perform frequency analysis of the blurring amount,in which the blurring correction processing unit performs the blurring correction processing on a basis of a result of the frequency analysis.(6)The image processing apparatus according to (2), includinga color distribution extraction unit configured to extract a color distribution of a medical image,in which the blurring correction processing unit performs the blurring correction processing on a basis of the color distribution.(7)The image processing apparatus according to (2), includinga spatial frequency distribution extraction unit configured to extract a spatial frequency distribution of a medical image,in which the blurring correction processing unit performs the blurring correction processing on a basis of the spatial frequency distribution.(8)The image processing apparatus according to (2), includinga mask diameter detection unit configured to detect a mask diameter of a medical image, in which the blurring correction processing unit performs the blurring correction processing on a basis of the mask diameter.(9)The image processing apparatus according to (2), includinga trocar presence/absence detection unit configured to detect presence or absence of use of a trocar in operation,in which the blurring correction processing unit performs the blurring correction processing on a basis of the presence or absence of the use of the trocar.(10)The image processing apparatus according to any one of (1) to (9), further including:a distance-related parameter acquisition unit configured to acquire a distance-related parameter related to a distance to a subject; anda gain adjustment unit configured to adjust a gain of the blurring correction processing on a basis of the distance-related parameter.(11)The image processing apparatus according to (10), in which the gain adjustment unit increases the gain as a distance to a subject becomes larger.(12)The image processing apparatus according to (10), in which the distance-related parameter is focus lens position information, a spatial frequency of a subject, disparity in a stereo image, or a zoom magnification.(13)The image processing apparatus according to any one of (1) to (12), including:a feature point extraction unit configured to extract a feature point from the medical image; anda motion vector extraction unit configured to extract a motion vector from the feature point,in which the blurring amount estimation unit estimates the blurring amount on a basis of the motion vector.(14)The image processing apparatus according to (13), in which, on a basis of a mask diameter of the medical image, the feature point extraction unit extracts the feature point from a narrower range in a mask as the mask diameter becomes smaller.(15)The image processing apparatus according to (2), in which the medical image is an endoscopic image or a microscopic image.(16)The image processing apparatus according to (15), in which the medical image is a microscopic image, and the image processing apparatus includes a filter generation unit configured to generate the smoothing filter for removing blurring caused by external vibration.(17)The image processing apparatus according to (16), in which the filter generation unit generates the smoothing filter on a basis of a detection value of a sensor configured to detect the external vibration.(18)An image processing method including:estimating a blurring amount of a medical image including a biological body motion of a subject; andperforming blurring correction processing so as not to remove a screen motion caused by the biological body motion, on a basis of the blurring amount.(19)A medical system including:an imaging apparatus configured to capture a medical image including a biological body motion of a subject; andan image processing apparatus includinga blurring amount estimation unit configured to estimate a blurring amount of the medical image, anda blurring correction processing unit configured to perform blurring correction processing so as not to remove a screen motion caused by the biological body motion, on a basis of the blurring amount.

REFERENCE SIGNS LIST