Image pickup system and signal processing apparatus

An image pickup system includes: a camera including a first image pickup unit and a second image pickup unit; and a processor configured to: define a first common area in a first image and a second common area in a second image, respectively; detect brightness of an image in the first common area; detect brightness of an image in the second common area; adjust brightness of the first image based on the brightness of the image in the first common area; and adjust brightness of the second image based on the brightness of the image in the second common area.

BACKGROUND OF INVENTION

1. Field of the Invention

The present invention relates to an image pickup system and a signal processing apparatus, and more particularly to an image pickup system and a signal processing apparatus that are capable of acquiring two picked-up images having parallax.

2. Description of the Related Art

Conventionally, an image pickup system capable of acquiring two picked-up images mutually having parallax, for example, an image pickup system configured to generate a three-dimensional image using two picked-up images having parallax has been known.

More specifically, an image pickup apparatus configured to pick up a three-dimensional image (3D image) using a plurality of image pickup portions is disclosed in Japanese Patent Application Laid-Open Publication No. 2012-065204 and Japanese Patent Application Laid-Open Publication No. 2001-148865. Each of the image pickup apparatus according to Japanese Patent Application Laid-Open Publication No. 2012-065204 and Japanese Patent Application Laid-Open Publication No. 2001-148865 is adapted to perform photometry of an image pickup signal outputted from each of two image pickup portions and control an exposure value of each image pickup portion based on a result of the photometry (a luminance value).

On the other hand, in specification of Japanese Patent No. 4955840, a three-dimensional endoscope configured to generate a three-dimensional image using two pick-up images having parallax in order to three-dimensionally observe a minute site of operation in a surgical operation in a body cavity is disclosed. The three-dimensional endoscope is provided with a pair of left and right observation optical systems and a pair of left and right image pickup portions corresponding to the observation optical systems.

Further, in specification of Japanese Patent No. 2716936, a three-dimensional endoscope configured to combine left and right image signals from two image pickup devices into one and perform image processing by one camera control unit is disclosed.

In the three-dimensional endoscopes as shown in the specification of Japanese Patent No. 4955840 and the specification of Japanese Patent No. 2716936, even though two image pickup portions are provided as described above, it is thought that one light source apparatus for illuminating an object in a body cavity is provided. That is, even if a light source of the light source apparatus is to be light-adjusted, it is thought that the number of light adjustment means is one.

SUMMARY OF THE INVENTION

An image pickup system of an aspect of the present invention includes: a camera including: a first image pickup unit configured to pick up an optical image of an object and generate a first image pickup signal representing a first image; and a second image pickup unit configured to pick up an optical image of the object and generate a second image pickup signal representing a second image having parallax relative to the first image; and a processor including hardware wherein the processor is configured to: define areas where an optical image of the object is picked up in common in the first image and the second image as a first common area in the first image and a second common area in the second image, respectively; detect brightness of an image in the defined first common area in the first image; detect brightness of an image in the defined second common area in the second image; adjust brightness of the first image based on the brightness of the image in the first common area; and adjust brightness of the second image based on the brightness of the image in the second common area.

A signal processing apparatus of an aspect of the present invention is a signal processing apparatus configured to process a first image picked up by a first image pickup unit and a second image picked up by a second image pickup unit, the second image having parallax relative to the first image, and the signal processing apparatus including: a processor including hardware wherein the processor is configured to: define areas where an optical image of an object is picked up in common in the first image and the second image as a first common area in the first image and a second common area in the second image, respectively; detect brightness of an image in the defined first common area; detect brightness of an image in the defined second common area; adjust brightness of the first image based on the brightness of the image in the first common area; and adjust brightness of the second image based on the brightness of the image in the second common area.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENT(S)

A configuration of an endoscope system of a first embodiment will be described with use ofFIGS. 1 to 4.FIG. 1is an external perspective view showing a whole configuration of the endoscope system of the first embodiment of the present invention;FIG. 2is a block diagram showing the configuration of the endoscope system of the first embodiment;FIG. 3is a front view of a distal end portion of an insertion portion of an endoscope in the endoscope system of the first embodiment; andFIG. 4is a cross-sectional view of the distal end portion of the insertion portion of the endoscope in the endoscope system of the first embodiment.

As shown inFIG. 1, main portions of an endoscope system1of the first embodiment of the present invention are configured with a three-dimensional endoscope2, which is a so-called 3D endoscope configured to generate a three-dimensional image by two image pickup units mutually having parallax, a processor3to which the three-dimensional endoscope2is attachably and detachably connected and which is configured to perform predetermined signal processing for image pickup signals from the image pickup units and is provided with a light source portion configured to supply illuminating light to the three-dimensional endoscope2, and a monitor5as a display device configured to display an image signal generated by the processor3as an endoscopic image.

Note that, when the endoscope system1of the present embodiment is in an operation room, various kinds of medical equipment including the processor3, for example, devices such as an electrosurgical knife device, a pneumoperitoneum device and a video recorder, and a gas cylinder filled with carbon dioxide are placed on a cart9.

In the present embodiment, the three-dimensional endoscope2is a rigid endoscope applied to three-dimensionally observe a site of operation in a body cavity, for example, applied to an abdominal cavity procedure. The three-dimensional endoscope2is provided with a rigid insertion portion6which is an insertion portion to be inserted into a body cavity and which has a length applied to the abdominal cavity procedure, an operation portion7configured to be grasped by an operator and perform various operations of the three-dimensional endoscope2, and a universal cord8extended from the insertion portion7and connected to the processor3.

The insertion portion6is configured with a distal end rigid portion11, a bending portion12and a rigid portion13coupled in that order from a distal end portion6aside toward a proximal end portion6bside. That is, a proximal end portion of the distal end rigid portion11is coupled with a distal end portion of the bending portion12, and a proximal end portion of the bending portion12is coupled with a distal end portion of the rigid portion13. Further, the rigid portion13is a rigid tube which is elongated and rigid, and a proximal end portion of the rigid portion13is coupled with the operation portion7as the proximal end portion6bof the insertion portion6.

Further, as shown inFIGS. 2, 3 and 4, a left side image pickup unit21L for left side image (left eye) and a right side image pickup unit21R for right side image (right eye) are arranged in the distal end rigid portion11of the insertion portion6in order to three-dimensionally observe a site of operation.

The left side image pickup unit21L is configured with an image pickup optical system22L for left side image (left eye) and an image pickup device23L, and the right side image pickup unit21R is configured with an image pickup optical system22R for right side image (right eye) and an image pickup device23R.

Further, as shown inFIG. 4, each of both the image pickup optical system22L for left side image and the image pickup optical system22R for right side image is configured with an objective lens for observing a site of operation and an image forming lens for forming an image of the site of operation observed by the objective lens.

Furthermore, at image forming positions of the image forming lenses of the respective image pickup optical system22L and the image pickup optical system22R, the image pickup device23L and the image pickup device23R are arranged, respectively. Each of the image pickup device23L and the image pickup device23R is configured, for example, with a CCD image sensor, and each of the image pickup device23L and the image pickup device23R is adapted to photoelectrically convert an image (a site of operation) formed by the image forming lens after being transmitted through the objective lens of the image pickup optical system22L or the image pickup optical system22R, and generate a predetermined image pickup signal.

Further, on subsequent stages of the image pickup device23L and the image pickup device23R, correlated double sampling circuits (hereinafter referred to as CDS circuits)26L,26R and analog/digital conversion circuits (hereinafter referred to as A/D conversion circuits)27L,27R are provided, respectively.

The image pickup device23L and the image pickup device23R photoelectrically convert object images formed on image pickup surfaces and output image pickup signals to the CDS circuits26L,26R, respectively. The CDS circuits26L,26R perform correlated double sampling processing for the image pickup signals and output the image pickup signals to the A/D conversion circuits27L,27R. The A/D conversion circuits27L,27R are adapted to convert the image pickup signals from analog signals to digital signals and output the digital signals to the processor3.

On the other hand, various cables15such as signal lines related to the image pickup device23L and the image pickup device23R are adapted to be connected to the processor3through insides of the insertion portion6, the operation portion7and the universal cord8. Further, a light guide cable16configured to transmit illuminating light from the light source portion of the processor3is inserted through the insertion portion6, the operation portion7and the universal cord8.

On a proximal end side of the universal cord8, a light source connector17a, which is an end portion of the light guide cable16, is arranged and is adapted to be attachably and detachably connected to a light source connector portion30aof the processor3. On the other hand, a signal connector17b, which is an end portion of the signal cables15, is arranged, branching from the light source connector17a, and is adapted to be attachably and detachably connected to a signal connector portion30bof the processor3.

Further, as shown inFIG. 3, on a distal end face11aof the distal end rigid portion11of the insertion portion6, an illumination window25is arranged at a position facing a distal end face of the light guide cable16. Note that, in the present embodiment, two light guide cables16are provided, and two illumination windows25are arranged accordingly.

The light source connector17ais connected to the light source connector portion30aof the processor3; illuminating light emitted from a light source portion4provided inside the processor3is transmitted via the light guide cables16; and the illuminating light is emitted from the illumination windows25provided facing the distal end face of the light guide cable16on the distal end face11aof the distal end rigid portion11.

On the other hand, on the distal end face11aof the distal end rigid portion11, two observation windows24L,24R are arranged adjacent to the illumination windows25so that an optical image of an illuminated object such as an affected part is inputted. Note that the observation windows24L,24R are arranged at positions facing the image pickup optical system22L and the image pickup optical system22R described above, respectively.

Further, as shown inFIG. 4, the three-dimensional endoscope2has a containing portion29L for left side configured to contain the left side image pickup unit21L and a containing portion29R for right side configured to contain the right side image pickup unit21R. Note that the containing portion29L and the containing portion29R are separate bodies, respectively.

Note that, though such a 3D endoscope is adopted that optical images mutually having parallax are inputted to the two left and right image pickup optical systems22L and22R to generate a left-side optical image and a right-side optical image, which are separate optical images, respectively, and the left and right separate optical images are photoelectrically converted by the separate image pickup device23L and the image pickup device23R, respectively, in the present embodiment as described above, this is not limiting.

That is, it is also possible to arrange one image pickup device at the image forming positions of the respective image forming lenses of the image pickup optical system22L and the image pickup optical system22R so that left and right separate optical images are formed in different areas on a same image pickup surface of the one image pickup device.

In this case, the processor3configured to perform signal processing of image pickup signals outputted from the one image pickup device can process the image pickup signals of the different areas on the same image pickup surface as left and right optical images.

Further, though CCD image sensors are adopted as the image pickup device23L and the image pickup device23R in the present embodiment, this is not limiting, and, for example, CMOS image sensors are also possible.

Next, the processor3in the endoscope system of the present first embodiment will be described in detail.

As shown inFIG. 2, the processor3in the present embodiment has a controlling portion31configured to control various circuits in the processor3; a first signal processing portion32configured to, when an image pickup signal generated by the image pickup device23L for left side image and passing through the CDS circuit26L and the A/D conversion circuit27L (hereinafter a first image pickup signal) between two image pickup signals in the three-dimensional endoscope2is inputted, perform predetermined signal processing under control of the controlling portion31; and a second signal processing portion33configured to, when an image pickup signal generated by the image pickup device23R for right side image and passing through the CDS circuit26R and the A/D conversion circuit27R (hereinafter a second image pickup signal) is inputted, perform predetermined signal processing under control of the controlling portion31.

Here, the controlling portion31performs control of the various circuits in the processor3and includes an area defining portion31aconfigured to define areas where an optical image of the object is picked up in common in the first image pickup signal representing a left side image and the second image pickup signal representing a right side image as a first common area in the left side image and a second common area in the right side image. Note that the area defining portion31awill be described in detail later.

Further, both of the first signal processing portion32and the second signal processing portion33perform predetermined signal processing for an image pickup signal from the image pickup device23L or the image pickup device23R. Each of the first signal processing portion32and the second signal processing portion33is provided with signal processing portions such as an automatic gain control circuit (AGC circuit), a white balance circuit, a gamma correction circuit, a magnification/reduction circuit and a contour emphasis circuit, which are publicly known, to perform signal processing appropriately.

Returning toFIG. 2, the processor3has a first light adjusting/detecting portion34configured to perform light adjustment/detection of the first image pickup signal processed by the first signal processing portion32, a second light adjusting/detecting portion35configured to perform light adjustment/detection of the second image pickup signal processed by the second signal processing portion33, and a 3D combining portion50configured to combine the first image pickup signal signal-processed by the first signal processing portion32and the second image pickup signal signal-processed by the second signal processing portion33to generate a predetermined 3D image signal.

Here, the first light adjusting/detecting portion34has a first photometry portion38configured to perform photometry of luminance of an image pickup signal which is the first image pickup signal processed by the first signal processing portion32under the control of the controlling portion31and which corresponds to the first common area in the left side image defined by the area defining portion31a, and output an information signal (first photometric value signal) about a first photometric value (first luminance value), and a first exposure time period controlling portion36configured to output a first exposure control signal for controlling an exposure time period for the image pickup device23L according to the first luminance value obtained by photometry by the first photometry portion38.

On the other hand, the second light adjusting/detecting portion35has a second photometry portion39configured to perform photometry of luminance of an image pickup signal which is the second image pickup signal processed by the second signal processing portion33and which corresponds to the second common area in the right side image defined by the area defining portion31a, and output an information signal (second photometric value signal) about a second photometric value (second luminance value), and a second exposure time period controlling portion37configured to output a second exposure control signal for controlling an exposure time period for the image pickup device23R according to the second luminance value obtained by photometry by the second photometry portion39.

The first exposure time period controlling portion36generates a control signal for electronic shutter control of the image pickup device23L according to the first photometric value obtained by performing photometry of the luminance of the first image pickup signal and sends out the control signal to the image pickup device23L. On the other hand, the second exposure time period controlling portion37generates a control signal for electronic shutter control of the image pickup device23R according to the second photometric value obtained by performing photometry of the luminance of the second image pickup signal and sends out the control signal to the image pickup device23R.

Furthermore, the processor3has a diaphragm control signal generating portion52configured to generate a diaphragm control signal according to the first photometric value signal and the second photometric value signal under the control of the controlling portion31.

The diaphragm control signal generating portion52is adapted to generate the diaphragm control signal according to the first photometric signal and the second photometric signal corresponding to the first common area in the left side image and the second common area in the right side image defined by the area defining portion31a, respectively, under the control of the controlling portion31.

Further, in the present embodiment, the processor3is internally provided with the light source portion4for emitting illuminating light to the light guide cable16in order to supply the illuminating light to the three-dimensional endoscope2.

The light source portion4is configured, being provided with a lamp42, a diaphragm43, a lens44and a light source controlling portion41. Illuminating light from the lamp42is emitted toward the lens44via the diaphragm43which is controlled by the light source controlling portion41. Further, the lens44is adapted to condense light to a proximal end portion of the light guide cable16.

Furthermore, the light condensed to the proximal end portion of the light guide cable16is emitted from a distal end portion of the light guide cable16after being transmitted through the light guide cable16, as illuminating light supplied to the three-dimensional endoscope2.

The light source controlling portion41is adapted to control the diaphragm43based on the diaphragm control signal generated by the diaphragm control signal generating portion52.

That is, in the present first embodiment, an amount of illuminating light is controlled by controlling the diaphragm43by the light source controlling portion41based on the diaphragm control signal generated according to the first photometric signal and the second photometric signal.

Thus, in the endoscope system of the present first embodiment, first, an exposure time period for an image pickup signal is controlled separately for the image pickup device23L and the image pickup device23R by the first light adjusting/detecting portion34and the second light adjusting/detecting portion35, respectively, as described above to perform control so that the luminance values of the first image pickup signal and the second image pickup signal are equalized.

On the other hand, the endoscope system of the present embodiment is adapted to perform light adjustment of a light source by generating a diaphragm control signal by the diaphragm control signal generating portion52according to the first photometric value signal or the second photometric value signal detected by the first photometry portion38and the second photometry portion39of the first light adjusting/detecting portion34and the second light adjusting/detecting portion35, and controlling the diaphragm43of the light source portion4by the diaphragm control signal.

<Process by Area Defining Portion>

FIG. 5is a diagram showing operation of the area defining portion in the endoscope system of the first embodiment; andFIG. 6is a diagram showing a 3D composite image generation process in a conventional three-dimensional endoscope system.

Here, description will be made on a light adjustment/detection process for left and right images at time of generating an ordinary 3D composite image with reference toFIG. 6.

Generally, on left and right images mutually having parallax (inFIG. 6, a left side image70L and a right side image70R) of a three-dimensional endoscope, a deviation occurs in a left and right direction on left and right ends of each of the left and right images due to the parallax as shown inFIG. 6.

Note that it is assumed that, inFIG. 6, the left side image70L and the right side image70R are 3D-combined, and a 3D composite image70S is generated and displayed on a predetermined monitor.

Further, inFIG. 6, reference numeral72in the left side image70L and the right side image70R denotes an optical image of an object to be picked up in common in left and right optical systems. Further, reference numeral74inFIG. 6denotes coordinate points schematically showing positions separated from the respective object images72in a same direction and by a same distance in the left side image70L and the right side image70R.

Here, it is assumed that, in the three-dimensional endoscope, light adjustment/detection is performed for each of the left side image70L and the right side image70R described above on the whole screen. Then, for example, as shown inFIG. 6, when a pair of forceps73is inserted from a right side end in both of the left side image70L and the right side image70R, a degree of advance of a distal end portion of the pair of forceps73at a right side edge portion looks different in the left side image70L and the right side image70R because the left side image70L and the right side image70R have parallax.

More specifically, an exposed part of the distal end portion of the pair of forceps73in the right side image70R looks relatively larger than that of the pair of forceps73in the left side image70L as shown inFIG. 6.

Since the distal end portion of the pair of forceps73as described above is generally configured with a member with a high reflectivity, it is assumed that such an object with a high reflectivity significantly influences light adjustment/detection.

Therefore, the right side image70R is detected relatively brighter than the left side image70L in the situation described above, and, if light adjustment control is performed to cause a whole 3D-combined screen to be of a target brightness, difference occurs in brightness of an object portion at an almost central part of the screen. More specifically, as for a central part of the screen, there is a possibility that the right side image70R becomes darker.

The invention of the present application has been made in view of such a point and provides an image pickup system capable of performing appropriate exposure control even if a large deviation occurs between luminance values of end portions in a left and right direction in two left and right field-of-view ranges, in a three-dimensional endoscope system capable of acquiring two left and right picked-up images having parallax.

A process for defining a common area by the area defining portion of the present embodiment will be described below with reference toFIGS. 2 to 5.

As described above, in the present embodiment, the controlling portion31of the processor3performs control of the various circuits in the processor3and has the area defining portion31a.

As shown inFIG. 5, the area defining portion31adefines areas where an optical image of the object is picked up in common in a first image pickup signal representing a left side image60L, which has been processed by the first signal processing portion32, and a second image pickup signal representing a right side image60R, which has been processed by the second signal processing portion33, as a first common area61L in the left side image60L and a second common area61R in the right side image60R.

Note that, inFIG. 5also, reference numeral62in the left side image60L and the right side image60R denotes an optical image of an object to be picked up in common in the left and right optical systems similar to inFIG. 6. Further, reference numeral64inFIG. 5denotes coordinate points schematically showing positions separated from the respective object images62in a same direction and by a same distance in the left side image60L and the right side image60R.

More specifically, the area defining portion31adefines areas in which object images picked up by the left-side image pickup unit21L and the right-side image pickup unit21R are displayed mutually in common in the left side image60L and the right side image60R as the first common area61L and the second common area61R, respectively.

<Light Adjustment/Detection Process in Common Area>

Further, in the present embodiment, the first common area61L and the second common area61R correspond to areas where light adjustment/detection is performed in the left side image60L and the right side image60R, respectively.

Then, under the control of the controlling portion31, the first photometry portion38of the first light adjusting/detecting portion34performs photometry of a luminance of an image pickup signal which is the first image pickup signal processed by the first signal processing portion32and which corresponds to the first common area61L in the left side image60L defined by the area defining portion31a, and outputs an information signal (first photometric value signal) about a first photometric value (first luminance value).

Further, the first exposure time period controlling portion36outputs a first exposure control signal for controlling an exposure time period to the image pickup device23L according to the first luminance value obtained by photometry by the first photometry portion38.

On the other hand, under the control of the controlling portion31, the second photometry portion39of the second light adjusting/detecting portion35performs photometry of luminance of an image pickup signal which is the second image pickup signal processed by the second signal processing portion33and which corresponds to the second common area61R in the right side image60R defined by the area defining portion31a, and outputs an information signal (second photometric value signal) about a second photometric value (second luminance value).

Further, the second exposure time period controlling portion37outputs a second exposure control signal for controlling an exposure time period to the image pickup device23R according to the second luminance value obtained by photometry by the second photometry portion39.

Then, the first exposure time period controlling portion36generates a control signal for electronic shutter control of the image pickup device23L according to the first photometric value obtained by performing photometry of the luminance of the first image pickup signal in the first common area61L and sends out the control signal to the image pickup device23L.

On the other hand, the second exposure time period controlling portion37generates a control signal for electronic shutter control of the image pickup device23R according to the second photometric value obtained by performing photometry of the luminance of the second image pickup signal in the second common area61R and sends out the control signal to the image pickup device23R.

Furthermore, the processor3has the diaphragm control signal generating portion52configured to generate a diaphragm control signal according to the first photometric value signal and the second photometric value signal under the control of the controlling portion31.

Further, the diaphragm control signal generating portion52generates a diaphragm control signal according to the first photometric signal and the second photometric signal corresponding to the first common area61L in the left side image and the second common area61R in the right side image, respectively, defined by the area defining portion31a, respectively, under the control of the controlling portion31.

After that, the image combining portion35of the present embodiment performs a process for combining the first image pickup signal representing the left side image60L, which has been processed by the first signal processing portion32, and the second image pickup signal representing the right side image60R, which has been processed by the second signal processing portion33, under the control of the controlling portion31to generate a 3D composite image60S as shown inFIG. 5, and outputs a composite image signal of the 3D composite image60S to the monitor5.

Here, in the endoscope system of the present embodiment configured as described above, suppose that, for example, a pair of forceps63is inserted from a right side end of each of both of the left side image60L and the right side image60R as shown inFIG. 5.

Even in such a situation, that is, even in a case where a degree of advance of a distal end portion of the pair of forceps63at a right side edge portion differs between mutual field-of-view ranges of the left side image60L and the right side image60R, and a reflectivity of the distal end portion of the pair of forceps63differs in the left side image60L and the right side image60R, an operation/effect is achieved in that brightness deviation between the left side image60L and the right side image60R is eliminated, and light adjustment is performed to obtain appropriate brightness in the endoscope system of the present embodiment because the first common area61L and the second common area61R, which are a common area, are defined as light adjustment/detection ranges in the left side image60L and the right side image60R.

As described above, according to the present embodiment, it is possible to provide an image pickup system and a signal processing apparatus that are capable of performing appropriate exposure control even if a large deviation occurs between luminance values of end portions in a left and right direction in two left and right field-of-view ranges, in a three-dimensional endoscope system capable of acquiring two left and right picked-up images having parallax.

Note that the present invention is not limited to the embodiment described above as it is, and it is possible to modify and embody components within a range not departing from the spirit of the invention at an implementation stage. Further, it is possible to form various aspects of the invention by an appropriate combination among the plurality of components disclosed in the above embodiment. For example, some components may be deleted from all the components shown in the embodiment. Furthermore, components among different embodiments may be appropriately combined.