Patent Description:
The wider the field of view (FOV) of an endoscopic optical system, the more visual information is presented to the surgeon. Uniformly illuminating the scene viewed by a wide FOV endoscope is challenging because light output tends to fall-off towards the perimeter of the field of view. Traditional autoexposure algorithms calculate luma using a center area of a scene as the center point. However, in a wide FOV application, the user may choose to digitally zoom-in or center-in on an area towards the periphery of the scene to better view a region of interest. If so, then given the light and optical properties at the edge of the scene/scope, the user may experience poor image quality (darker than normal) and poor responsiveness due to light changes at the periphery that would not adjust the overall scene exposure. Additionally, the exposure levels of the scene in the region of interest may be different than for the wide FOV as a whole. The lack of proper illumination and exposure response may result in an image that is distracting to the surgeon, may cause eye fatigue, and, may potentially lead to misidentification of tissue and anatomy. <CIT> al discloses an auto exposure of a camera in a surgical robot. <CIT> discloses an exposure control method and system for an image capture device. <CIT> discloses an imaging device and camera body. <CIT> discloses an endoscope field stop encoding system and method.

There exists a need for an improved autoexposure system that remedies the shortcomings of the prior art.

The present invention relates to an endoscopic camera system according to claim <NUM> and the corresponding method according to claim <NUM>. In an implementation, the luminance value is a weighted sum of at least one of: an average green intensity, an average red intensity, and an average blue intensity in the region of the interest. The luminance value may be a weighted sum of an average green intensity in the region of the interest. Adjusting the exposure may include adjusting at least one of an exposure time, a light source intensity, a gain, a sensitivity, and a variable aperture.

Optionally, the camera has a longitudinal axis and captures the image at a non-zero angle to the longitudinal axis. The camera may capture the image at a capture angle of <NUM> degrees relative to the longitudinal axis. In an implementation, the user input device is usable to select a region of interest having an apparent capture angle that is different than the actual capture angle. The camera may capture and output an image having a field of view greater than <NUM> degrees. In an implementation, the camera captures and outputs an image having a field of view of about <NUM> degrees. The user input device may be usable to select a region of interest having an apparent field of view that is smaller than the image field of view.

In an implementation the image has an image center and an orientation indicator; the region of interest has a region of interest center positioned at a fixed distance from the image center; and the camera controller changes the region of interest center based on changes in position of the orientation indicator. Adjusting the exposure may further comprise adjusting a gain of at least some pixels within the region of interest. Optionally, the gain adjustment is not uniform across all of the pixels within the region of interest. The pixel gains may be adjusted using a gradient depending on a position of the pixels from a center of the image.

According to an implementation, a method of adjusting an exposure for an imaging system, comprises: receiving an image from a camera; receiving a region of interest in the image from a user, the region of interest being a sub-part of the image; computing a measured luminance value for the region of interest; and adjusting an exposure in response to a comparison of the measured luminance value with a target luminance value. Computing the measured luminance value may further comprise computing a weighted sum of an average green intensity, an average red intensity, and an average blue intensity in the region of the interest. In an implementation, computing the measured luminance value further comprises computing a weighted sum of an average green intensity in the region of interest. Adjusting the exposure may further comprise adjusting at least one of an exposure time, a light source intensity, a gain, a sensitivity, and a variable aperture.

The image may be captured at an actual capture angle greater than zero relative to a longitudinal axis and the received region of interest may have an apparent capture angle that is different than the actual capture angle. In an implementation, the image has a field of view greater than <NUM> degrees and the received region of interest has an apparent field of view that is less than the image field of view.

In an implementation, the image further comprises an image center and an orientation indicator; the region of interest has a region of interest center positioned at a fixed distance from the image center; and the method further comprises calculating a new region of interest center based on a change in a position of the orientation indicator. Adjusting the exposure may further comprise adjusting a gain of at least some pixels within the region of interest. Optionally, the gain adjustment is not uniform across all of the pixels within the region of interest. Optionally, pixel gains are adjusted using a gradient depending on a position of the pixels from a center of the image.

These and other features are described below.

The features, aspects and advantages of the present invention will become better understood with regard to the following description, appended claims and accompanying figures wherein:.

In the following description of the preferred implementations, reference is made to the accompanying drawings which show by way of illustration specific implementations in which the invention may be practiced.

With reference to <FIG>, an endoscopic camera system <NUM> according to an implementation has a camera <NUM>. The camera <NUM> has a shaft <NUM> couplable to a handpiece <NUM>. The handpiece <NUM> may have an input device <NUM>, such as buttons, switches or dials. The handpiece <NUM> is connectable to a camera controller <NUM> ("CCU" or "camera controller"). The handpiece <NUM> and the camera controller <NUM> may be connected via wire to facilitate data transfer between the camera and the camera controller. The camera <NUM> and the camera controller <NUM> may also be wirelessly connected to facilitate data transfer, such as via IEEE <NUM>. 11b or IEEE <NUM>. 11n or ultra-wide band (UWB). The camera controller <NUM> may be connectable to at least one input device <NUM> such as a mouse, keyboard, touchpad, or touchscreen monitor. Additionally, the camera controller <NUM> may be connectable to a display <NUM> and a storage device <NUM>, such as for storing images.

An image sensor <NUM> may be positioned inside the shaft <NUM> and proximal to a distal tip <NUM> of the shaft <NUM>. The image sensor <NUM> may be, for example, a charge couple device (CCD) or complementary metal oxide semiconductor (CMOS). Optics, such as a wide angle lens <NUM>, direct light to the image sensor <NUM>.

The position of the image sensor <NUM> and optics may provide a field of view approximately along a longitudinal axis <NUM> of the shaft <NUM> (a capture angle of approximately <NUM> degrees relative to the longitudinal axis) so that the image field is directly in front of the distal tip of the shaft. In some implementations, the optics may provide an image at a non-zero capture angle relative to the longitudinal axis of the shaft <NUM>. For example, the capture angle may be about <NUM> degrees or about <NUM> degrees relative to the longitudinal axis. As shown in <FIG>, in an implementation, the optics may provide an image along an image axis <NUM> with a capture angle of about <NUM> degrees relative to the longitudinal axis <NUM> of the shaft <NUM>. Additionally, the camera <NUM> may be coupled to a light source <NUM>. The light source <NUM> may be inside of the camera <NUM>.

The light source <NUM> includes a lamp. The lamp may be, for example, a semiconductor light source such as laser or LED to illuminate the field of view. The light source <NUM> is configured to appropriately illuminate the field of view of the video camera. Further, the light generated as well as camera sensitivity may extend beyond the visible spectrum. The illumination may be intended to excite fluorescence directly in a target, or in a fluorescent substance such as indocyanine green, that is then sensed by the camera. For example, the light source <NUM> might produce illumination in the near infrared (NIR) range and the camera sense the fluorescence at a longer IR wavelength. The illumination and camera sensitivity could extend from UV to NIR continuously or be composed of separate narrow bands.

Referring to <FIG>, the camera controller <NUM> is preferably a programmable unit containing sufficient processing capacity to accommodate a wide range of control, user interface and image acquisition/processing functions. The camera controller <NUM> has a processor <NUM> that runs program applications providing for a variety of capabilities. For instance, an image capture and display capability allows for both display of a live feed of an image through the display <NUM> coupled to the camera controller <NUM>, as well as image capture. Captured images may be stored, such as in an internal storage device <NUM> or external storage device <NUM>, or transmitted to other devices.

Timing in video cameras must be very precise and consistent. A processor field programmable gate array (FPGA) <NUM> may be used to control and process the output from the image sensor <NUM>. Although other controllers may be used, use of one or more FPGAs for processing video images allows the system to achieve the precise timing needed to generate a standard video output signal. User interface logic and possible external network connectivity might be performed by software running on the processor <NUM>.

In an implementation, analog RGB data is transmitted from the image sensor <NUM> to the camera controller <NUM>. The Analog RGB data passes through an Analog/Digital converter <NUM> to the processor FPGA <NUM> where the video is processed. The processed video is then passed to a video output that may include a formatter FPGA <NUM> where the video is formatted into various display formats. The formatter FPGA <NUM> may also overlay information, such as patient and/or doctor information, onto the video. The formatted video may be converted back to an analog signal for display. The formatted video is sent to the display <NUM> and/or the storage device <NUM>. Alternatively, an Analog/Digital converter may be located in the camera head and digital RGB data transmitted from the camera head <NUM> to the camera controller <NUM>. Additionally, the image sensor <NUM> itself may include an Analog/Digital converter.

The camera controller <NUM> issues commands to the camera <NUM> to adjust its operating characteristics, and the camera <NUM> may send confirmation to the camera controller <NUM> that the camera received the commands. The processor FPGA <NUM> and/or the processor <NUM> may communicate with a shutter driver either in the camera controller <NUM> or the camera <NUM> to control an exposure period of the image sensor <NUM>. Additionally, the processor FPGA <NUM> and/or the processor <NUM> communicates with the light source <NUM> to control the drive current to the lamp of the light source <NUM>.

As shown in <FIG> and <FIG>, the wide angle lens <NUM> allows for a wide angle image <NUM>. The image may have a field of view that is greater than about <NUM> degrees and more preferably greater than about <NUM> degrees. Using an input device, such as the camera input device <NUM> or the camera controller input device <NUM>, a user may select a region of interest <NUM> within the wide angle image <NUM>. A user may select a region of interest <NUM> as desired, such as for example to magnify a portion of the wide angle image <NUM> or to simulate an apparent capture angle that is different than the actual capture angle. For example, and without limitation, a user may select an apparent capture angle of <NUM> degrees, <NUM> degrees or <NUM> degrees.

For example, as shown in <FIG>, the camera may be configured to take a wide angle image <NUM> along an image axis <NUM> with a capture angle of about <NUM> degrees relative to the longitudinal axis <NUM>. However, a user may select a region of interest <NUM> to simulate an endoscopic camera taking an image with a capture angle of about <NUM> degrees relative to the longitudinal axis. The region of interest has a center <NUM> and an area <NUM>. Once the region of interest has been identified, the center <NUM> and area <NUM> are used for autoexposure correction as explained below. In some instances the region of interest will be circular and the area <NUM> will be calculated based on a radius of the region of interest. However, the region of interest may have different shapes.

As shown in <FIG>, as the shaft <NUM> is rotated from a first position to a second position, an image center <NUM> does not change, but the region of interest <NUM> rotates along with the shaft. Once a region of interest <NUM> has been selected, the position of the region of interest relative to the image center <NUM> is known. The orientation of the scope is obtained from an orientation indicator <NUM>, which is a shape in a stop mask. As seen in <FIG>, the orientation indicator <NUM> rotates along with the scope and may be used to track and update the position of the region of interest <NUM>. The exposure of the region of interest is automatically updated as the scope is rotated.

With reference to <FIG>, the camera controller <NUM> receives an image from the camera <NUM>, step <NUM>. The camera controller <NUM> further receives a region of interest identification from a user, the region of interest being a sub-part of the image, step <NUM>. One the controller has received a region of interest identification from a user, including a center and area of the region of interest, the camera controller <NUM> computes the measured luminance value for the region of interest, step <NUM>. A measured luminance value may be obtained by computing a weighted sum of at least one of an average green intensity, an average red intensity, and an average blue intensity in the region of interest. In a preferred implementation, the measured luminance value is obtained by computing a weighted sum of an average green intensity in the region of interest.

Once the measured luminance value has been computed, the measured luminance value is compared to a target luminance value, step <NUM>. In an implementation, the target luminance value is adjustable by a user, such as by using the camera input device <NUM> or the camera controller input device <NUM>.

Depending on the comparison, the camera controller adjusts exposure to move the measured luminance value closer to the target luminance value, step <NUM>. The camera controller may adjust exposure by adjusting one or more of several variables depending on the configuration of the camera and how different the measured luminance value is from the target luminance value. For example, the camera controller <NUM> may adjust an exposure time of the image sensor <NUM>. If the measured luminance value is lower than the target luminance value, then the camera controller <NUM> may increase the exposure time to increase the measured luminance value. If the measured luminance value is higher than the target luminance value, then the camera controller <NUM> my decrease the exposure time.

Additionally, the camera controller <NUM> may adjust an intensity of the light source <NUM>. If the measured luminance value is lower than the target luminance value, then the camera controller <NUM> my increase the intensity of the light source time to increase the measured luminance value. If the measured luminance value is higher than the target luminance value, then the camera controller <NUM> my decrease the intensity of the light source.

Additionally, the camera controller <NUM> may adjust a digital gain applied to the acquired image. If the measured luminance value is lower than the target luminance value, then the camera controller <NUM> my increase the digital gain applied to the acquired image to increase the measured luminance value. If the measured luminance value is higher than the target luminance value, then the camera controller <NUM> my decrease the digital gain.

Additionally, if the camera has a variable aperture controlling the amount of light reaching the image sensor <NUM>, then the camera controller may control the variable aperture to alter the amount of light reaching the image sensor. If the measured luminance value is lower than the target luminance value, then the camera controller <NUM> my increase the aperture size to allow more light to reach the image sensor <NUM> to increase the measured luminance value. If the measured luminance value is higher than the target luminance value, then the camera controller <NUM> my decrease the aperture size to allow less light to reach the image sensor. Additionally, the camera controller <NUM> may adjust the sensitivity of the image sensor <NUM>.

The camera controller <NUM> may need to adjust multiple parameters. For example, if the camera controller <NUM> has already increased the exposure time to a maximum possible exposure time and the measured luminance value is still less than a target luminance value, then the camera controller may adjust another parameter, such as digital gain, to further increase measured luminance value.

In an implementation, known camera spatial lighting characteristics are further considered in adjusting exposure, and exposure correction is non-uniform across the region of interest. For example, if a camera <NUM> is known to have reduced light toward a periphery of a wide angle image, a gradient may be calculated and applied to alter autoexposure settings. If a selected region of interest includes pixels near the periphery of the wide angle image, then pixels within the region of interest nearer to the periphery may be provided with increased digital gain. In an implementation, a gradient is applied depending on how far each pixel of the region of interest is from the center of the wide angle image.

Claim 1:
An endoscopic camera system (<NUM>) comprising:
a camera (<NUM>) that captures and outputs an image;
a camera controller (<NUM>) coupled to the camera (<NUM>); and
a user input device coupled to the camera (<NUM>) or the camera controller (<NUM>), wherein the user input device is usable to select a region of interest (<NUM>) in the image, the region of interest (<NUM>) being a sub-part of the image;
wherein the camera controller (<NUM>):
computes a measured luminance value for the region of interest (<NUM>); and
adjusts an exposure in response to a comparison of the measured luminance value with a target luminance value,
characterized in, that
the image further comprises an image center (<NUM>) and an orientation indicator (<NUM>);
the orientation of the scope of the endoscopic camera system is obtained from the orientation indicator (<NUM>), which is a shape in a stop mask;
the region of interest (<NUM>) has a region of interest center (<NUM>) positioned at a fixed distance from the image center (<NUM>); and
the camera controller (<NUM>) changes the region of interest center (<NUM>) based on changes in position of the orientation indicator (<NUM>).