Patent Description:
Dual image acquisition can be a useful feature in the field of endoscopy. Two video streams of the same scene are captured, but each of the captured image streams has different associated characteristics such as a variation in light spectrum, depth of field, or light intensity. In prior dual image systems, images have generally been collected, split in image space, and then focused onto two independent detectors. Such a configuration allows for more versatility than a single image acquisition system, but is generally more expensive and complex, requiring at least two sensors and associated electronics and mounting assemblies.

Some prior art systems do capture multiple images from a single chip, however they generally contain a beamsplitter in the image space of the camera. Such designs have significant limitations due to lack of flexibility in positioning desired optical elements such as filters, lenses, and other elements in the optical paths after the beam is split. Further, the cost of a dual image system may be higher due to the duplication of certain optical components used in focusing and detecting the image light of the dual channels.

<CIT> discloses an image separating device connected to an endoscope and a first and a second CCD camera, wherein the single optical image light is directly separated after receival by a half mirror in the image separating device and the received circular image is displayed on the second CCD camera from the reflected path, while in the transmitted path, the received image is enlarged and decentered by means of a Pechan prism and a lens group to be focused on a separate first CCD camera.

<CIT> describes an image sensors for picking up images according to the prior art, whereby a double beam splitting is applied and for the second beam splitting the transmitted white light is further divided into two bundles of light rays by a half mirror and consequently the second transmitted light is focused on the focal plane of an image sensor by a condenser lens, while the light reflected by the half mirror is further reflected by a mirror and focused on a position that differs from the focal plane of the image sensor by another condenser lens.

In <CIT> an imaging system comprising an image pick-up device and an endoscope with a circular exit window forming part of an object plane is disclosed, wherein by an optical system, comprising a collimating lens, an anamorphic prism system and a further lens, the circle situated in the object plane is imaged as an ellipse on a detection face of the image sensor.

What is needed are devices and methods to enable an endoscopic camera to acquire dual images in a cost effective manner. What is further needed are devices allowing the use of varied existing endoscopes for dual imaging applications, and allowing the detection of the varied characteristics in the dual images.

It is an object of the invention to provide improved endoscope acquisition of dual images, and to allow the use of varied existing endoscopes for dual imaging applications. It is another object to make the most effective use of high definition image sensors in dual imaging applications. It is a further object of the invention to allow detection of varied characteristics in the dual images, based on ability to vary the characteristics of the optical channels of the dual images.

Endoscopic camera head devices and methods are provided using light captured by an endoscope system. Substantially afocal light from the endoscope is manipulated and split by a beamsplitter. The resulting first and second beams are passed through focusing optics to fall on a single sensor. In order to take better advantage of the available number image sensor pixels, the beam may pass through cylindrical lens elements (or prisms) to generate an anamorphic aspect ratio prior to being split, increasing the resolution of the image in one dimension. The afocal, anamorphic beam is then split, and both images are focused on the image sensor. The anamorphism is compensated for in image processing, permitting higher resolution in one dimension along the image sensor. The manipulation of the beams prior to being split (and in some cases after or while being split) can take several forms, each offering distinct advantages over existing systems.

The problem is solved by optical imaging system for use with a medical scope as defined in claim <NUM>. The optical imaging system comprises a common image sensor for acquiring a first image and a second image, a first optical group comprising a beamsplitter optically arranged to receive single optical image light in an afocal state from the assigned medical scope or from a collimating lens or lens group of the optical imaging system and split the single optical image into a first portion of light directed along a first optical path and a second portion of light directed along a second optical path; a second optical group comprising refractive elements optically arranged to receive the first and second portions of light from the beamsplitter and focus the first portion as the first image onto a first area of the common image sensor and focus the second portion as the second image onto a second area of the common image sensor wherein the first and second image areas of the common image sensor do not overlap; and one or more manipulating optical means located upstream of the second optical group to manipulate one or more of the single optical image light, the first portion of light and the second portion of light.

According to a first aspect of the invention, an optical imaging system for use with a medical scope is provided, including a first optical group with a beamsplitter optically arranged to receive single optical image light in an afocal state and split the single optical image into a first portion of light directed along a first optical path and a second portion of light directed along a second optical path. A second optical group includes refractive elements optically arranged to receive the first and second portions of light from the beamsplitter and focus the first portion as a first image onto a first area of a common image sensor and focus the second portion as a second image onto a second area of the common image sensor wherein the first and second image areas of the common sensor do not overlap. One or more manipulating optical means are located upstream of the second optical group to manipulate one or more of the single optical image light, the first portion of light and the second portion of light.

According to some implementations of the first aspect, one or more of the manipulating optical means is an element of the first optical group. The one or more of the manipulating optical means may include an anamorphic optical element in the first optical group, optically arranged to receive the single optical image light in an afocal state such that the resulting images have an anamorphic aspect ratio. The manipulating optical means may be the beamsplitter. The anamorphic optical elements may be prisms constructed to induce the anamorphic aspect ratio, or may be lenses constructed to induce the anamorphic aspect ratio.

According to some implementations of the first aspect, the one or more manipulating optical means include a spectral filter whereby the spectral content of the first portion of light differs substantially from the spectral content of the second portion of light. The first portion of light may include infrared content, with the second portion of light including visible light. The system may include an image processor programmed to process the first and second images to generate a single combined image with the two different spectral contents overlaid.

According to some implementations of the first aspect, the one or more manipulating optical means includes a means to manipulate the light intensity of the first portion of light such that it has a different intensity than the second portion of light. The system may include an image processor programmed to process the first and second images to generate a single combined image with higher dynamic range than either the first or second image taken individually.

According to some implementations of the first aspect, the one or more manipulating optical means includes an optical element in the first optical path that is not present in the second optical path such that the first image is brought to a different focus on the common image sensor. The system may include an image processor programmed to process the first and second image to generate a single image with an enhanced depth of field over either the first or second image taken individually.

According to some implementations of the first aspect, the one or more manipulating optical means include a magnification means to manipulate the first portion of light such that the first and second image have a different magnification at the image sensor. According to some implementations of this aspect, the single optical image light is split unequally by intensity such that the majority of divided intensity is dedicated to the magnified beam in order to compensate for a lower intensity collected in the field of view of the magnified image. The image splitting inequality may be specifically selected such that the resulting first and second image have substantially equal intensities.

According to a second aspect of the invention, a method is provided for producing enhanced images from a medical scope as defined in claim <NUM>. The method includes collecting image light from an illuminated scene. The method then directs the image light from the medical scope along a single optical channel with the image light in a substantially afocal state, and splits the image light from the single optical channel into a first portion of light and a second portion of light. The method optically manipulates one or more of the image light directed along a single optical channel, the first portion of light, and the second portion of light. The method directs the first portion of light along a first optical path and the second portion of light along a second optical path, and focuses the first portion of image light on a first area of an image sensor and forms a first detected image. The method focuses the second portion of image light on a second portion of the image sensor, separate from the first area of the image sensor, and forms a second detected image. The method then processes image data from the first and second portions of the image sensor.

According to some implementations of the second aspect, the method generates a single combined image by combining elements of the first detected image and elements of the second detected image.

According to some implementations of the second aspect, the step of optically manipulating one or more portions of light includes manipulating the image light in the single optical channel to have an anamorphic aspect ratio.

According to some implementations of the second aspect, the step of optically manipulating one or more portions of light includes filtering the spectral content of the first portion of light such that it differs substantially from the spectral content of the second portion of light. Filtering the first portion of light may include filtering wavelengths outside of the infrared spectrum such that the content of the first portion of light includes infrared light and the second portion of light includes visible light.

According to some implementations of the second aspect, the step of optically manipulating one or more portions of light includes attenuating the first portion of light such that it has a different intensity than the second portion. The step of processing the image data may also include generating a single combined image with higher dynamic range than either the first or second detected image taken individually.

According to some implementations of the second aspect, the step of optically manipulating one or more portions of light includes focusing the first image on the common image sensor differently than focusing the second image on the common image sensor, and the image processing step includes generating a single image with an enhanced depth of field over that of either the first or second detected images taken individually.

According to a third aspect of the invention, an optical device for use in endoscope procedures is provided, including a first optical group with one or more anamorphic optical elements optically arranged to receive light of a single optical image, and a beamsplitter optically arranged to receive the single optical image light from the one or more anamorphic optical elements in an afocal state and split the single optical image into a first portion of light directed to a first optical path and a second portion of light directed to a second optical path. A second optical group including refractive elements is optically arranged to receive the first and second portions of light from the beamsplitter. The second optical group is constructed to focus the first portion as a first image onto a first area of a common image sensor and to focus the second portion as a second image onto a second area of the common image sensor.

According to some implementations of the third aspect, the optical device is a camera head including a connector adapted to attach to and detach from an endoscope. The optical device may be a videoendoscope including a scope shaft connected to a camera head containing the first and second optical groups and the common image sensor.

According to some implementations of the third aspect, the first optical group may include a spectral filter whereby the spectral content of the first portion of light differs substantially from the spectral content of the second portion of light. The first portion of light may include infrared content, with the second portion of light including visible light.

According to some implementations of the third aspect, the first optical group may include a means to manipulate the light intensity of the first portion of light such that it has a different intensity than the second portion of light. The device may be connected to an image processor programmed to process the first and second images to generate a single combined image with higher dynamic range than either the first or second image taken individually.

According to some implementations of the third aspect, the device may include a focusing optical element in the first optical path that is not present in the second optical path such that the first image is brought to a different focus on the common image sensor. The device may be connected to an image processor programmed to process the first and second image to generate a single image with an enhanced depth of field over either the first or second image taken individually.

According to some implementations of the third aspect, the device may include a magnification means to manipulate the first portion of light such that the first and second image have a different magnification at the image sensor.

These and other features of the invention will be apparent from the following description of the preferred embodiments, considered along with the accompanying drawings.

The present invention will become more fully understood from the detailed description given herein and the accompanying drawings which are given by way of illustration only, and thus are not limitative of the present invention, and wherein:.

As used herein, first elements (e.g., sensors and lenses) that are "optically arranged" in relation to other elements, refers to the first elements° position along a common optical path that includes first and other elements. For example, a lens group optically arranged between an image sensor and an objective, means that the lens group occupies a portion of the optical path that light travels (e.g., from the objective to the image sensor) for capturing images or video. Directions such as upstream and downstream refer to the direction of light travel.

Because digital cameras, image sensors and related circuitry for signal capture and processing are well-known, the present description will be directed in particular to elements forming part of, or cooperating more directly with, a method and apparatus in accordance with the invention. Elements not specifically shown or described herein are selected from those known in the art. Certain aspects of the embodiments to be described are provided in software. Given the system as shown and described according to the invention in the following materials, software not specifically shown, described or suggested herein that is useful for implementation of the invention is conventional and within the ordinary skill in such arts.

<FIG> is a block diagram of a medical imaging device <NUM> according to an example embodiment of the invention. Medical imaging device <NUM> ("device <NUM>") includes a camera head <NUM> which may have an endoscope <NUM> attached via connectors <NUM> and <NUM>. In some embodiments, an endoscope <NUM> and camera head <NUM> may be integrated into a single housing with no connectors needed. In some embodiments, the device <NUM> is provided as only the camera head <NUM> adapted to be connected to a suitable endoscope. Connectors <NUM> and <NUM> in this embodiment are standard eyecup style optical connectors but may be any suitable connector allowing light to pass from endoscope <NUM> to camera head <NUM>. Various structural components supporting the depicted elements are omitted in the diagrams herein, as well as other components such as illumination lights sources and controls, which are known in the art and are not shown in order to avoid obscuring the relevant details of the example embodiments of the invention.

Camera head <NUM> includes a collimating lens or lens group <NUM> positioned at or behind a central window of connector <NUM> to receive and condition optical image light from the endoscope <NUM>. Positioned in the optical channel after collimating lens <NUM> is a first substantially afocal optical group <NUM> that may include one or more anamorphic optical elements <NUM> optically arranged to receive the optical image light and alter the image light to have an anamorphic aspect ratio. It is noted that the beamsplitting techniques herein may be used with or without the anamorphic elements in various embodiments. Further, other manipulating optical means, such as a spectral filter or polarized filter, may be employed in place of or in addition to the anamorphic optical elements <NUM>. In addition, these manipulating optical means may also be elements of the beamsplitter <NUM>. Other versions may include another type of manipulating optical means such as, for example, a magnification means such as a magnifying lens to manipulate the first portion of light such that the first and second image have a different magnification at the image sensor. By the term "substantially afocal optical group," it is meant that collimating lens group <NUM> ensures that light passing through optical group <NUM> is substantially afocal, and, optical group <NUM> is not positioned in the image space of the optical system. A beamsplitter <NUM> is optically arranged to receive the optical image light in a substantially afocal state from the endoscope <NUM>, passing through collimating lens <NUM>, and split the optical image light into a first portion of light directed to a first optical path and a second portion of light directed to a second optical path as depicted by the two arrows showing the light path to common second optical group <NUM>. The first and second optical paths are further described with respect to the example embodiments below. The splitting of afocal light ahead of the second, focusing, lens group <NUM>, rather than in the image space after the light is focused, has the advantage of allowing a common back end to be used with various optical groups <NUM> having disparate functions, simplifying the optical design, development, and construction of the camera head. The use of a common image sensor allows efficient use of high resolution sensors, which provide enough pixel resolution to capture a plurality of images with sufficient resolution for many endoscope applications. Enhanced depth of field, high dynamic range (HDR), fluorescence imaging (FI) analysis (including indocyanine green (ICG) analysis), and polarization studies can benefit from the collection of varying versions of the same image. A further advantage is that the device may be used with a basic endoscope head allowing the pairing of stock endoscopes with a plurality of possible inventive imaging devices utilizing different front end components <NUM> for a variety of different applications.

The second optical group <NUM> is generally for focusing the substantially afocal light received from the first optical group <NUM> onto the image sensor. Second optical group <NUM> includes refractive elements optically arranged to receive the first and second portions of light from the beamsplitter <NUM> and focus the first portion as a first image onto a first area of a common image sensor <NUM> and the focus second portion as a second image onto a second area of the common image sensor <NUM>, different from the first area. The second optical group <NUM> typically includes at least one focusing lens, with the group having a total positive power. Many suitable lenses and combinations of lenses may be used for second optical group <NUM>. The sensor signal, containing two images, is generally processed as further described with respect to <FIG> and <FIG> to provide a combined image.

In some embodiments, system <NUM> includes an endoscope <NUM> as depicted at the left of the block diagram. The depicted endoscope is an example only, and many endoscope and borescope designs are suitable, including rigid and flexible endoscopes and borescopes. Typically the endoscope will have an ocular (not shown) proximal to the rod lenses <NUM>. The exemplar endoscope <NUM> includes a cover glass <NUM> at its distal tip, which in this version faces directly along the longitudinal axis of the endoscope <NUM>, but may also be positioned at an angle relative to the longitudinal axis as is known in the art. Behind, or on the proximal side of, the cover glass <NUM> is shown a preferred position for the objective lens <NUM>, set against or very near cover glass <NUM> and preferably assembled together with the cover glass in construction. While a wide-angle lens is preferred for objective lens <NUM>, this is not limiting and any suitable lens may be used in various embodiments. Objective lens <NUM> may be part of an objective lens group <NUM> which may include one or more additional lenses <NUM>. The particular number and arrangement of lenses in the endoscope <NUM> will vary widely depending on the application. Optically arranged or attached at the proximal side of objective lens <NUM> or objective lens group <NUM> is a series of one or more rod lenses <NUM> that serve to pass the light down endoscope <NUM> towards its proximal end. Typically several rod lenses <NUM> are employed, which may be separated by spacers or other lenses in any suitable manner known in the art. Also, while the endoscope <NUM> is typically rigid, known shaft design variations allow rod lenses to be used in a semi-flexible shaft in which flexible joints are present at one or more places along the shaft between the rod lenses while the shaft is rigid along the portions containing a rod lens. Such a shaft design may be used in various embodiments of the invention.

<FIG> is a partial cross section diagram of a camera head <NUM> showing the optical assembly construction according to an example embodiment. The cross section includes a light ray diagram showing the passage of image light through the assembly to image sensor <NUM>. The depicted optical elements are in diagram form only and are not drawn to scale. The depicted optical assembly may be employed with endoscope devices and systems having an integrated camera or an external detachable camera head. As shown, the optical assembly starts at collimating lens <NUM> where the image light enters the camera head. Collimating lens <NUM> may have a slightly positive or negative power in order to adjust the image light to the desired condition to be received by first optical group <NUM>, preferably with light rays close to parallel. The collimating lens group <NUM> generally comprises a group of at least two elements, including a bi-concave element followed by a bi-convex element, the two elements being adjustable with respect to each other to ensure that light exiting the collimating lens <NUM> is substantially afocal. First optical group <NUM> in this version includes one or more anamorphic optical elements <NUM> optically arranged to receive the optical image light and alter the image light to have an anamorphic aspect ratio. While anamorphic optical elements <NUM> are cylindrical lens elements in this version, any suitable anamorphic lens or combination of lenses may be used. Further, the anamorphic optical elements <NUM> may be constructed with any suitable optical elements for producing an anamorphic effect, such as shaped mirrors, for example.

A beamsplitter <NUM> is optically arranged to receive the optical image light in an afocal state and split the optical image light into a first portion of light directed to a first optical path <NUM> and a second portion of light directed to a second optical path <NUM>. In this embodiment, beamsplitter <NUM> is constructed of prisms, including the two lower right angle prisms <NUM>-<NUM> and <NUM>-<NUM> with a suitable partially reflective coating along their adjacent surface, by which the image light is split with a first portion passing straight through along first optical path <NUM> and a second portion reflected upward along second optical path <NUM> as depicted. As discussed above, the first and second portions of light may comprise different spectral content, for example, as a result of the interface of prisms <NUM>-<NUM> and <NUM>-<NUM> comprising a dichroic filter, or, alternately by placing a color absorbing filter along optical paths <NUM> and/or <NUM>. The second portion of light reflects off the reflective inner surface of the upper prism <NUM>-<NUM>, which as shown has an angle greater than a right angle prism, approximately <NUM>°, to direct the second optical path toward the second optical group <NUM>.

The second optical group <NUM> includes refractive elements optically arranged in both the first and second optical paths to receive the first and second portions of light from the beamsplitter <NUM> and focus the first portion as a first image onto a first area of a common image sensor <NUM> and the focus second portion as a second image onto a second area of the common image sensor <NUM>, different from the first area. Second optical group <NUM> typically includes at least one focusing lens, with the group having a total positive power. Many suitable lenses and combinations of lenses may be used for second optical group <NUM>, with some additional examples shown in <FIG>. In this embodiment, second optical group <NUM> includes a doublet achromat lens <NUM> having a positive optical power, including a biconvex lens and concave-plano lens. As can be understood from the ray diagram, lens <NUM> and the rest of second optical group <NUM> are preferably symmetrically arranged with respect to the first and second optical paths <NUM> and <NUM>, and large enough to span both paths. That is, second optical group <NUM> is positioned with the axis pointing between the first and second paths such that each path has similar incidence on lens <NUM>, symmetrical about the central axis of group <NUM>. As depicted, that is achieved by positioning second optical group <NUM> with its central axis at an angle that bifurcates the angle between first and second optical paths <NUM> and <NUM>. , in the diagram, first optical path <NUM> is horizontal while second optical path <NUM> is at a downward angle of approximately <NUM>° as it exits the upper prism <NUM>-<NUM> of beamsplitter <NUM> because prism <NUM>-<NUM> is not at a right angle, making paths <NUM> and <NUM> converge. To provide a symmetrical effect on the light of both paths <NUM> and <NUM>, the optical axis of second optical group <NUM> bifurcates the converging paths, in this example the second optical group <NUM> having its distal end tilted upward in the diagram at an angle of <NUM>°, thereby providing similar processing to both the first and second portions of light. All the optical elements in the second group <NUM> are so tilted, and the sensor <NUM> and cover glass <NUM> are similarly tilted. Optically arranged proximal to doublet achromat lens <NUM> is a convex-planar lens <NUM> which further focuses both portions of light toward the sensor. Optically arranged to receive both portions of light from lens <NUM> is a lens <NUM>, which is a meniscus having positive optical power. Next, a smaller concave-convex lens <NUM> with a negative power directs both portions of light toward cover glass <NUM> and image sensor <NUM>, spreading both portions of light create a pair of images of the desired size at image sensor <NUM>. <FIG> contrasts the various image detection possibilities at the rectangular sensor <NUM>. It is immediately evident that much of the active sensor area is not utilized with traditional single, circular image collection methods known in the art and shown in <FIG>. A much more effective use of the sensor area is achieved by manipulating the image to have an anamorphic aspect ratio, as shown in <FIG>. Dual image systems according to the present invention, as shown in <FIG>, utilize a large number of available pixels and enable all of the benefits discussed above associated with capturing two versions of the same scene. <FIG> shows a still more improved result by combining both the benefits of the anamorphic aspect ratio as well as the capturing of two images simultaneously, making, thereby maximal use of a single image sensor. It should also be noted that while the dual image embodiments shown in <FIG> show a vertical distribution of images on the sensor, there may be embodiments wherein the images are made to fall upon the sensor in a horizontal or even diagonal orientation. For example, the anamorphic image creating means could result in a pair of images with longer vertical axes, rather than longer horizontal axes as shown, and the resulting images would fall side-by-side on the sensor.

Referring again to <FIG>, while in this embodiment beamsplitter <NUM> directs the first and second optical paths <NUM> and <NUM> at a converging angle to each other for ease of focusing both images, this is not limiting, and other versions may not require a converging angle between the paths. For example, in other embodiments the beamsplitter <NUM> may direct the paths at a converging angle and use a second optical assembly large enough to receive and focus the paths onto the image sensor. Further, while lenses <NUM>, <NUM>, <NUM>, and <NUM> in this embodiment focus and direct both portions of light, other versions may include one or more lenses that perform focusing or diverging operations on only a single one of the optical paths. For example, lens <NUM> and/or <NUM> might be replaced with a separate focusing lens for each path.

<FIG> is a cross section diagram of an optical assembly according to another embodiment, also including a light ray diagram showing the passage of image light through the assembly to image sensor <NUM>. The depicted optical elements are in diagram form only and are not drawn to scale. Like the other embodiments disclosed herein, this optical assembly may be employed with endoscope devices and systems having an integrated camera or an external detachable camera head. As shown, the optical assembly starts at collimating lens <NUM> where the image light enters the camera head. Collimating lens group <NUM> may have a slightly positive or negative power in order to adjust the image light to the desired condition to be received by first optical group <NUM>. In this version, anamorphic elements are not used, and the collimating lens group <NUM> includes a convex-concave lens cemented to the proximal side of a meniscus lens with a negative optical power optically arranged to spread the image light to a desired size. Optically arranged in the proximal direction form these two elements is the third lens of the collimating lens group, a bi-convex lens, which has a positive optical power to collimate the image light. A beamsplitter <NUM> is optically arranged to receive the optical image light in an afocal state from the endoscope <NUM>, via the collimating lens group, and split it into a first portion of light directed to a first optical path <NUM> and a second portion of light directed to a second optical path <NUM>. In this embodiment, as in that shown in <FIG>, beamsplitter <NUM> is constructed of prisms, and the first and second optical paths <NUM> and <NUM> are at a converging angle.

The second optical group <NUM> in this version also includes refractive elements optically arranged in both the first and second optical paths to receive the first and second portions of light from the beamsplitter <NUM> and focus the first portion as a first image onto a first area of a common image sensor <NUM> and the focus second portion as a second image onto a second area of the common image sensor <NUM>, different from the first area. Similarly to the previous embodiment, both optical paths are incident on a doublet achromat lens <NUM> having a positive optical power, including a biconvex lens and a concave-concave lens. Also like the previous embodiment, second optical group <NUM> is positioned with the axis pointing between the first and second paths such that each path has similar incidence on lens <NUM>, symmetrical about the central axis of group <NUM>. The optical elements in the second group <NUM> are tilted to provide this effect, and the sensor <NUM> and cover glass <NUM> are similarly tilted. In this embodiment, optical group <NUM> is relatively longer in comparison to first group <NUM>, allowing the use of smaller elements with larger spacing. Optically arranged in the proximal direction to doublet achromat lens <NUM> is a concave-plano lens <NUM> which further focuses both portions of light toward the sensor.

Optically arranged to receive both portions of light from lens <NUM> is a lens <NUM>, having positive optical power. Next, a smaller plano-concave lens <NUM> with a negative power directs both portions of light toward cover glass <NUM> and image sensor <NUM>, spreading both portions of light create a pair of images of the desired size at image sensor <NUM>.

<FIG> is a partial cross section diagram of an alternative embodiment of the inventive camera head optical assembly, using prisms as anamorphic elements. The cross section includes a light ray diagram showing the passage of image light through the assembly to image sensor <NUM>. As shown, the optical assembly starts with a collimating and directing lens group <NUM>, <NUM> where the image light enters the camera head. In this version the collimating and directing lens group includes a doublet lens <NUM> optically arranged to receive the image light from lens <NUM>. Doublet lens <NUM> includes a convex-concave lens and a convex-plano lens. The overall collimating and directing lens group <NUM>, <NUM> has a positive power for directing the image light in a substantially afocal state toward anamorphic optical elements <NUM> of first optical group <NUM>. Optically arranged to receive the image light from lens <NUM> are anamorphic elements <NUM>, which alter the image light to have an anamorphic aspect ratio. In contrast to the embodiment shown in <FIG>, anamorphic optical elements <NUM> of this embodiment comprise four triangular prisms labeled <NUM>-<NUM> through <NUM>-<NUM>. Prisms <NUM>-<NUM> and <NUM>-<NUM> together act as one prism of an anamorphic prism pair, paired with prism <NUM>-<NUM> and <NUM>-<NUM> acting as the other half of the anamorphic prism pair, to enlarge the vertical dimension of the image light while leaving the horizontal dimension (the direction into the page) unchanged. Each element of the pair is a doublet to make them achromatic. Other suitable combinations of prisms, including prism pairs, may be used to provide the desired anamorphic effect.

Beamsplitter <NUM> is optically arranged to receive the optical image light in an afocal state and split the optical image light into a first portion of light directed to a first optical path <NUM> and a second portion of light directed to a second optical path <NUM>. In this embodiment, beamsplitter <NUM> is constructed of prisms, including the two lower right-angle prisms with a suitable partially reflective coating along their adjacent surface, by which the image light is split with a first portion passing straight through along first optical path <NUM> and a second portion reflected upward along second optical path <NUM> as depicted. As discussed above, the first and second portions of light may comprise different spectral content. The second portion of light reflects off the reflective inner surface of the upper prism, which in this version is a less-than right-angle prism, approximately <NUM>° as can be seen on the drawing, to direct the second optical path toward the second optical group <NUM> with optical path <NUM> slightly diverging from optical path <NUM> as can be seen in the ray diagram.

The second optical group <NUM> includes refractive elements optically arranged in both the first and second optical paths <NUM> and <NUM> to receive the first and second portions of light from the beamsplitter <NUM> and focus the first portion as a first image onto a first area of a common image sensor <NUM> and the focus second portion as a second image onto a second area of the common image sensor <NUM>, different from the first area. In this embodiment, second optical group <NUM> includes a doublet achromat lens <NUM> having a positive optical power, including a biconvex lens and a concave-plano lens. As can be understood from the ray diagram, lens <NUM> and the rest of second optical group <NUM> are symmetrically arranged with respect to the first and second optical paths <NUM> and <NUM>, and large enough to span both paths. That is, second optical group <NUM> is positioned with the axis pointing between the first and second paths such that each path has similar incidence on lens <NUM>, symmetrical about the central axis of group <NUM>. Similarly to the previous embodiment, this is done by positioning the optical axis of group <NUM> to bifurcate the angle between first and second optical paths <NUM> and <NUM>, although in this embodiment group <NUM> is not tilted with respect to group <NUM>. However in this case the optical paths are diverging. As can be seen on the ray diagram, the positive optical power of doublet lens <NUM> refracts both optical paths to converge toward doublet lens <NUM>. The doublet lens <NUM> includes two adjacent convex-concave lenses which further focus both portions of light in the direction of sensor <NUM>. Optically arranged to receive both portions of light from lens <NUM> is a biconcave lens <NUM>, which has a negative power and serves to reduce the angle of incidence at which the focused image light hits sensor <NUM> behind cover glass <NUM>. As with the version of <FIG>, this version provides two anamorphic images formed on the same image sensor <NUM>, which may include different characteristics either produced by the beamsplitter (such as spectrum, polarization, or intensity) or by additional lenses or filters in one of the first or second paths providing characteristics for one of the images that are not in the other. For example, an indocyanine green (ICG) imaging filter may be used, which only allows the wavelengths fluoresced by ICG dye (typically the near infrared region) to be passed in a selected one of the first or second optical paths. Such a filter may be included as part of the beamsplitter, with light split along the second optical path by reflective and dichroic surfaces or band-pass/band-reflect properties at the surfaces described. As with the other embodiments herein, the two images detected at sensor <NUM> are typically processed by the system camera control unit (CCU) to produce a final image based on both images. However for some applications the individual captured images may be displayed as may a combination of a single resultant image along with the generally unprocessed individual images.

<FIG> is a partial cross section diagram of a camera head <NUM> optical assembly similar to that of <FIG>, but including an additional lens in the first optical path. The cross section includes a light ray diagram showing the passage of image light through the assembly to image sensor <NUM>. The anamorphic elements <NUM> of first optical group <NUM>, the beamsplitter, and the lenses of second optical group <NUM> are similar to those of <FIG> and will not be described again. An additional plano-convex lens <NUM> with a positive optical power is included in this version, optically arranged in the first optical path <NUM> to receive the first portion of light from beamsplitter <NUM> and focus it, providing a modified focus to the image produced from the second portion of light in second path <NUM>. This provides that the two images formed at sensor <NUM> each have a different focal depth and therefore include focal characteristics not present in the other image. The two images can be combined in processing to create an image with a larger depth of field. Various other optical elements or manipulating optical means may be placed in one or both of the optical paths such as at the location of lens <NUM> or optically arranged after prism <NUM>-<NUM> in the second optical path <NUM>. For example, polarization filters, intensity filters, spectral filters, field stops, magnifying lenses, and other optical elements may be used. Further, one optical path length may be extended relative to the other by any appropriate means such as reflecting along a longer physical path or by including an optical element with a higher index of refraction in one path not provided in the other.

<FIG> shows a block diagram of a system including an endoscope device and an image capture device having an improved dual image optical path as described above. The invention is applicable to more than one type of device enabled for image capture, such as FI-capable endoscopes and other medical imaging devices. The preferred version is an imaging scope system, such as an endoscope.

As shown in the diagram of an endoscope device system, a light source <NUM> illuminates subject scene <NUM> with visible light. Fluorescent excitation light may also be used, which may be outside the visible spectrum, in the ultra-violet range or the infra-red/near infrared range, or both. Light source <NUM> may include a single light emitting element configured to provide light throughout the desired spectrum, or a visible light emitting element and a one or more fluorescent excitation light emitting elements. Further, light source <NUM> may include fiber optics passing through the body of the scope, or other light emitting arrangements such as LEDs or laser diodes positioned at or near the front of the scope.

As shown in the drawing, light <NUM> reflected from (or, alternatively, as in the case of fluorescence, excitation light <NUM> absorbed and subsequently emitted by) the subject scene is input to an optical assembly <NUM>, where the light is split as described herein and focused to form two images at a solid-state image sensor <NUM>. Optical assembly <NUM> includes the optics of the endoscope and of the camera head. As discussed above, portions of the optical assembly may be embodied in a camera head or in a single imaging device. Image sensor <NUM> converts the incident light to an electrical signal by integrating charge for each picture element (pixel). The image sensor <NUM> may be constructed with any suitable sensor technology such as active pixel complementary metal oxide semiconductor sensor (CMOS APS) or a charge-coupled device (CCD), for example.

The total amount of light <NUM> reaching the image sensor <NUM> is regulated by the light source <NUM> intensity, the optical assembly <NUM> aperture, and the time for which the image sensor <NUM> integrates charge. An exposure controller <NUM> responds to the amount of light available in the scene given the intensity and spatial distribution of digitized signals corresponding to the intensity and spatial distribution of the light focused on image sensor <NUM>. If fluorescence imaging is used, exposure controller <NUM> also controls the emission of fluorescent excitation light from light source <NUM>, and may control the visible and fluorescent light emitting elements to be on at the same time, or to alternate to allow fluoresced light frames to be captured in the absence of visible light if such is required by the fluorescent imaging scheme employed. Exposure controller <NUM> may also control the optical assembly <NUM> aperture, and indirectly, the time for which the image sensor <NUM> integrate charge. The control connection from exposure controller <NUM> to timing generator <NUM> is shown as a dotted line because the control is typically indirect.

Timing generator <NUM> produces various clocking signals to select rows and pixels and synchronizes the operation of image sensor <NUM>, analog signal processor <NUM>, and A/D converter <NUM>. Image sensor assembly <NUM> includes the image sensor <NUM>, adjustment control, the analog signal processor <NUM>, the A/D converter <NUM>, and the timing generator <NUM>. The functional elements of the image sensor assembly <NUM> can be fabricated as a single integrated circuit as is commonly done with CMOS image sensors or they can be separately-fabricated integrated circuits.

Analog signals from the image sensor <NUM> are processed by analog signal processor <NUM> and applied to analog-to-digital (A/D) converter <NUM> for digitizing the analog sensor signals. The digitized signals each representing streams of images or image representations based on the data are fed to image processor <NUM> as image signal <NUM>. Typically both images will be transmitted together in signal <NUM> as a single image, which is separated in the image processing circuitry into dual image areas of the sensor (for example, image <NUM> and image <NUM>, of <FIG>).

The system camera control unit (CCU) <NUM> includes image processing circuitry <NUM> performing digital image processing functions to process and filter the received images as is known in the art. Image processing circuitry may include separate, parallel pipelines for processing the first and second images separately. CCU <NUM> may be implemented in a single assembly or may include two or more camera control modules performing different functions such as communication with a specific camera model, and image processing. Such circuitry is known in the art and will not be further described here. Image processing circuitry <NUM> may provide algorithms, known in the art, for combining two images of the same view but containing different characteristics in a combined image display.

The system controller <NUM> controls the overall operation of the image capture device based on a software program stored in program memory <NUM>. This memory can also be used to store user setting selections and other data to be preserved when the camera is turned off. System controller <NUM> controls the sequence of data capture by directing exposure controller <NUM> to set the light source <NUM> intensity, the optical assembly <NUM> aperture, and controlling various filters in optical assembly <NUM> and timing that may be necessary to obtain image streams. A data bus <NUM> includes a pathway for address, data, and control signals.

Processed image data are continuously sent to video encoder <NUM> to produce a video signal. This signal is processed by display controller <NUM> and presented on image display <NUM>. This display is typically a liquid crystal display backlit with light-emitting diodes (LED LCD), although other types of displays may be used as well. The processed image data can also be stored in system memory <NUM> or other internal or external memory device.

The user interface <NUM>, including all or any combination of image display <NUM>, user inputs <NUM>, and status display <NUM>, is controlled by a combination of software programs executed on system controller <NUM>. User inputs typically include some combination of typing keyboards, computer pointing devices, buttons, rocker switches, joysticks, rotary dials, or touch screens. The system controller <NUM> manages the graphical user interface (GUI) presented on one or more of the displays (e.g. on image display <NUM>). In particular, the system controller <NUM> will typically have a mode toggle user input (typically through a button on the endoscope or camera head itself, but possibly through a GUI interface), and in response transmit commands to adjust image processing circuitry <NUM> based on predetermined setting stored in system memory. Preferably a system employed with any of the device designs herein provides ability to toggle between an individual view of either single image (for example, image <NUM> or image <NUM>), both individual images, and/or a view of the combined image created with processing of data from both images. Settings may be provided to adjust the manner in which characteristics from the individual images are combined and displayed or stored. Settings may also include different settings for different models of scopes that may be attached to a camera head or other imaging device containing image sensor assembly <NUM>.

Image processing circuitry <NUM> is one of three programmable logic devices, processors, or controllers in this embodiment, in addition to a system controller <NUM> and the exposure controller <NUM>. Image processing circuitry <NUM>, controller <NUM>, exposure controller <NUM>, system and program memories <NUM> and <NUM>, video encoder <NUM>, and display controller <NUM> may be housed within CCU <NUM>.

CCU <NUM> may be responsible for powering and controlling light source <NUM>, image sensor assembly <NUM>, and/or optical assembly <NUM>. In some versions, a separate front end camera module may perform some of the image processing functions of image processing circuitry <NUM>.

Although this distribution of imaging device functional control among multiple programmable logic devices, processors, and controllers is typical, these programmable logic devices, processors, or controllers can be combinable in various ways without affecting the functional operation of the imaging device and the application of the invention. These programmable logic devices, processors, or controllers can comprise one or more programmable logic devices, digital signal processor devices, microcontrollers, or other digital logic circuits. Although a combination of such programmable logic devices, processors, or controllers has been described, it should be apparent that one programmable logic device, digital signal processor, microcontroller, or other digital logic circuit can be designated to perform all of the needed functions. All of these variations can perform the same function and fall within the scope of this invention.

<FIG> is a flowchart of a method for producing endoscopy images according to an example embodiment. The method may be performed employing any of the various example embodiments of a camera head or joint endoscope and camera head devices as described herein, with a suitable camera control unit such as that described above to process the image data. Other types of medical scopes or borescopes may be used in place of the endoscope as discussed above. The method begins at process block <NUM> where it includes receiving the image light from an endoscope. The endoscope device used may be a separate device attached to a camera head or an endoscope integrated with a camera head. At process block <NUM>, the process directs the received image light along a single optical channel. At block <NUM>, the process alters the image light, still in a single optical channel, to have an anamorphic aspect ratio. The final anamorphic ratio is selected to allow improved use of image sensor area when the image light is directed at the sensor. Some embodiments may not include block <NUM>, or may instead optically manipulate the image light in another manner at this step. Next at process block <NUM>, with the image light in an afocal state, the process includes splitting the image light from the single optical channel into a first portion of light and a second portion of light. Process block <NUM> may also operate to manipulate the first and second resulting beams relative to each other by selecting of the characteristics of the resulting beams that are allowed to pass directly through or be reflected by the beam splitting means. For example, beam splitting means at block <NUM> may include a dichroic beam splitter, resulting in the first portion of light to have different spectral content than the second portion of light. Then at block <NUM>, the process directs the first portion of light along a first optical path and the second portion of light along a second optical path. Directing the light is preferably done with a beam splitter such as the example splitters described herein. Optionally, at block <NUM>, the process may further optically manipulate the first and second portions of light relative to each other, for example by filtering, focusing, modifying the optical path length, or performing other optical manipulations to one or both of the portions of light. Next at process block <NUM> the process includes focusing the first portion of image light on a first area of an image sensor and focusing the second portion of image light on a second portion of the image sensor separate from the first portion. An example of the resulting areas is depicted in <FIG>, which show a example areas of rectangular image sensors <NUM>, with the first portion of light hitting the sensor on the area shown as Image <NUM> and second portion of light hitting the sensor on the area shown as Image <NUM>. As can be seen, the anamorphic aspect ratio makes greater use of the total sensor area than do the contrasting circular image areas on the sensor <NUM> shown in <FIG>. The anamorphic images are then corrected in image processing to yield the proper image aspect ratio.

Next at process block <NUM>, image processing is performed on the image data from the sensor. The processing is generally applied to image data from both the first and second portions to generate a single combined image including first characteristics resulting only from the first portion of light and second characteristics resulting only from the second portion of light. The processing is performed by a CCU such as the CCU <NUM> of <FIG>, or other suitable image processing circuitry.

The image characteristics from the different portions of light may be designed to be any of a number of characteristics desired to be measured through the endoscope. For example, in some versions the spectral content of the first portion of light differs substantially from the spectral content of the second portion of light. The first portion of light may include infrared content with the second portion of light including visible light, for example. A combined image based on such a scheme may use designated colors to show the infrared content superimposed on the visible light content in a manner known in the art. In another embodiment, the first portion of light has a different intensity range than the second portion. This may be accomplished by reflective characteristics of the beamsplitter, or by a filter or other suitable optical element placed in one of first or second optical paths. Processing the image data with different intensity ranges can be used to provide a high dynamic range (HDR) single combined image with higher dynamic range than either the first or second image taken individually. In another example embodiment, the process includes focusing the first image on the common image sensor differently than the focusing of the second image. The image processing of block <NUM> may extract features that are in sharper focus in one of the two images. Such an embodiment results in a single image with an enhanced depth of field over the first or second image taken individually. Still other embodiments may make use of a polarizing beamsplitter or with a polarizing filter in the first and/or second optical paths, allowing the dual images to each comprise light with a different polarization. Known image processing techniques for polarization studies may be applied in processing the dual images from such an embodiment, permitting, for example, glare reduction in the combined image.

As used herein the terms "comprising," "including," "carrying," "having" "containing," "involving," and the like are to be understood to be open-ended, that is, to mean including but not limited to. Any use of ordinal terms such as "first," "second," "third," etc., in the claims to modify a claim element does not by itself connote any priority, precedence, or order of one claim element over another, or the temporal order in which acts of a method are performed. Rather, unless specifically stated otherwise, such ordinal terms are used merely as labels to distinguish one claim element having a certain name from another element having a same name (but for use of the ordinal term).

The foregoing has outlined rather broadly the features and technical advantages of the invention in order that the detailed description of the invention that follows may be better understood. It should be appreciated by those skilled in the art that the conception and specific embodiments disclosed may be readily utilized as a basis for modifying or designing other structures for carrying out the same purposes of the invention. It should also be realized by those skilled in the art that such equivalent constructions do not depart from the scope of the invention as set forth in the appended claims.

Although the invention and its advantages have been described in detail, it should be understood that various changes, substitutions and alterations can be made herein without departing from the scope of the invention as defined by the appended claims. The combinations of features described herein should not be interpreted to be limiting, and the features herein may be used in any working combination or sub-combination according to the invention. This description should therefore be interpreted as providing written support, under U. patent law and any relevant foreign patent laws, for any working combination or some sub-combination of the features herein.

Claim 1:
Optical imaging system for use with a medical scope, comprising:
a common image sensor for acquiring a first image and a second image,
a first optical group (<NUM>) comprising a beamsplitter (<NUM>) optically arranged along a common optical path with the associated medical scope (<NUM>) to receive single optical image light in an afocal state from the associated medical scope (<NUM>) or from a collimating lens or lens group (<NUM>) of the optical imaging system and split the single optical image into a first portion of light directed along a first optical path (<NUM>) and a second portion of light directed along a second optical path (<NUM>); characterized in that
a second optical group (<NUM>) comprising refractive elements optically arranged to receive the first and second portions of light from the beamsplitter (<NUM>) and focus the first portion as the first image onto a first area of the common image sensor (<NUM>) and focus the second portion as the second image onto a second area of the common image sensor (<NUM>) wherein the first and second image areas of the common image sensor (<NUM>) do not overlap; and
one or more manipulating optical means located upstream of the second optical group (<NUM>) to manipulate one or more of the single optical image light, the first portion of light and the second portion of light.