Patent Description:
Ophthalmic microscope devices, such as slit lamp microscopes or fundus cameras, have an illumination source, such as a slit lamp, for illuminating the subject's eye as well as a microscope for viewing it. Often, the microscope includes a camera.

When recording images with the camera, the exposure time, aperture width, gain, and/or illumination time may be adjusted automatically to implement exposure correction. For example, the average brightness value of the recorded image may be calculated, and the exposure parameters may then be changed to adjust this average brightness value to a desired value.

<CIT> describes the selection of automatic exposure settings in a fluorescence microscope. <CIT> describes controlling the settings of a digital camera.

The problem to be solved by the present invention is to provide a method and a microscope of the type mentioned above that provide improved image quality.

This problem is solved by the method and microscope of the independent claims.

Accordingly, the method for controlling image exposure in an ophthalmic microscope device having a camera and an illumination source comprises at least the following steps:.

This method is based on the understanding that, in ophthalmologic microscopy, the depth of field is small and typically will only cover the region of interest or a part thereof. A substantial part of the scene is typically out-of-focus and will not exhibit sharp pixel intensity changes in the corresponding image areas.

Hence, by weighting edge-rich sub-regions of the camera image more strongly than more uniform sub-regions for calculating the brightness parameter, a better exposure can be achieved for the sub-region(s) of interest. This may result in more "homogeneous" parts of the image being underexposed or overexposed, but these parts are typically not of interest.

The steps of the method are advantageously carried out automatically, e.g. by the control unit of the microscope.

The step of determining the first and second sub-regions may include the step of computing an edge intensity image having pixel values that are indicative of the presence of edges at the respective pixels.

Such an edge intensity image may e.g. be calculated using a discrete convolutional operator, such as a Sobel or Laplace filter. The operation may be carried out e.g. on the camera image or on an image derived therefrom, such as on a grayscale version of the camera image and/or a scaled-down version of the camera image. The edge intensity image may then be spatially averaged by e.g. spatial smoothing or down-scaling to obtain an image that reflects both intensity and density of edges.

In the step of determining the first and second sub-regions, different weights may be assigned to different predetermined parts of the image. For example, parts of the image closer to its center may be weighed more strongly than parts further away from the center. Then, the edge density is weighed with the different weights before identifying the first sub-region and the second sub-region. In other words, the different parts will gain different importance. For example, by weighing the center of the image more strongly, the calculated brightness parameter becomes more sensitive to edge structures in the center of the image.

The present method allows to concentrate the exposure control on parts of the image that are in focus, namely on the parts that have a high density of edges. Other areas, which may be much larger but that are more or less uniform, are weighed less strongly or not at all when calculating the brightness parameter. This is in contrast to conventional exposure control methods, where a large, uniform area may dominate the exposure parameters.

The invention also relates to an ophthalmic microscope device, in particular a slit lamp, which has an illumination source, a camera, and a control unit. The control unit is connected to the illumination source and the camera, and it is adapted (e.g. by programming and/or hardware) to execute the method of the present invention.

The invention will be better understood and objects other than those set forth above will become apparent when consideration is given to the following detailed description thereof.

Note: <FIG> and <FIG> are dithered for print. The actual images are e.g. true scalar images.

"Multiplying" two pixel-based images is understood as multiplying each pixel value of the first image with the corresponding pixel value of the second image to calculate the respective pixel value of the resulting image. Optionally, the images may be rescaled before such an operation such that each pixel of the first image is assigned to a defined pixel of the second image or vice versa.

If one of the images has multi-component pixel values (e.g. RGB-values) and the second one has scalar pixel-values, the multiplication may e.g. consist of multiplying each multi-component pixel value of the first image with the pixel value of the second image, resulting in an image that again has multi-component pixel values. In another variant, the multiplication may first combine the multi-component pixel values of a pixel of the first image into a single value, e.g. using an averaging operation, and subsequent multiplication of the single value with the pixel value of the second image, in which case the resulting image has scalar pixel-values.

The term "edge density" is any measure that correlates with the number and magnitude of discontinuities in pixel intensity in a given sub-region of the image. It may e.g. be the value of the discrete Laplace transform as described below, the power of the 2D Fourier spectrum above a given frequency, etc..

<FIG> shows a slit lamp microscope as an example for an ophthalmic microscope device. The device comprises a microscope <NUM> and an illumination source <NUM>, such as a slit lamp. Both these elements can e.g. be pivoted about a common pivot axis <NUM> and be used to view a subject's eye.

A headrest <NUM> may be provided for the subject to rest his/her head on.

The device further may have an external display <NUM>, which is advantageously a touch screen.

The user of the microscope device may view the image of the subject's eye either on display <NUM> or, directly, by using an eyepiece <NUM> of microscope <NUM>.

<FIG> shows a functional block circuit diagram of the microscope device. The device comprises a control unit <NUM>, which is e.g. a microprocessor equipped with suitable memory and programmed to provide the functionality of the device. Control unit <NUM> may be built into the device itself or be an external part of the device.

Control unit <NUM> is connected to illumination source <NUM>, to an electronic camera <NUM> of microscope <NUM>, and to further parts <NUM> of the device, such as to an adjustable aperture in the imaging optics of microscope <NUM>.

Control unit <NUM> may also be connected to external display <NUM> and/or to an internal display <NUM> (see below).

<FIG> shows a schematic drawing of the optics of microscope <NUM>. It comprises objective optics <NUM> for taking an image of the subject's eye <NUM>. Objective optics <NUM> may have a variable magnification and/or an adjustable aperture controlled by control unit <NUM>.

The light from objective optics <NUM> is sent through a beam splitter <NUM>, where part of it is transmitted to eyepiece <NUM> to be directly viewed by the user <NUM>.

Beam splitter <NUM> reflects part of the light from objective optics <NUM> to camera <NUM>, which may e.g. be a CCD camera.

In addition, microscope device <NUM> may comprise the internal display <NUM> mentioned above. The light from internal display <NUM> is sent to beam splitter <NUM>, where part of it is reflected into eyepiece <NUM>, which allows the user to view the information displayed on internal display <NUM>.

Control unit <NUM> is adapted and programmed to automatically select suitable exposure parameters when recording an image with camera <NUM>. These parameters may be one or more of the following:.

To find the suitable exposure parameter(s), control unit <NUM> first takes a "camera image" with camera <NUM>. From this camera image, it calculates a brightness parameter B as described below. If the brightness parameter B indicates that the image is not bright enough, control unit <NUM> increases one or more of the exposure parameters mentioned above and then takes an "adjusted image" by means of camera <NUM>.

In particular, control unit <NUM> may implement a control loop where it tries to adjust the brightness parameter B of several consecutive images taken by camera <NUM> to a desired value by changing the one or more exposure parameters.

The present method is based on measuring at least one brightness parameter B from an image recorded by the camera. In the following, steps to calculate this parameter are described in more detail.

In a first step, the camera image M is recorded by means of camera <NUM>. The camera image is a pixel-based representation of the image recorded by the camera. This may e.g. be a color image, a gray-scale image, or a false-color image. As mentioned above, it can be the raw image taken by camera <NUM> or be derived from this raw image by first image processing steps.

In a next step, a grayscale image G may be calculated from camera image M, i.e. an image where each pixel Gij (with i = <NUM>. N and j = <NUM>. M) is a scalar value indicative of the "brightness" at the given pixel area. If camera image M is a color image, gray scale image G may e.g. be calculated from a weighted average of the RGB-values of camera image M.

In a next step, an edge intensity image E is calculated, e.g. from grayscale image G (if the grayscale image has been calculated) or from camera image M directly. The pixel values Eij of edge intensity image E are indicative of the presence of discontinuities between Gij and its neighboring pixels.

For example, edge intensity image E can be calculated from gray-scale image G using the 2D discrete Laplace operator D in a convolution operation and taking the absolute values of the result, i.e. <MAT>.

The discrete Laplace operator D is e.g. calculated using a convolution with a suitable kernel, with the kernel calculating weighted differences between the values of a pixel and its neighboring pixels, see e.g. https://en. org/wiki/Discrete_Laplace_operator. The Laplacian is a 2D isotropic measure of the 2nd spatial derivative of an image. It highlights regions of rapid intensity change.

Instead of using the absolute discrete Laplace operator D, other edge sensitive filters such as Prewitt, Roberts, Sobel, Scharr, etc. might be used instead. Scharr is presently preferred. Also the edge intensity image may be spatially averaged e.g. by down-scaling to obtain a filtered image that reflects both, intensity and density of edges.

Note that the edge intensity image may also be computed from the camera image M directly, which e.g. allows to detect edges in hue or saturation that may not be apparent in grayscale space.

Instead of calculating a complete edge intensity image, edge detection can be used, generally, to identify at least one region of the camera image that has a high edge density and another region that has a low edge density, e.g. by dividing the camera image into several sub-regions, calculating the Fourier transform of each sub-region, and identifying the one that has a the strongest high-frequency spectral components.

In optional Step <NUM>, the edge intensity image is offset by some fixed value <MAT>.

This causes areas with a low density / intensity of edges to still have a minimal effect on the final exposure.

In an optional Step <NUM>, the edge intensity image E (or E') is multiplied, pixel-wise, with a filter image F that has higher values in the center of the image than at the edges of the image. The result of this step is a weight image W: <MAT>.

Filter image F may e.g. be a Gaussian distribution of values centered on the image, e.g. <MAT> with a shape variable k><NUM>.

Alternatively, filter image F can be any other pixel distribution that favors certain predefined parts of the image while reducing the importance others. Advantageously, it has highest pixel values Fij in the center of the image and the pixel values decrease monotonously with increasing distance from the center of the image.

The result of the previous steps, the weight image W may be normalized (unless normalization is carried out in another step of the method), i.e. its pixel values are scaled such that their sum equals a given, predefined value (i.e. a value independent of the camera image), in particular <NUM>. For example, the normalized weight image W' can be calculated as <MAT>.

Using the normalized weight image W', the brightness parameter B can be calculated as a weighted sum of all pixels of grayscale image G, with the weights given by the weight image: <MAT>.

Brightness parameter B is a weighted sum of the brightness of the pixels of grayscale image G, with the weight being higher for the pixels at edge structures and (if vignetting was used) for the pixels closer to e.g. the center of the image.

Instead of calculating the brightness parameter B from grayscale image G, it may also be calculated e.g. from a multiplication with camera image M directly, again by weighing its pixel values with the pixel values of the weight image W.

In the above example for calculating the brightness value, it was assumed that all images have the same resolution.

It must be noted, though, that the resolution of the various images may vary. Mapping operations may be used between the individual steps and/or when mathematically combining individual images.

For example, the resolution of camera image M is not necessarily the same as the resolution of the camera. For example, a part of the image recorded by the camera may e.g. have been cut off (e.g. when only a square region of a rectangular, non-square camera is used), and/or the resolution of the image of the camera may have been downscaled to derive camera image M for faster processing.

Similarly, edge intensity image E may be downscaled before offsetting, vignetting, normalizing, and/or calculating the brightness parameter. This can be seen as a way of spatial averaging turning the edge intensity into a combined edge intensity / density information. The advantages of such downscaling are two-fold. On the one hand, processing becomes faster. On the other hand, the pixels close to an edge structure in the camera image M will be given larger weight even if they are not right at an edge structure.

<FIG> shows a (dithered) grayscale image G as taken by a slit-lamp microscope. The image shows an edge-rich region corresponding to a section of cornea highlighted by a slit illumination.

<FIG> shows weight image W as calculated from the grayscale image of <FIG> using the steps above. As can be seen, it has high (bright) values in the center within the edge-rich structures of <FIG>.

In above Steps <NUM> to <NUM>, edge detection is used to identify at least one "first sub-region" of the camera image that has a high edge density. This first sub-region may e.g. correspond to the pixels of edge intensity image E or E' having high values or, if edge detection based on a 2D Fourier-transform is used, to the sub-region(s) having strongest high-frequency spectral components. Similarly, at least one "second sub-region" is identified that has low edge density. This second sub-region may e.g. correspond to the pixels of edge intensity image E or E' having low values or, if edge detection based on a 2D Fourier-transform is used, to the sub-region(s) having the lowest high-frequency spectral components.

The identification process for the first and second sub-regions may, as in step <NUM>, use a second set of weights that are assigned to predetermined parts of the image and multiplied with the weights from step <NUM>. In this context, "predetermined parts" are parts whose location is independent of the contents of the camera image M.

In the example above, the pixels of filter image F are indicative of these predetermined parts in that they have high values in parts that are to be weighed more strongly. When calculating Eq. (<NUM>), the edge density as embodied by edge density image E or E' is weighed with the different weights of the predetermined parts of the image, with the weights being embodied by filter image F. In the specific example, parts closer to a center of the image are weighed more strongly than parts further away from the center.

In particular, the identification process for the first and second sub-regions may favor, as the first sub-region, sub-regions close to the center of the image. In other words, parts closer to the center of the image are weighed more strongly than parts further away from the center.

Steps <NUM> and <NUM> then calculate the brightness parameter B of the camera image by weighing the pixel brightness in the first sub-region more strongly than the pixel-brightness in the second sub-region. In the example, it calculates the average of the weighted sum of the pixels of the grayscale image.

In a more general embodiment, Eq. (<NUM>) may be replaced by <MAT> wherein f<NUM> and f<NUM> are monotonous functions. In other words, the brightness parameter may depend non-linearly on the product Wij · Gij.

Claim 1:
A method for controlling image exposure in an ophthalmic microscope device, wherein the ophthalmic microscope device has a camera (<NUM>) and an illumination source (<NUM>), and wherein the method comprises the steps of
recording a camera image (M) with the camera (<NUM>) using first exposure parameters,
recording an adjusted image with the camera (<NUM>) using second exposure parameters,
said method being characterized by the steps of
using edge detection to identify at least one first sub-region and at least one second sub-region of the camera image (M), wherein the first sub-region has a higher edge density than the second sub-region, and
calculating a brightness parameter (B) of the camera image (M) by weighing pixel brightness in the first sub-region more strongly than pixel-brightness in the second sub-region,
depending on the brightness parameter (B), adjusting the first exposure parameters to obtain the second exposure parameters.