Patent Description:
In the field of fluorescence microscopy, multi-channel systems have been established which enable a user to simultaneously image a plurality of fluorophore species having different emission spectra onto several image sensors. In order to spectrally split the fluorescent light emitted by the different fluorophore species, spectral image splitting arrangements are used. Such an image splitting arrangement comprises at least one splitting surface which may be formed from a cemented layer, e.g. in case of cemented dichroic coatings in prism arrangements, or an outside surface of a thin substrate such as provided in standard dichroic mirrors.

In addition to moderate edge steepness and spectral selectivity, the spectral splitting achieved by such a splitting surface is highly dependent on the incidence angle of the fluorescent light. This means that the spectral separation is highly dependent on the angle at which the fluorescent light is incident on the splitting surface, wherein this angle is assigned to a specific point within the field of view (FOV) from which the fluorescent light emerges towards the splitting surface. Thus, the afore-mentioned angle is to be understood as a chief ray angle for this particular object point.

In order to achieve a FOV-independent spectral separation, it may be considered to use an optical system which is configured to be telecentric on an image side. Thus, in case of a telecentric system, the chief ray incidence angle at which the fluorescent light is incident on the splitting surface does not vary over the FOV. Further, when a FOV independent spectral separation is secured, linear unmixing methods as disclosed e.g. in the publication of <NPL> may be applied to analyze the detected images.

However, a telecentric optical system has disadvantages as regards the size of the optical components thereof. Thus, the lenses and the spitting surface included in the optical system must not be smaller than the FOV which is detrimental in terms of costs and optical design. Also, the requirements of the pupil imaging for telecentricity usually enlarge the optical system.

A fluorescence microscope according to the preamble of claim <NUM> is known from <CIT>.

It is an object to provide a fluorescence microscope and a method enabling a reliable multi-channel imaging of an object including different fluorophore species having distinct spectral emission characteristics by means of a compact detector design.

The afore-mentioned object is achieved by the subject-matter according to the independent claims <NUM> and <NUM>, which define the present invention.

Advantageous embodiments are defined in the dependent claims and the following description.

A fluorescence microscope for imaging an object including different fluorophore species having distinct spectral emission characteristics comprises an optical system configured to collect fluorescent light emitted from the different fluorophore within a field of view (FOV) and to focus the fluorescent light for detection. The fluorescence microscope comprises a spectral splitting arrangement (spectral splitting device) configured to split the fluorescent light collected within the FOV into at least two spectrally different fluorescent light components. The fluorescence microscope comprises a multi-channel detector system comprising at least two image sensors configured to detect at least two spatial light intensity distributions based on the at least two spectrally different fluorescent light components, wherein each spatial light intensity distribution represents an image of the object over the FOV. The fluorescence microscope further comprises a processor configured to determine spatial distributions of the different fluorophore species based on a spectral unmixing analysis of each spatial light intensity distribution. The processor is further configured to obtain compensation information representing a variation of spectral characteristics of the dispersive system over said FOV and to determine a spatial distribution of each fluorophore species by taking into account the compensation information. The optical system is configured to be non-telecentric on the image side.

The fluorescence microscope as claimed allows for considering a FOV-dependent variation of the spectral separation effected by the spectral splitting arrangement. Thus, on the one hand, it is not required to configure the optical system to be telecentric on an image side or in the area of the spectral splitting arrangement, providing benefits in terms of costs and optical design. In particular, the lenses and the spectral splitting arrangement included in the optical system need not to be as large as the FOV in order to secure proper imaging. On the other hand, utilizing the compensation information which indicates a FOV-dependent variation of the spectral separation, enables a suitable unmixing analysis without having to rely on conventional linear unmixing methods.

Preferably, the processor is configured to determine intensity contributions to each spatial light intensity distribution, which are induced by the different fluorophore species, based on said spectral unmixing analysis.

The optical system may be formed by a wide field optical system. However, the fluorescence microscope is not limited thereto. According to an alternative embodiment, the optical system may serve to sequentially image the object pixel by pixel, wherein a resulting image is composed from a plurality of pixels.

According to the claimed invention, the optical system configured to be non-telecentric in the area of the spectral splitting arrangement, on an image side. As explained above, using a non-telecentric configuration has significant benefits in terms of design and costs. Moreover, the wide field optical system may comprise an exit pupil having a finite pupil position. This enables more flexibility when using a magnification changing system which needs to be configured to be an afocal system having a constant exit pupil position in order to maintain telecentricity. In particular, as telecentricity is not required with the configuration disclosed herein, a simple tube lens changer may be used which is generally not telecentric on the image side for all magnification settings.

Preferably, the spectral splitting arrangement comprises at least one splitting surface whose spectral characteristic varies depending on an incidence angle of a chief ray of the fluorescent light. The FOV-dependent characteristic of the splitting surface may be utilized to obtain the aforementioned compensation information.

The splitting surface may be configured to transmit the fluorescent light in a first spectral range and to reflect the fluorescent light in a second spectral range, said first and second spectral ranges being separated by a spectral transition range which varies depending on the incidence angle of the chief ray of the fluorescent light.

The spectral splitting arrangement may comprise at least one prism. For instance, the dispersive system may be configured as a Bauernfeind prism known from the art. Alternatively, the spectral splitting arrangement may comprise at least one plate beam splitter.

According to a preferred embodiment, the processor is configured to use a model-based spatial light intensity distribution for each image sensor, wherein the model-based spatial light intensity distribution includes the afore-mentioned compensation information as a preset parameter and the spatial distribution of each fluorophore species as a fit parameter. The processor is further configured to determine the spatial distribution of fluorophore species by optimizing the model-based spatial light intensity distribution to match the spatial light intensity distribution detected by the respective image sensor. For instance, the processor may apply an optimization procedure which is executed in order to maximize an objective function. The objective function may represent a probability for obtaining the experimental data, i.e. the spatial light intensity distribution detected by the respective image sensor, when considering all potential spatial distributions of the fluorophore species.

The model-based spatial light distribution Im(x') may be given by the following relationship: <MAT> wherein:.

On the right side of equation (<NUM>), all terms apart from cn(x) may be known. In other words, these terms may be considered as preset parameters so that the term cn(x) designating the spatial distribution of fluorophore species n is the only unknown parameter to be determined.

The model-based spatial light distribution Im(x') according to equation (<NUM>) may be derived as follows:
In a multi-color fluorescence experiment, an excitation probability of a fluorophore of the species n = a1,. , N - <NUM> when imaging in a colored channel m = <NUM>, <NUM>,. , M - <NUM> (M ≥ N) may be modeled as indicated in equation (<NUM>): <MAT>.

As already mentioned above, the term Exn(λ) designates the fluorescence excitation spectrum of fluorophore species n, and the term Illm(x,λ) designates the illumination spectrum of the light source of channel m representing the aforementioned color channel.

Further, a detection probability may be modeled as indicated in equation (<NUM>): <MAT>.

As mentioned above, Emn(λ') designates the fluorescence emission spectrum of fluorophore species n, and the term Dm(x,λ') designates the detection spectrum of image sensor of channel m in equation (<NUM>).

It is to be noted that both the detection spectrum Dm(x,λ') and the illumination spectrum Illm(x',λ) may be location-dependent. Accordingly, the spatial light distribution Im(x') according to equation (<NUM>) results from a model as defined by equations (<NUM>) and (<NUM>).

According to a preferred embodiment, the processor may be configured to control the fluorescence microscope to sequentially execute the following steps: a first step of detecting the at least two spatial light intensity distributions by means of the at least two image sensors; and a second step of determining the spatial distribution of each fluorophore species by optimizing the model-based spatial light intensity distribution to match the spatial light intensity distribution detected by the respective image sensor.

According to another preferred embodiment, the processor may be configured to control the fluorescence microscope to sequentially execute the following steps: a first step of detecting the at least two spatial light intensity distributions by means of the at least two image sensors; a second step of shifting the object relative to the optical system; a third step of detecting the at least two spatial light intensity distributions by means of the at least two image sensors on the object shifted in the second step, wherein the second and third steps are sequentially performed at least once; and a fourth step of determining the spatial distribution of each fluorophore species by optimizing the model-based spatial light intensity distribution to match the spatial light intensity light distributions detected by the respective image sensor in the first and third steps. Preferably, in the second step, the object is shifted relative to the optical system such that spectral imaging of at least one object point differs after movement. In particular, the object may be shifted perpendicular to the optical axis of the optical system. By applying this method, the spectral information to be used for the unmixing analysis can be enhanced.

The fluorescent microscope may comprise a microscope stage configured to be shifted relative to the optical system perpendicular to an optical axis thereof.

In a preferred embodiment, the fluorescence microscope comprises an illumination device configured to provide at least one of epifluorescence illumination, TIRF illumination, and light sheet illumination.

According to another aspect, a method is provided for imaging an object including different fluorophore species having distinct spectral emission characteristics, comprising the following steps: collecting fluorescent light emitted from the different fluorophore species within a FOV and focusing the fluorescent light for detection by means of an optical system; splitting the fluorescent light collected within the FOV into a plurality of spectrally different fluorescent light components by means of a spectral splitting arrangement; detecting at least two spatial light intensity distributions based on the at least two spectrally different fluorescent light components by means of a multi-channel detector system, each spatial light intensity distribution representing an image of the object over the FOV; and determining spatial distributions of the different fluorophore species based on a spectral unmixing analysis of each spatial light intensity distribution, wherein compensation information is obtained, said compensation information representing a variation of spectral characteristics of the spectral splitting arrangement over the FOV and to determine a spatial distribution of each fluorophore species by taking into account said compensation information, wherein the optical system is configured to be non-telecentric on the image side.

<FIG> shows a fluorescence microscope <NUM> which is adapted to image an object <NUM> which includes different fluorophore species having distinct spectral emission characteristics. Accordingly, the fluorescence microscope <NUM> shown in <FIG> may be used to execute a multi-color fluorescence experiment in which fluorophores of different species are excited to emit fluorescent light in different wavelength ranges. According to the embodiment of <FIG>, the fluorescence microscope <NUM> is configured as a wide field microscope without being limited thereto.

The fluorescence microscope <NUM> comprises an optical system <NUM> including an objective <NUM> facing the object <NUM> from below a microscope stage <NUM>. The microscope stage <NUM> may be a motorized stage which can be moved in a direction perpendicular to an optical axis O of the optical system <NUM>. The fluorescence microscope <NUM> further comprises an illumination device <NUM> which is configured to emit illumination light <NUM> in order to excite the fluorophore species included in the object <NUM> to emit fluorescent light. The concrete implementation of illumination may be selected depending on the specific application. For instance, the illumination device <NUM> may be configured to provide for epifluorescence illumination, TIRF illumination, or light sheet illumination as illustrated in <FIG> by different optical illumination paths <NUM>.

The fluorescence microscope <NUM> further comprises a spectral detector unit <NUM> (shown in more detail in <FIG>) and a processor <NUM> which may be configured to control the entire operation of the fluorescence microscope <NUM>. In the present context, the processor <NUM> is used to control the spectral detector unit <NUM> and the motorized microscope stage <NUM>.

As shown in <FIG>, the spectral detector unit <NUM> comprises a spectral splitting arrangement <NUM> which is configured to spatially split fluorescent light <NUM> which is collected within a field of view FOV by the optical system <NUM>. It is to be noted that the objective <NUM> being part of the optical system <NUM> is omitted in <FIG>. Further, in addition to the objective <NUM>, the optical system <NUM> comprises a tube lens <NUM> as shown in <FIG>.

The spectral detector unit <NUM> comprises a multi-channel detector system <NUM> being formed by at least two image sensors <NUM> and <NUM>. Accordingly, the multi-channel detector system <NUM> provides at least two color channels enabling multi-color imaging of the object <NUM>.

According to the specific embodiment shown in <FIG>, the spectral splitting arrangement <NUM> may be formed by a prism arrangement <NUM>, e.g. a Bauernfeind prism, comprising a cemented splitting surface <NUM>. The splitting surface <NUM> transmits the fluorescent light <NUM> in a first spectral range in order to generate a first fluorescent light component <NUM> which is received by the first image sensor <NUM>. Likewise, the splitting surface <NUM> reflects the fluorescent light <NUM> in a second spectral range being different from the first spectral range in order to generate a second fluorescent light component <NUM> which is received by the second image sensor <NUM>.

As illustrated in the diagram of <FIG>, the splitting surface <NUM> being formed by a cemented layer exhibits a spectral characteristic which varies depending on an incidence angle of the fluorescent light <NUM>. More specifically, the spectral characteristic of the splitting surface <NUM> depends on the incidence angle under which a chief ray of a fluorescent light bundle emerging from a specific point within the field of view FOV falls onto the splitting surface <NUM>. <FIG> shows three fluorescent light bundles 220a, 220b, 220c, the chief rays Pa, Pb, Pc thereof (i.e. the central ray of the respective bundle) emerging from three different points within the FOV. In <FIG>, each light bundle 220a, 220b, 220c is illustrated by three parallel light rays wherein a central ray of these light rays represents the respective chief ray Pa, Pb, Pc. Accordingly, the spectral characteristic of the splitting surface varies over the FOV. According to the example illustrated in <FIG>, the first and second spectral ranges provided by the splitting surface <NUM> are separated from each other by a spectral transition range having a width of more than <NUM> where the transmission raises from <NUM> to <NUM> (for a specific incidence angle). The width of the transition range defines an edge steepness of the spectral characteristic of the splitting surface <NUM>. Further, the transition range is shifted towards larger wavelengths with an increasing incidence angle of the fluorescent light as illustrated by an arrow Q in <FIG>. In the example of <FIG>, an increase of the incidence angle from <NUM>° to <NUM>° results in a shift of the transition range of approximately <NUM> to <NUM>.

As a result of the spectral characteristic shown in <FIG>, the spectral splitting of the fluorescent light <NUM> into the spectrally different fluorescent light components <NUM>, <NUM> varies over the FOV, i.e. is not constant for the different points within the FOV which correspond to the different chief rays Pa, Pb, Pc. Thus, a spectral gradient occurs in the direction of the splitting. As a result, conventional spectral unmixing procedures cannot be applied as these procedures require a constant spectral splitting over the entire FOV.

In order to avoid the problems due to a variation of the spectral splitting over the FOV, it may be considered to configure the optical system <NUM> as a system which is telecentric on the image side. <FIG> shows such a telecentric configuration as comparative example.

In order to enable the splitting surface <NUM> to achieve a constant spectral splitting over the entire FOV, an entrance pupil <NUM> is required to be imaged to infinity. In case that the entrance pupil <NUM> is imaged to infinity, the chief rays Pa, Pb, Pc of all fluorescent light bundles 220a, 220b, 220c emerging from the different points within the FOV fall under the same incidence angle onto the splitting surface <NUM>. As a result, there is no variation of the spectral characteristic of the splitting surface <NUM> over the FOV. In other words, the spectral characteristic is translationally invariant over the FOV so that conventional unmixing methods can be applied.

However, as can also be seen from <FIG>, the cross-sections of the beam path at the tube lens <NUM> and the splitting surface <NUM> are relatively large. Specifically, the lens and prism surfaces must be at least as large as the FOV.

In order to avoid the disadvantage of large lens and prism surfaces, the configuration shown in <FIG> is configured to be non-telecentric on the image side. For this, the entrance pupil <NUM> is e.g. located such that it coincides with the tube lens <NUM>. Accordingly, the optical system <NUM> comprises an exit pupil having a finite pupil position. As can be seen in <FIG>, the non-telecentric configuration enables the cross-sections of the beam path at the locations of the tube lens <NUM> and the splitting surface <NUM> to be reduced. As a result, a compact design can be achieved.

According to the configuration shown in <FIG>, the processor <NUM> controls each image sensor <NUM>, <NUM> to detect a spatial light intensity distribution based on the respective fluorescent light component <NUM>, <NUM>. Each of the spatial light intensity distributions detected by the image sensors <NUM>, <NUM> represents an image of the object over the entire FOV. Then, the processor <NUM> performs a spectral unmixing analysis of each spatial light intensity distribution detected by the respective image sensor <NUM>, <NUM> in order to determine spatial distributions of the different fluorophore species.

In order to cope with the fact that according to the configuration of <FIG> the spectral splitting caused by the splitting surface <NUM> varies over the FOV, the processor <NUM> is configured to take into account compensation information representing the variation of the spectral characteristic of the splitting surface <NUM> over the FOV when performing the spectral unmixing analysis.

More specifically, when performing the spectral unmixing analysis, the processor <NUM> determines intensity contributions to each spatial light intensity distribution. For instance, the processor may apply a model-based spatial light intensity distribution for each image sensor. This model-based spatial light distribution may be given by equation (<NUM>) as explained above. The model-based spatial light intensity distribution includes the required compensation information as a preset parameter and the spatial distribution of each fluorophore species as a fit parameter. Then, the processor <NUM> determines the spatial distribution of each fluorophore species by optimizing the model-based spatial light intensity distribution to match the spatial light intensity distribution detected by the respective image sensor.

<FIG> shows a flow diagram illustrating an embodiment of a method for determining the spatial distribution of each fluorophore species.

In step S2, the processor <NUM> causes each of the image sensors <NUM>, <NUM> to detect a spatial light intensity distribution which represents an image of the object over the entire FOV.

In step S4, the processor <NUM> optimizes the model-based spatial light intensity distribution according to equation (<NUM>) such that it matches the experimental data which is represented by the spatial light intensity distribution detected by the respective image sensor <NUM>, <NUM>. In other words, the processor reconstructs the term cn(x) included in the model-based spatial light intensity distribution according to equation (<NUM>) by applying a suitable optimization method.

<FIG> shows a flow diagram illustrating a modified method which uses additional spectral information.

In step S6 of the method shown in <FIG>, the processor <NUM> causes each of the image sensors <NUM>, <NUM> to detect a spatial light intensity distribution which represents an image of the object over the entire FOV.

In step S8, the processor <NUM> controls the motorized microscope stage <NUM> to move the object <NUM> relative to the optical system <NUM> perpendicular to the optical axis O thereof. In other words, the processor controls the FOV to be moved over the object <NUM>. Subsequently, in step S10, the processor <NUM> causes each of the image sensors <NUM>, <NUM> to detect a spatial light intensity distribution on the object <NUM> which has been shifted in step S8. Thus, the light intensity distribution detected by each image sensor <NUM>, <NUM> represents the shifted FOV.

The steps S8 and S10 are sequentially performed once or repeated in a number of loops L in order to increase the spectral information which be used in the spectral unmixing analysis.

Finally, in step S12, the processor <NUM> determines the spatial distribution of each fluorophore species. For this, the processor <NUM> applies an optimization method on the model-based spatial light intensity distribution according to equation (<NUM>) such that it matches the spatial light intensity distributions detected by the respective image sensor <NUM>, <NUM> in steps S6 and S8.

Needless to say that the invention shall not be limited to the specific embodiments described above. For instance, the optical system <NUM> is formed by a wide field system. However, the optical system may also be configured to sequentially image the object pixel by pixel, e.g. in a scanning microscope. In this case, the processor is configured to combine a plurality of pixel signals to a resulting image to be analyzed as explained above.

Further, the optimization based on equation (<NUM>) is to be understood merely as an example. Any other suitable algorithm may be applied to compensate for the FOV dependent spectral splitting.

Claim 1:
A fluorescence microscope (<NUM>) for imaging an object (<NUM>) including different fluorophore species having distinct spectral emission characteristics, comprising:
an optical system (<NUM>) configured to collect fluorescent light (<NUM>) emitted from the different fluorophore species within a field of view (FOV) and to focus the fluorescent light for detection,
a spectral splitting arrangement (<NUM>) configured to split the fluorescent light (<NUM>) collected within said field of view (FOV) into at least two spectrally different fluorescent light components (<NUM>, <NUM>),
a multi-channel detector system (<NUM>) comprising at least two image sensors (<NUM>, <NUM>) configured to detect at least two spatial light intensity distributions based on said at least two spectrally different fluorescent light components (<NUM>, <NUM>), each spatial light intensity distribution representing an image of the object (<NUM>) over said field of view (FOV),
a processor (<NUM>) configured to determine spatial distributions of the different fluorophore species based on a spectral unmixing analysis of each spatial light intensity distribution,
wherein the processor (<NUM>) is further configured to obtain compensation information representing a variation of spectral characteristics of the spectral splitting arrangement (<NUM>) over said field of view (FOV) and to determine a spatial distribution of each fluorophore species by taking into account said compensation information,
characterized in that the optical system (<NUM>) is configured to be non-telecentric on the image side.