Patent Description:
Hyperspectral imaging may be used to analyze biological samples. Maximizing information throughput from biological samples may help distinguish between different cell types and tissue types. A high throughput may be helpful in a surgical environment to make quick decisions at an operating table, or to efficiently build up a database of information for machine learning applications. In addition, a high throughput may enable the capturing of rapid dynamics or motion, or studies of cell signaling dynamics or protein diffusion.

Some hyperspectral imaging systems use tunable filters, such as liquid crystal filters, in the emission path to capture the fluorescence at reasonably high spectral resolution (~<NUM>-<NUM>). However, this approach suffers from the disadvantage of bleaching, because the filters use absorption to selectively capture a small slice of the fluorescence. Bleaching is problematic because fluorophores often have a limited number of photons that they can emit before going dark. Bleaching is especially problematic for tunable filters because it causes the emission intensity to change while the filter is scanned, distorting the spectrum and causing errors when spectral unmixing is used to estimate fluorophore concentrations from the hyperspectral data. Also, these systems use a wide-field approach that prevents confocality or depth sectioning from being performed, which makes their use with thick tissue samples problematic.

Other hyperspectral imaging systems use a digital micromirror device (DMD) or a cylindrical lenslet array to produce patterned illumination on the sample. A grating or prism structure spatially disperses the fluorescence across a two-dimensional imaging sensor to acquire a hyperspectral image. The patterned illumination is then scanned across the sample (or the sample is scanned with the patterned illumination remaining fixed) to obtain a hyperspectral data cube. These systems reduce bleaching, because all of the excitation photons are collected, and since the entire spectrum is collected at once, the spectral distortion is further reduced. Furthermore, these systems can provide depth sectioning, which allow them to be used with thicker tissue samples. However, these systems rely on scanning either one depth at a time or one excitation laser at a time, thereby limiting the data acquisition speed and throughput.

<CIT> discloses an apparatus for three-dimensional laser scanning microscopy, where excitation light is focused into the sample in a three-dimensional matrix of focal points. Real-time three-dimensional image acquisition is obtained by fast scanning in the xy plane only. <CIT> discloses an imager including front-end optics for outputting a polychromatic, collimated image beam of the scene; a beam displacer configured for splitting the collimated image beam into spatially displaced, mutually parallel beams, and an imaging-sensor array configured for registration of the spatially displaced wavelength sets at disparate locations along the imaging-sensor array. <CIT> discloses snapshot imaging spectrometer systems having high spectral and spatial resolutions. <CIT> discloses a hyperspectral imaging system, comprising: a sample holder configured to hold a sample; a light source configured to emit excitation light having one or more wavelengths; a two-dimensional imaging device; and an optical system comprising: a first spatial light modulator; a first dispersive element; and a second dispersive element. <CIT> discloses a single-shot spectral imager which uses dispersive optics together with spatial light modulators to encode a mathematical transform onto acquired spatial-spectral data. A multitude of encoded images is recorded simultaneously on a focal plane array and subsequently decoded to produce a spectral/spatial hypercube.

Exemplary embodiments of the invention provide high-throughput hyperspectral imaging systems. A system according to a first aspect of the invention is set out in claim <NUM>. The system includes an excitation light source that is configured to emit excitation light; an objective that is configured to receive the excitation light from the excitation light source and image the excitation light onto the sample, such that the excitation light causes the sample to emit fluorescence light; a channel separator that is configured to receive the fluorescence light from the sample and separate the fluorescence light into a plurality of spatially dispersed spectral channels; and a sensor that is configured to receive the plurality of spatially dispersed spectral channels from the channel separator. The excitation light source includes a light source and a plurality of first lenslet arrays. Each of the plurality of first lenslet arrays is configured to receive light from the light source and to generate a pattern of light, and the patterns of light generated by the plurality of first lenslet arrays are combined to form the excitation light. The objective is configured to simultaneously image each of the patterns of light to form a plurality of parallel lines or an array of circular spots at different depths of the sample.

In some embodiments, the channel separator may include a reflective layer having a plurality of first reflective elements, wherein each first reflective element of the plurality of first reflective elements is configured to reflect a first portion of the fluorescence light that is generated by a first pattern of light of the patterns of light; a patterned layer that is configured to transmit a second portion of the fluorescence light that is generated by a second pattern of light of the patterns of light; first dispersion optics that are configured to receive the first portion of the fluorescence light from the reflective layer and to spatially disperse spectral components of the first portion of the fluorescence light; and second dispersion optics that are configured to receive the second portion of the fluorescence light from the reflective layer and to spatially disperse spectral components of the second portion of the fluorescence light.

In other embodiments, the channel separator may include a second lenslet array that is configured to focus the fluorescence light, and dispersion optics that are configured to receive the fluorescence light from the second lenslet array and to spatially disperse spectral components of the fluorescence light. The second lenslet array may include a plurality of linear arrays of lenslets that are configured to receive the fluorescence light as a plurality of parallel lines.

In other embodiments, the channel separator may include dispersion optics that are configured to spatially disperse spectral components of the fluorescence light, and a second lenslet array that is configured to receive the fluorescence light from the dispersion optics and to focus the fluorescence light. The second lenslet array comprises a plurality of linear arrays of lenslets that are configured to receive the fluorescence light as a plurality of parallel lines.

The system may also include a dichroic beamsplitter that is configured to reflect the excitation light from the excitation light source toward the objective, and to transmit the fluorescence light from the sample toward the channel separator. A transmission spectrum of the dichroic beamsplitter may include a notch that coincides with a wavelength of the light from the light source.

The excitation light source may include a plurality of light sources, wherein each of the plurality of light sources emits light having a different wavelength, wherein each of the plurality of first lenslet arrays is configured to receive light from one of the plurality of light sources and to generate a pattern of light.

In other embodiments, the channel separator may include dispersion optics that are configured to spatially disperse spectral components of the fluorescence light, and a second lenslet array that is configured to receive the fluorescence light from the dispersion optics and to focus the fluorescence light. The second lenslet array may include a plurality of linear arrays of lenslets that are configured to receive the fluorescence light as a plurality of parallel lines.

The system may also include a dichroic beamsplitter that is configured to reflect the excitation light from the excitation light source toward the objective, and to transmit the fluorescence light from the sample toward the channel separator. A transmission spectrum of the dichroic beamsplitter may include a plurality of notches that coincide with the different wavelengths from the light sources.

According to yet another aspect of the invention, a system includes an excitation light source that is configured to emit excitation light; an objective that is configured to receive the excitation light from the excitation light source and image the excitation light onto the sample, such that the excitation light causes the sample to emit fluorescence light; a channel separator that is configured to receive the fluorescence light from the sample and separate the fluorescence light into a plurality of spatially dispersed spectral channels; and a sensor that is configured to receive the plurality of spatially dispersed spectral channels from the channel separator. The excitation light source includes a plurality of light sources, wherein each of the plurality of light sources emits light having a different wavelength, and a plurality of first lenslet arrays, wherein each of the plurality of first lenslet arrays is configured to receive light from one of the plurality of light sources and to generate a pattern of light, and the patterns of light generated by the plurality of first lenslet arrays are combined to form the excitation light. The objective is configured to simultaneously image the patterns of light to form a plurality of parallel lines or an array of circular spots at a plurality of depths of the sample.

In some embodiments, the channel separator includes a reflective layer having a plurality of first reflective elements, wherein each first reflective element of the plurality of first reflective elements is configured to reflect a first portion of the fluorescence light that is generated by a first pattern of light of the patterns of light; a patterned layer that is configured to transmit a second portion of the fluorescence light that is generated by a second pattern of light of the patterns of light; first dispersion optics that are configured to receive the first portion of the fluorescence light from the reflective layer and to spatially disperse spectral components of the first portion of the fluorescence light; and second dispersion optics that are configured to receive the second portion of the fluorescence light from the reflective layer and to spatially disperse spectral components of the second portion of the fluorescence light.

In other embodiments, the channel separator includes a second lenslet array that is configured to focus the fluorescence light, and dispersion optics that are configured to receive the fluorescence light from the second lenslet array and to spatially disperse spectral components of the fluorescence light. The second lenslet array may include a plurality of linear arrays of lenslets that are configured to receive the fluorescence light as a plurality of parallel lines.

In other embodiments, the channel separator includes dispersion optics that are configured to spatially disperse spectral components of the fluorescence light, and a second lenslet array that is configured to receive the fluorescence light from the dispersion optics and to focus the fluorescence light. The second lenslet array includes a plurality of linear arrays of lenslets that are configured to receive the fluorescence light as a plurality of parallel lines.

In some embodiments, a number of the plurality of light sources may equal a number of the plurality of first lenslet arrays. In other embodiments, a number of the plurality of light sources may be less than a number of the plurality of first lenslet arrays.

Other objects, advantages, and novel features of the present invention will become apparent from the following detailed description of the invention when considered in conjunction with the accompanying drawings.

<FIG> shows a block diagram of a high-throughput hyperspectral imaging system <NUM> according to exemplary embodiments of the invention. The high-throughput hyperspectral imaging system <NUM> shown in <FIG> may acquire data in three, four, or five dimensions. As discussed above, three-dimensional imaging data is acquired for two spatial dimensions (x,y) and one spectral dimension (λ) of the emission light. Four-dimensional imaging data adds either the excitation wavelength or the depth where the excitation light is focused on the sample as the fourth dimension. Five-dimensional imaging data adds both the excitation wavelength and the depth where the excitation light is focused on the sample. Advantageously all of the dimensions may be scanned simultaneously, thereby increasing the throughput of the hyperspectral imaging system <NUM>.

As shown in <FIG>, the system <NUM> includes an excitation light source <NUM> that emits excitation light <NUM>. The excitation light source <NUM> may include a single light source that emits light at a single wavelength, or a plurality of light sources that emit light at different wavelengths. The light sources may be lasers, light-emitting diodes (LEDs), or any other suitable source of light for hyperspectral imaging. The excitation light source <NUM> includes a plurality of lenslet arrays. The positions of each of the lenslet arrays may be adjusted to select the depth of the sample <NUM> at which light passing through the respective lenslet array is focused. Each of the lenslet arrays may include a plurality of parallel cylindrical lens elements or a two-dimensional matrix of circular lens elements. A lenslet array having the plurality of parallel cylindrical lens elements generates a pattern of light that is imaged on the sample <NUM> as a series of parallel lines. In contrast, a lenslet array having the two-dimensional matrix of circular lens elements generates a pattern of light that is imaged on the sample <NUM> as a two-dimensional matrix of circular spots.

As shown in <FIG>, the excitation light <NUM> may be collimated by a first tube lens <NUM> and transmitted to an excitation/emission separator <NUM>. As discussed in further detail below, the excitation/emission separator <NUM> may reflect the excitation light <NUM> toward the sample <NUM>. An objective <NUM> then images the excitation light <NUM> onto the sample <NUM>. When the excitation light <NUM> is incident on the sample <NUM>, the excitation light <NUM> causes the sample <NUM> to emit fluorescence light <NUM>. If the sample <NUM> is labeled with fluorophores, the fluorescence light <NUM> may be emitted by the fluorophores. Alternatively or in addition, the fluorescence light <NUM> may be autofluorescence light from the sample <NUM>. Although the system <NUM> shown in <FIG> is configured to operate in an epi-illumination mode in which the excitation light <NUM> is incident on the sample <NUM> from above, the system <NUM> could be modified to operate in a transillumination mode in which the excitation light <NUM> is transmitted through the sample <NUM> from below. In this embodiment, the excitation/emission separator <NUM> may be removed from the system <NUM>, or it may be replaced by a single-notch, long-pass, or multi-notch filter that blocks the excitation light <NUM> that is transmitted through the sample <NUM>. In this embodiment, the sample <NUM> should be thin enough to transmit the excitation light <NUM> through the depth of the sample, such that sufficient fluorescence light <NUM> is emitted toward the objective <NUM>.

After passing through the objective <NUM>, the fluorescence light <NUM> may be transmitted by the excitation emission separator <NUM>, which blocks the further transmission of the excitation light <NUM>. The fluorescence light <NUM> may then be focused by a second tube lens <NUM> onto a channel separator <NUM>. As discussed in further detail below, the channel separator <NUM> separates the fluorescence light <NUM> into a plurality of spatially dispersed spectral channels, which are incident on the sensor <NUM>. Although only one sensor <NUM> is shown, the system <NUM> may include a plurality of sensors <NUM>. For example, each of the plurality of sensors <NUM> may receive the portion of the fluorescence light <NUM> that was generated by a single wavelength of the excitation light <NUM>.

<FIG> shows a diagram of various illumination patterns that may be produced on the sample <NUM> by the high-throughput hyperspectral imaging system <NUM>. This embodiment uses a cylindrical lenslet array within the excitation light source <NUM>. As shown in <FIG>, the excitation light <NUM> is incident on an excitation area <NUM> of the sample <NUM>. The excitation light <NUM> may be focused at various depths along the z direction of the sample <NUM>. For example, <FIG> shows a first projection <NUM> of the illumination on the x-z plane for several z slices. Similarly, <FIG> shows a second projection <NUM> of the illumination on the y-z plane for several z slices. The thicknesses of the lines within illumination patterns <NUM>, <NUM>, <NUM>, and <NUM> represent the widths of the lines of the excitation light <NUM> at different depths of the sample <NUM>. The focus of the excitation light <NUM> is shown as the thinnest lines. For example, the second row of illumination pattern <NUM> shows where the excitation light <NUM> is focused on the sample. The other rows of illumination pattern <NUM> show how the widths of the lines are different at different axial positions.

<FIG> shows illumination patterns <NUM>, <NUM>, <NUM>, and <NUM> of the second projection <NUM> in further detail for various embodiments. The numbers beneath illumination patterns <NUM>, <NUM>, <NUM>, and <NUM> indicate the wavelength of the excitation light <NUM>. For example, a first wavelength (labeled as "<NUM>") is used for illumination patterns <NUM> and <NUM>, while a second wavelength (labeled as "<NUM>"), a third wavelength (labeled as "<NUM>", and a fourth wavelength (labeled as "<NUM>") are used for illumination pattern <NUM>. Each of the wavelengths may have any suitable value. Specifically, illumination pattern <NUM> corresponds to three-dimensional imaging in which there are two spatial dimensions and one spectral dimension of the emission light. Illumination pattern <NUM> may be obtained by using a single light source that emits a single wavelength with a single lenslet array in the excitation light source <NUM>. Illumination pattern <NUM> corresponds to four-dimensional imaging in which there are two spatial dimensions and one spectral dimension of the emission light, along with the depth where the excitation light is focused on the sample as the fourth dimension. Illumination pattern <NUM> may be obtained by using a single light source that emits a single wavelength with a plurality of lenslet arrays in the excitation light source <NUM>. Alternatively, illumination pattern <NUM> may be obtained by using a single lenslet array in which adjacent lenslets have different focal lengths. Illumination pattern <NUM> corresponds to four-dimensional imaging in which there are two spatial dimensions and one spectral dimension of the emission light, along with the excitation wavelength as the fourth dimension. Illumination pattern <NUM> may be obtained by using a plurality of light sources that emit a plurality of wavelengths with a single lenslet array in the excitation light source <NUM>. Illumination pattern <NUM> corresponds to five-dimensional imaging in which there are two spatial dimensions and one spectral dimension of the emission light, along with the depth where the excitation light is focused on the sample and the excitation wavelength as the fourth and fifth dimensions. Illumination pattern <NUM> may be obtained by using a plurality of light sources that emit a plurality of wavelengths with a plurality of lenslet arrays in the excitation light source <NUM>.

<FIG> shows another diagram of various illumination patterns that may be produced on the sample <NUM> by the high-throughput hyperspectral imaging system <NUM>. This embodiment uses a circular lenslet array within the excitation light source <NUM>. As shown in <FIG>, the excitation light <NUM> is incident on an excitation area <NUM> of the sample <NUM>. The excitation light <NUM> may be focused at various depths along the z direction of the sample <NUM>. For example, <FIG> shows a first projection <NUM> of the illumination on the x-z plane for several z slices. Similarly, <FIG> shows a second projection <NUM> of the illumination on the y-z plane for several z slices. The thicknesses of the lines within illumination patterns <NUM>, <NUM>, <NUM>, and <NUM> represent the widths of the lines of the excitation light <NUM> at different depths of the sample <NUM>. The focus of the excitation light <NUM> is shown as the thinnest lines. For example, the second row of illumination pattern <NUM> shows where the excitation light <NUM> is focused on the sample. The other rows of illumination pattern <NUM> show how the widths of the lines are different at different axial positions.

In order to generate a hyperspectral data cube, the sample <NUM> may be scanned along the x direction shown in <FIG> for embodiments that use cylindrical lenslet arrays within the excitation light source <NUM>. Alternatively, the lenslet arrays within the excitation light source <NUM> may be scanned along the x direction for these embodiments. For illumination pattern <NUM>, no additional scanning is needed if only one excitation wavelength is needed, provided that the axial resolution and range of focal depths are such that all the desired depth slices of the sample can be interrogated simultaneously. On the other hand, if the spacing is more sparse, such as if the spacing between the rows in illumination pattern <NUM> is <NUM> and the axial resolution is <NUM>, and data is desired at steps greater than <NUM> steps, the depth may also be scanned by moving the sample <NUM> or the lenslet arrays axially. If more than one excitation wavelength is required, then the excitation wavelength would be changed for all lines. For illumination pattern <NUM>, no additional scanning is needed if the sample <NUM> is very thin and only one depth is needed. However, if an axial scan is needed, then the sample <NUM> may be scanned along the z direction. For illumination pattern <NUM>, the sample <NUM> may be scanned along the z direction. It should be noted that the information content obtained per scan is the same for each of the illumination patterns <NUM>, <NUM>, <NUM>, and <NUM>. Therefore, the choice of which illumination pattern to use may depend on the characteristics of the sample <NUM> and what kind of information is desired. For example, for a sufficiently thin sample <NUM>, depth scanning may not be required. If information from only one depth is needed, then illumination pattern <NUM> may be used. Further, for a thick sample <NUM> with a family of fluorophores only excited by a single wavelength, wavelength scanning may not be required. If information from only one excitation wavelength is needed, illumination pattern <NUM> may be used. If information from an instantaneous coarse snapshot at a few excitation wavelengths and a few depths is needed, illumination pattern <NUM> may be used.

For embodiments that use circular lenslet arrays within the excitation light source <NUM>, the sample <NUM> may be scanned along the y direction shown in <FIG>. Alternatively, the lenslet arrays within the excitation light source <NUM> may be scanned along the y direction for these embodiments. For illumination pattern <NUM>, no additional scanning is needed if only one excitation wavelength is needed, provided that the axial resolution and range of focal depths are such that all the desired depth slices of the sample can be interrogated simultaneously. On the other hand, if the spacing is more sparse, such as if the spacing between the rows in illumination pattern <NUM> is <NUM> and the axial resolution is <NUM>, and data is desired at steps greater than <NUM> steps, the depth may also be scanned by moving the sample <NUM> or the lenslet arrays axially. If more than one excitation wavelength is required, then the excitation wavelength would be changed for all lines. For illumination pattern <NUM>, no additional scanning is needed if the sample <NUM> is very thin and only one depth is needed. However, if an axial scan is needed, then the sample <NUM> may be scanned along the z direction. For illumination pattern <NUM>, the sample <NUM> may be scanned along the z direction. It should be noted that the information content obtained per scan is the same for each of the illumination patterns <NUM>, <NUM>, <NUM>, and <NUM>. Therefore, the choice of which illumination pattern to use may depend on the characteristics of the sample <NUM> and what kind of information is desired. For example, for a sufficiently thin sample <NUM>, depth scanning may not be required. If information from only one depth is needed, then illumination pattern <NUM> may be used. Further, for a thick sample <NUM> with a family of fluorophores only excited by a single wavelength, wavelength scanning may not be required. If information from only one excitation wavelength is needed, illumination pattern <NUM> may be used. If information from an instantaneous coarse snapshot at a few excitation wavelengths and a few depths is needed, illumination pattern <NUM> may be used.

<FIG> show a diagram of an excitation light source that may be used to produce illumination pattern <NUM> for four-dimensional imaging in which there are two spatial dimensions and one spectral dimension of the emission light, along with the depth where the excitation light is focused on the sample as the fourth dimension. The excitation light source <NUM> may include a light source <NUM> that emits light <NUM> at a single wavelength. As shown in <FIG>, the light <NUM> may be divided into four different paths, such that each path travels through a different lenslet array. Light from the four paths is then combined at the non-polarizing beamsplitter <NUM> to form the excitation light <NUM>. Although <FIG> show a specific example for dividing and recombining the light, any suitable components and layout may be used.

For example, as shown in <FIG>, a first portion of the light <NUM> passes through a half-wave plate <NUM>, a polarizing beamsplitter <NUM>, a half-wave plate <NUM>, and a polarizing beamsplitter <NUM>. The first portion of the light <NUM> is then reflected by two reflectors <NUM> and <NUM>, and passes through a first lenslet array <NUM>. The first portion of the light <NUM> then passes through polarizing beamsplitter <NUM> and is reflected by reflector <NUM> and non-polarizing beamsplitter <NUM> to form part of the excitation light <NUM>. A dump <NUM> is included to absorb any extra light that is transmitted through the non-polarizing beamsplitter <NUM>.

Further, as shown in <FIG>, a second portion of the light <NUM> passes through the half-wave plate <NUM>, the polarizing beamsplitter <NUM>, and the half-wave plate <NUM>. The second portion of the light <NUM> is then reflected by the polarizing beamsplitter <NUM>, and passes through a second lenslet array <NUM>. The second portion of the light is then reflected by the polarizing beamsplitter <NUM>, the reflector <NUM>, and the non-polarizing beamsplitter <NUM> to form part of the excitation light <NUM>.

In addition, as shown in <FIG>, a third portion of the light <NUM> passes through the half-wave plate <NUM> and is reflected by the polarizing beamsplitter <NUM>. The third portion of the light <NUM> then passes through a half-wave plate <NUM>, a polarizing beamsplitter <NUM>, a third lenslet array <NUM>, and a polarizing beamsplitter <NUM>. Next the third portion of the light <NUM> is reflected by a reflector <NUM> and passes through the non-polarizing beamsplitter <NUM> to form part of the excitation light <NUM>.

Further, as shown in <FIG>, a fourth portion of the light <NUM> passes through the half-wave plate <NUM> and is reflected by the polarizing beamsplitter <NUM>. The fourth portion of the light <NUM> then passes through the half-wave plate <NUM> and is reflected by the polarizing beamsplitter <NUM>. Next the fourth portion of the light is reflected by reflectors <NUM> and <NUM>, passes through a fourth lenslet array <NUM>, is reflected by the polarizing beamsplitter <NUM> and the reflector <NUM>, and passes through the non-polarizing beamsplitter <NUM> to form part of the excitation light <NUM>.

<FIG> shows a diagram of an excitation light source, not falling under the scope of the invention, that may be used to produce illumination pattern <NUM> for four-dimensional imaging in which there are two spatial dimensions and one spectral dimension of the emission light, along with the excitation wavelength as the fourth dimension. The excitation light source <NUM> may include a plurality of light sources <NUM>, <NUM>, <NUM>, and <NUM> that emit light <NUM>, <NUM>, <NUM>, and <NUM>, respectively, where each light <NUM>, <NUM>, <NUM>, and <NUM> has a different wavelength. As shown in <FIG>, the light <NUM>, <NUM>, <NUM>, and <NUM> may be combined via dichroic mirrors <NUM>, <NUM>, <NUM>, and <NUM> before reaching a single lenslet array <NUM>. The combined light is then collimated by a tube lens <NUM> and dispersed by a dispersing element <NUM>, such as a prism or a grating, to form the excitation light <NUM>. Although <FIG> shows a specific example for combining the light <NUM>, <NUM>, <NUM>, and <NUM>, any suitable components and layout may be used.

<FIG> show a diagram of an excitation light source that may be used to produce illumination pattern <NUM> for five-dimensional imaging in which there are two spatial dimensions and one spectral dimension of the emission light, along with the depth where the excitation light is focused on the sample and the excitation wavelength as the fourth and fifth dimensions. The excitation light source <NUM> may include a plurality of light sources <NUM>, <NUM>, <NUM>, and <NUM> that emit light <NUM>, <NUM>, <NUM>, and <NUM> at different wavelengths. As shown in <FIG>, each light <NUM>, <NUM>, <NUM>, and <NUM> travels through a respective lenslet array <NUM>, <NUM>, <NUM>, and <NUM>. The light <NUM>, <NUM>, <NUM>, and <NUM> is then combined at dichroic <NUM> to form the excitation light <NUM>. Although <FIG> show a specific example for combining the light, any suitable components and layout may be used.

For example, as shown in <FIG>, the light <NUM> from light source <NUM> passes through lenslet array <NUM> and is reflected by dichroics <NUM> and <NUM> to form part of the excitation light <NUM>. As shown in <FIG>, the light <NUM> from light source <NUM> passes through lenslet array <NUM> and dichroic <NUM>, and is reflected by dichroic <NUM> to form part of the excitation light <NUM>. As shown in <FIG>, the light <NUM> from light source <NUM> passes through lenslet array <NUM>, is reflected by dichroic <NUM>, and passes through dichroic <NUM> to form part of the excitation light <NUM>. As shown in <FIG>, the light <NUM> from light source <NUM> passes through lenslet array <NUM>, dichroic <NUM>, and dichroic <NUM> to form part of the excitation light <NUM>.

<FIG> show a diagram of another excitation light source that may be used to produce illumination pattern <NUM> for five-dimensional imaging in which there are two spatial dimensions and one spectral dimension of the emission light, along with the depth where the excitation light is focused on the sample and the excitation wavelength as the fourth and fifth dimensions. The excitation light source <NUM> may include a plurality of light sources <NUM> and <NUM> that emit light <NUM> and <NUM> at different wavelengths. As shown in <FIG>, each light <NUM> and <NUM> is divided into two paths, such that each of the total of four paths travels through a different lenslet array. Light from the four paths is then combined at the dichroic <NUM> to form the excitation light <NUM>. Although <FIG> show a specific example for dividing and recombining the light, any suitable components and layout may be used.

For example, as shown in <FIG>, a portion of the light <NUM> from light source <NUM> passes through half-wave plate <NUM> and polarizing beamsplitter <NUM>, is reflected by reflectors <NUM> and <NUM>. and then passes through first lenslet array <NUM>. The light then passes through polarizing beamsplitter <NUM> and is reflected by reflector <NUM> and dichroic <NUM> to form part of the excitation light <NUM>.

Further, as shown in <FIG>, a portion of the light <NUM> from light source <NUM> passes through half-wave plate <NUM> and is reflected by the polarizing beamsplitter <NUM>. The light then passes through second lenslet array <NUM>. Next the light is reflected by the polarizing beamsplitter <NUM>, the reflector <NUM>, and the dichroic <NUM> to form part of the excitation light <NUM>.

In addition, as shown in <FIG>, a portion of the light <NUM> from light source <NUM> passes through half-wave plate <NUM> and is reflected by polarizing beamsplitter <NUM>. The light is then reflected by reflector <NUM> and passes through third lenslet array <NUM>. Next the light is reflected by polarizing beamsplitter <NUM> and passes through dichroic <NUM> to form part of the excitation light <NUM>.

Further, as shown in <FIG>, a portion of the light <NUM> from light source <NUM> passes through the half-wave plate <NUM> and the polarizing beamsplitter <NUM>. The light is then reflected by reflector <NUM> and passes through fourth lenslet array <NUM>. Next the light passes through the polarizing beamsplitter <NUM> and the dichroic <NUM> to form part of the excitation light <NUM>.

<FIG> shows a diagram of an example of an excitation/emission separator <NUM>. As shown in <FIG>, the excitation light <NUM> from the excitation light source <NUM> is incident on a dichroic beamsplitter <NUM>, which reflects the excitation light <NUM> toward the objective <NUM>. A beam dump <NUM> may also be provided to absorb leakage light <NUM> of the excitation light <NUM> that is not reflected by the dichroic beamsplitter <NUM>. After the excitation light <NUM> has caused the sample <NUM> to emit the fluorescence light <NUM>, the dichroic beamsplitter <NUM> transmits the fluorescence light <NUM> toward the channel separator <NUM>.

<FIG> show examples of transmission spectra of the dichroic beamsplitter <NUM>. These graphs show the transmission of the dichroic beamsplitter <NUM> as a function of wavelength. The notches in the spectra coincide with the center wavelengths of the excitation light <NUM>. This example assumes that the excitation light source <NUM> includes three light sources that emit different wavelengths. The transmission spectra shown in <FIG> having narrow notches may be preferable in order to capture as much of the spectrum of the fluorescence light <NUM> as possible. For example, the notches may have a line width of less than <NUM>. It is unnecessary for the transmission of the notches to go to zero.

<FIG> shows a diagram of an example of a channel separator that separates the fluorescence light <NUM> into a plurality of spatially dispersed spectral channels. Each of the channels includes a portion of the fluorescence light <NUM> that was generated by one of the illumination patterns shown in <FIG>. Specifically, each of the channels includes the portion of the fluorescence light <NUM> that was generated by a single excitation wavelength at a single depth of the sample <NUM>. <FIG> shows an example in which five channels are used; however, the channel separator may be modified for any suitable number of channels.

As shown in <FIG>, the channel separator <NUM> may include an optic that has a first reflective layer <NUM> and a patterned layer <NUM>. The first reflective layer <NUM> includes a plurality of reflective elements, and the patterned layer <NUM> includes regions that transmit or absorb the fluorescence light <NUM>. The fluorescence light <NUM> is imaged by the objective <NUM> and the tube lens <NUM> as a series of lines that matches the illumination pattern of the excitation light. The configuration of the first reflective layer <NUM> and the patterned layer <NUM> is based on the spacing between the lines and the magnification provided by the objective <NUM> and the tube lens <NUM>. For example, the first reflective layer <NUM> and the patterned layer <NUM> are configured such that a first channel <NUM> of the fluorescence light <NUM> is transmitted through the optic. A portion of the patterned layer <NUM> that is not covered by the reflectors of the first reflective layer <NUM> may transmit light from the first channel <NUM>. The light from the first channel <NUM> may then be collimated by a lens <NUM>, dispersed by dispersion optics <NUM> and <NUM>, and imaged by a lens <NUM> onto a sensor <NUM>. For example, the dispersion optics <NUM> and <NUM> may be a set of double Amici prisms. The dispersion optics <NUM> and <NUM> may be adjusted to provide more or less dispersion, such that lines of the first channel <NUM> are dispersed across the entire sensor <NUM>. For example, if the first channel <NUM> was generated by a light source emitting a long wavelength, the dispersion of the dispersion optics <NUM> and <NUM> may be increased to cover the sensor <NUM> uniformly. Most fluorophores emit light at wavelengths between approximately <NUM> and approximately <NUM>. If the excitation wavelength is <NUM>, fluorescence may be generated from <NUM> to <NUM> from the different fluorophores. On the other hand, if the excitation wavelength is <NUM>, fluorescence may be generated from <NUM> to <NUM>. Because this is a much smaller wavelength range, the dispersion may be increased to spread the spectrum over the same number of pixels as the wider wavelength range.

The reflectors of the first reflective layer <NUM> may be configured to reflect channels from different portions of the fluorescence light <NUM> in different directions. For example, light from a second channel <NUM> of the fluorescence light <NUM> may be incident on a subset of the reflectors, and reflected toward a second reflective layer <NUM>. Reflectors within the second reflective layer <NUM> may flatten the wavefront, due to the pitch and angle of the reflectors, and reflect the light from the second channel <NUM> toward a lens <NUM>. The light from the second channel <NUM> may then be collimated by the lens <NUM>, dispersed by dispersion optics <NUM> and <NUM>, and imaged by a lens <NUM> onto a sensor <NUM>. For example, the dispersion optics <NUM> and <NUM> may be a set of double Amici prisms. As discussed above, the dispersion optics <NUM> and <NUM> may be adjusted to provide more or less dispersion, such that lines of the second channel <NUM> are dispersed across the entire sensor <NUM>. For example, if the second channel <NUM> was generated by a light source emitting a long wavelength, the dispersion of the dispersion optics <NUM> and <NUM> may be increased to cover the sensor <NUM> uniformly.

Although only one column of optics including the second reflective layer <NUM> is shown in <FIG>, it should be understood that there is an additional column of optics for each of the remaining channels of the fluorescence light <NUM>. In particular, there is an additional column of optics including another set of the second reflective layer <NUM>, the lens <NUM>, the dispersion optics <NUM> and <NUM>, the lens <NUM>, and the sensor <NUM> for each of the third channel <NUM>, the fourth channel <NUM>, and the fifth channel <NUM>. Depending on the numerical aperture of the fluorescence light <NUM>, a staggered array of lenslet pairs may be provided between the first reflective layer <NUM> and the second reflective layer <NUM> in order to relay the lines or the spots from the first reflective layer <NUM> to the second reflective layer <NUM>. Further, a relay and an array of apertures may be provided between the second reflective layer <NUM> and the lens <NUM> to provide confocality. Additional filters, such as long-pass filters and/or notch filters, may be provided before the first reflective layer <NUM>, such as between the objective <NUM> and the tube lens <NUM>, to condition the collimated fluorescence light <NUM>.

For illumination schemes <NUM> and <NUM>, among others, where different stripes focus at different depths within the sample <NUM>, the fluorescence light <NUM> focused by the tube lens <NUM> will focus at different depths at the image plane. Thus a single reflective layer <NUM> cannot be in-focus for all stripes at the same time, as needed for confocal pinholing. As discussed above, it is possible to include additional optics and pinholes within the channel separator <NUM> shown in <FIG>. Another possible solution would be to add several plates at different planes within the channel separator <NUM>. Each plate would have a sparse set of reflective facets that match up with different illumination stripes. As yet another alternative, a corrector plate could be inserted within the channel separator <NUM> before the image plane. <FIG> show examples of corrector plates. <FIG> is a lenslet array in which the curvature is exaggerated for clarity. <FIG> is a spacer array that is thicker than the lenslet array shown in <FIG>.

To avoid having signals pass through the wrong corrector plate facet, the distance z from the corrector plate to the image plane must be less than M*dx*f#, where M is the magnification due to the objective <NUM> and the tube lens <NUM>, dx is the excitation stripe spacing at the sample <NUM>, and f# is the focal ratio at the image plane. Using f#=M/(<NUM>*NA), where NA is the numerical aperture of the objective <NUM>, the distance z can be expressed as z<M<NUM>dx/(<NUM>*NA). At the same time, it is undesirable for the downstream optics if the corrector plate changes the focal ratio significantly at the image plane. Thus the distance z may be chosen to meet z>>M<NUM>dz, where dz is the focal shift between different excitation stripes at the sample <NUM>. Thus, for this scheme to be possible, the excitation stripe spacing dx may be chosen to meet dx >> dz*(<NUM>*NA). For example, dz may be <NUM> and dx may be <NUM> at a NA of <NUM>. The curvature of the corrector plate facets of the corrector plate shown in <FIG> or the thickness of the corrector plate shown in <FIG> may be designed to achieve a particular focal shift dz. This corrector plate could also be inserted into the excitation path, along with a uniform lenslet array, to generate multiple focal depths. <FIG> shows an example of the focal planes if the focus of the fluorescence light <NUM> collected from different depths is not corrected, while <FIG> shows an example of the focal planes if the focus of the fluorescence light <NUM> collected from different depths is corrected with a corrector plate <NUM>.

<FIG> shows a diagram of another example of a channel separator that separates the fluorescence light <NUM> into a plurality of spatially dispersed spectral channels. As shown in <FIG>, the channel separator <NUM> includes a lenslet array <NUM> that concentrates each stripe of the fluorescence light <NUM>, a lens <NUM> that re-collimates the fluorescence light <NUM>, a prism <NUM> that disperses the fluorescence light <NUM>, and a lens <NUM> that images the fluorescence light <NUM> onto a sensor <NUM> as a series of spectral stripes. The sample <NUM> may be scanned to collect data from the entire sample <NUM>. The sensor <NUM> may be a large-area sensor or a mosaic of smaller sensors that are optimized for rapid electronic readout.

<FIG> shows an example of an illumination pattern that may be formed on the sample <NUM>. The illumination pattern shown in <FIG> is similar to illumination pattern <NUM> shown in <FIG>, but includes stripes corresponding to excitation light having five wavelengths λ<NUM>-λ<NUM>. The fluorescence light <NUM> that is generated by this illumination pattern has a similar structure of stripes and is imaged onto the lenslet array <NUM> with a large magnification.

<FIG> shows an example of a layout of the lenslet array <NUM>. The lenslet array <NUM> includes a plurality of linear arrays of lenslets that are configured to receive the fluorescence light <NUM> that is generated by the illumination pattern shown in <FIG>. Each of the stripes of the fluorescence light <NUM> is imaged onto a respective one of the linear arrays of lenslets within the lenslet array <NUM>. The regions between the lenslets may be masked to provide confocality and improve contrast. The linear arrays may be offset vertically relative to one another by a fraction of the lenslet pitch in order to minimize overlap between spectra on the sensor. The lenslets may have circular, square, or any other shape of aperture.

<FIG> shows an example of an intensity pattern that is generated by the lenslet array <NUM> shown in <FIG>. This intensity pattern is formed at the focal plane of the lenslet array <NUM> when the stripes of the fluorescence light <NUM> are imaged onto the linear arrays of the lenslet array <NUM>.

<FIG> shows an example of the spectral stripes that are formed on the sensor <NUM>. The spectral stripes are formed by the lens <NUM> that re-collimates the fluorescence light <NUM>, the prism that disperses the fluorescence light <NUM>, and the lens <NUM> that images the fluorescence light <NUM> onto the sensor <NUM>. The vertical stagger within each set of horizontal stripes shown in <FIG> corresponds to the vertical stagger between the linear arrays of lenslets within the lenslet array <NUM> shown in <FIG>.

<FIG> shows an example of another illumination pattern that may be formed on the sample <NUM>. The illumination pattern shown in <FIG> is similar to illumination pattern <NUM> shown in <FIG>. The fluorescence light <NUM> that is generated by this illumination pattern has a similar structure of circular spots and is imaged onto the lenslet array <NUM> with a large magnification.

<FIG> shows another example of a layout of the lenslet array <NUM>. The lenslet array <NUM> includes an array of lenslets that are configured to receive the fluorescence light <NUM> that is generated by the illumination pattern shown in <FIG>. Each of the circular spots of the fluorescence light <NUM> is imaged onto a respective one of the circular lenslets within the lenslet array <NUM>. The regions between the lenslets may be masked to provide confocality and improve contrast.

<FIG> shows an example of an intensity pattern that is generated by the lenslet array <NUM> shown in <FIG>. This intensity pattern is formed at the focal plane of the lenslet array <NUM> when the circles of the fluorescence light <NUM> are imaged onto the linear arrays of the lenslet array <NUM>.

<FIG> shows an example of the spectral stripes that are formed on the sensor <NUM>. The spectral stripes are formed by the lens <NUM> that re-collimates the fluorescence light <NUM>, the prism that disperses the fluorescence light <NUM>, and the lens <NUM> that images the fluorescence light <NUM> onto the sensor <NUM>.

<FIG> shows an example of another illumination pattern that may be formed on the sample <NUM>. The illumination pattern shown in <FIG> is similar to illumination pattern <NUM> shown in <FIG>. The fluorescence light <NUM> that is generated by this illumination pattern has a similar structure of circular spots and is imaged onto a pinhole array that replaces the lenslet array <NUM> with a large magnification.

<FIG> shows an example of the pinhole array that replaces the lenslet array <NUM>. The pinhole array includes an array of pinholes that are configured to receive the fluorescence light <NUM> that is generated by the illumination pattern shown in <FIG>. Each of the circular spots of the fluorescence light <NUM> is imaged onto a respective one of the pinholes within the pinhole array.

<FIG> shows an example of an intensity pattern that is generated by the pinhole array shown in <FIG>. This intensity pattern is formed at the focal plane of the pinhole array when the circular spots of the fluorescence light <NUM> are imaged onto the pinhole array.

<FIG> shows an example of the spectral stripes that are formed on the sensor <NUM>. The spectral stripes are formed by the lens <NUM> that re-collimates the fluorescence light <NUM>, the prism that disperses the fluorescence light <NUM>, and the lens <NUM> that images the fluorescence light <NUM> onto the sensor <NUM>.

<FIG> shows a diagram of another example of a channel separator that separates the fluorescence light <NUM> into a plurality of spatially dispersed spectral channels. As shown in <FIG>, the channel separator <NUM> includes a prism array <NUM> and a diffraction grating <NUM> that disperse the fluorescence light <NUM>, and a lenslet array <NUM> that concentrates each stripe of the fluorescence light <NUM> onto a sensor <NUM> as a series of spectral stripes. The lenslet array <NUM> may have the same structure as the lenslet array shown in <FIG>. The prism array <NUM> may have facet angles chosen to cancel the first-order diffraction angle of the grating for a specified design wavelength. This allows the dispersed light to enter the lenslet array <NUM> nearer to normal incidence, thereby reducing astigmatism, and in some cases improving the performance of the sensor <NUM> as well. The pitch (period) of the prism array <NUM> may be chosen to match the pitch of the lenslet array <NUM> for improved focusing. The pitch of the lenslet array <NUM> may also be chosen to be an integer multiple of the pitch of the diffraction grating <NUM> for improved uniformity of the spectra generated at the sensor <NUM>. The sample <NUM> may be scanned to collect data from the entire sample. The sensor <NUM> may be a large-area sensor or a mosaic of smaller sensors that are optimized for rapid electronic readout.

In addition to the dimensions discussed above, the system <NUM> may be used to acquire data in a sixth dimension of time. The system <NUM> may acquire hyperspectral data at different sample locations as discussed above, and then monitor those locations as a function of time. For example, this could be used to unmix fluorophores that have different bleaching rates. It could also be used in conjunction with fast sensors to determine the fluorescence lifetime.

Exemplary embodiments of the invention may provide several advantages. For example, it may be possible to maximize collection of the fluorescence photons, which would minimize bleaching and maximize the signal-to-noise ratio to improve spectral unmixing. In addition, the dispersion of each fluorescence channel may be adjusted to maximize the use of the pixels in the sensor. Further, multiple excitation channels may be simultaneously acquired, such that multiple depths may be scanned and/or multiple excitation wavelengths may be used.

However, it is understood that the embodiments can be practiced without these specific details. For example, circuits can be shown in block diagrams in order not to obscure the embodiments in unnecessary detail. In other instances, well-known circuits, processes, algorithms, structures, and techniques can be shown without unnecessary detail in order to avoid obscuring the embodiments.

Implementation of the techniques, blocks, steps and means described above can be done in various ways. For example, these techniques, blocks, steps and means can be implemented in hardware, software, or a combination thereof. For a hardware implementation, the processing units can be implemented within one or more application specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), processors, controllers, micro-controllers, microprocessors, other electronic units designed to perform the functions described above, and/or a combination thereof.

Also, it is noted that the embodiments can be described as a process which is depicted as a flowchart, a flow diagram, a data flow diagram, a structure diagram, or a block diagram. Although a flowchart can describe the operations as a sequential process, many of the operations can be performed in parallel or concurrently. In addition, the order of the operations can be rearranged. A process is terminated when its operations are completed, but could have additional steps not included in the figure. A process can correspond to a method, a function, a procedure, a subroutine, a subprogram, etc. When a process corresponds to a function, its termination corresponds to a return of the function to the calling function or the main function.

Furthermore, embodiments can be implemented by hardware, software, scripting languages, firmware, middleware, microcode, hardware description languages, and/or any combination thereof. When implemented in software, firmware, middleware, scripting language, and/or microcode, the program code or code segments to perform the necessary tasks can be stored in a machine readable medium such as a storage medium. A code segment or machine-executable instruction can represent a procedure, a function, a subprogram, a program, a routine, a subroutine, a module, a software package, a script, a class, or any combination of instructions, data structures, and/or program statements. A code segment can be coupled to another code segment or a hardware circuit by passing and/or receiving information, data, arguments, parameters, and/or memory contents. Information, arguments, parameters, data, etc. can be passed, forwarded, or transmitted via any suitable means including memory sharing, message passing, ticket passing, network transmission, etc..

For a firmware and/or software implementation, the methodologies can be implemented with modules (e.g., procedures, functions, and so on) that perform the functions described herein. Any machine-readable medium tangibly embodying instructions can be used in implementing the methodologies described herein. For example, software codes can be stored in a memory. Memory can be implemented within the processor or external to the processor. As used herein the term "memory" refers to any type of long term, short term, volatile, nonvolatile, or other storage medium and is not to be limited to any particular type of memory or number of memories, or type of media upon which memory is stored.

Moreover, as disclosed herein, the term "storage medium" can represent one or more memories for storing data, including read only memory (ROM), random access memory (RAM), magnetic RAM, core memory, magnetic disk storage mediums, optical storage mediums, flash memory devices and/or other machine readable mediums for storing information. The term "machine-readable medium" includes but is not limited to portable or fixed storage devices, optical storage devices, wireless channels, and/or various other storage mediums capable of storing that contain or carry instruction(s) and/or data.

Claim 1:
A system (<NUM>) comprising:
an excitation light source (<NUM>, <NUM>, <NUM>, <NUM>) that is configured to emit excitation light (<NUM>);
an objective (<NUM>) that is configured to receive the excitation light from the excitation light source and image the excitation light onto the sample (<NUM>), such that the excitation light causes the sample to emit fluorescence light (<NUM>);
a channel separator (<NUM>, <NUM>, <NUM>, <NUM>) that is configured to receive the fluorescence light from the sample and separate the fluorescence light into a plurality of spatially dispersed spectral channels; and
a sensor (<NUM>) that is configured to receive the plurality of spatially dispersed spectral channels from the channel separator,
wherein the excitation light source comprises:
a light source (<NUM>); and
a plurality of first lenslet arrays (<NUM>, <NUM>, <NUM>, <NUM>), wherein each of the plurality of first lenslet arrays is configured to receive light from the light source and to generate a respective pattern of light;
wherein the excitation light source is configured to combine the respective patterns of light generated by the plurality of first lenslet arrays to form the excitation light; and
wherein the plurality of first lenslet arrays is arranged such that light passing through each first lenslet array is focused at a different depth of the sample, such that when the objective receives the excitation light from the excitation light source, the objective simultaneously images each of the patterns of light to form a plurality of parallel lines at different depths of the sample or an array of circular spots at different depths of the sample and wherein each of the spatially dispersed spectral channels includes a portion of the fluorescence light generated at a single depth of the sample.